The grips for the 4566 come off pretty easy, have changed the grips on my 4566TSW a couple times without any flying parts. (Always make sure gun is clear, no ammo is around and the magazine is removed) First you have to use a punch and drive the grip pin out of the bottom of the grip. I usually set the gun on a bunch up soft towel, with the grip pin hole exposed. This allows me to drive the pin through and not scratch the gun, plus the pin can land on the towel and not go rolling away. Once the pin is out, I hold the gun upside down so that the muzzle is facing me and use a wide flat screwdriver and slide it gently between the grip and the rear of the mag well. This allows the grip to be gently separated from the frame. Once the grips starts to come away from the frame, I use my hand and pull it back and up towards the hammer. All that is left is to gently pry the front of the grip around the frame to slide it off.
Hope this was somewhat helpful. I first thought the same thing about the straight backstraps, as I really loved the arched grips on my 5906. But after trying the factory arched grips on my 4566TSW, I didn't care for how long front to back it made the grip feel. I put the original straight grips on, and they felt a lot more comfortable. Just purchased some FBI Palm Swells, and the feel is a lot more close to the arched grips on my 5906.