Even with the sight pusher, you have to loosen the set screw first. You will not damage the plastic spring cap, on the blocker spring under the sight, unless you excessively heat the sight. The set screws are red loctited in at the factory and this means they usually need to be heated to get them to loosen. If you turn the slide upside down, as you heat the screw, the heat rises and this will allow less heat to loosen the set screw. Use a propane torch and set the point of the flame on the set screw for 3-4 seconds. It should loosen. If not apply a little more heat.

I'm a tool and die maker and the sight pusher is a crude design. It works, but just turning the screw to push the sight, can damage the sight finish. Unless it has very close thread tolerances, the screw is also likely to wobble some. I would just use a taped up punch to drive the sight off. If you make sure you check the tape after each hit, and get no metal to metal contact, you should not cause any damage to the sight. It does take some hard hits to get the sight to start to move. It also goes back on just as hard. I tried oiling the sight to get it back on. This did not make it any easier to get it back on. After having to take it off a few times, it did get easier to remove and replace. S&W does sell replacement stock sights, just in case you slip or?

Bob