I too was very leary of messing with the rear sight. However for 3 of my 4 M&Ps, it ended up very, very easy. All of my set screws easily loosened with no heat needed. Then they were very easy to drive out with a nylon punch. I would seriously recommend a nylon punch versus any sort of metal punch. I think I got 3 nylon punches from Brownells for something like $5. I didn't even have to put these in a vise - just rested them and held them on a roll of masking tape used as a block. These all were returned to zero with no markings or damage whatsoever to the slide or sight.
This was not the case on my Shield however. The set screw loosened easily enough but the sight just wouldn't budge. No damage but beyond my comfort zone. I think once I get the chance, I will send it to James at xd-xs2000.
Welcome
I've never used him before but he comes highly recommended on several of the other forums. I personally wouldn't touch a front sight myself so I plan to get him to install a fiber optic front sight. Hope this helps.