If you want to adjust it, after validating that the sights are off as recommended by folks suggesting shooting from a bench rest, the process is easy:
Borrow or buy a sight pusher -- the most common seems to be the MGW tool:
Maryland Gun Works Front & Rear Sight Tool
(Using a vise and a punch to move the sight can work but just as often leads to devastation and destruction of the sights and/or the slide, and does not allow delicate adjustments of the sight).
Loosen the set screw.
The set screw is locked in with red Loctite, which should be heated beforehand to loosen it up (e.g., the tip of a solder iron on the screw for a minute or two does nicely).
Apply some gun oil to the dovetail slot around the sight.
Use the sight tool to move the sight a small increment to the left (move a rear sight in the direction you want the bullet to strike, e.g., if you want it to hit to the left, move it to the left).
The rear sight dovetail is tapered smaller to the left; when the sight is removed all the way, it comes out to the right (towards the ejection port). It will only move so far to the left before it becomes too hard to move. You shouldn't have to go that far to make minor sight adjustments, and thus you shouldn't have to worry about the striker block and spring coming out (they are under the center of the rear sight).
Re-tighten the set screw.
Test fire from bench rest.
Loosen the set screw nut and adjust as necessary. I suspect you may need to move it a small amount, a bit less than a smidgen and a bit more than a RCH.
When done, re-tighten the set screw and apply a drop of red Loctite.
You could also use the sight pusher to move the front sight, although in that case you move it in the opposite direction from which you want the bullet to strike. Want to move to the left? Move the front sight to the right.
The sight pusher is the way to go. If you own M&Ps, it is a good investment. I've installed Warren Tactical rear sights and a Novak front fiber optic sight and WOW can I see the sights better now. But having the sight tool means I can change the sights again in the future if I want to.
When you are done with the sight tool someday, you can sell it for at least 80% of what you paid for it...