My Shield 40 came with rear sight the same way. Shot way left but also high, indicating front sight was too short as well. Plus, it had the 40 mag drop issue immediately. Also, the trigger was nowhere close to the rated 6.5lbs.
So back it went, and after S&W having it for only 4 days, they sent it back in 100% perfect expected condition. They moved rear sight to center, replaced front sight with taller, corrected mag drop issue, and did a factory trigger job bringing trigger pull down to advertised range. Now I can easily hit golf balls every time at 15 yards standing offhand (using cast lead sized to barrel). After adding Lasermax, golfball hitting distance increased to 25 yards.
While gun was being repaired, I had already ordered an Apex Shield Carry kit, so ended up installing that anyway with not much of a noticeable improvement over the S&W free trigger job. But it showed me that the rear sight can indeed by adjusted left or right. It is not a fixed rear sight. You will probably cause some damage to it when you beat it out though, usually just cosmetic. I later replaced front sight with Dawson Precision fiber optic, and the front sight was nearly as hard to remove as the rear. Not sure if any of that matters though since unlike most users, both my factory sights were moved by S&W under warranty, which might have made it easier or harder for me to later do it.