Swiped primer dimples are completely normal for a semi auto. IIRC it happens most frequently with the tilting barrel semi's due to the way the barrel drops out of engagement with the slide. If the firing pin is still forward at this point it will leave that teardrop dimple. As for damage to the firing pin, I've never seen any damage attributed to this in over 40 years of shooting. BTW, Sigs do this, Springfields do this, and S&W's do this and you'll have to look long and hard to find one single complaint of a broken firing pin.

Accuracy issues. Your description of the way your gun is throwing brass leads me to suspect 2 possibilities. One is that a 9mm recoil spring was installed in your 40 caliber. I would check to see if S&W color codes the spring for the caliber and check the spring in your pistol to insure it's correct. Second possibility is poor recoil management by the shooter. You may not want to hear it but there is a very good likelyhood that your "throwing" your shots either by flinching or not using consistent recoil management.