Maybe the front locking lug is not to size in the frame and it wobbling in the frame as you press on it. Perhaps the cylinder stop has slop where it fits in the frame or there is slop in the cylinder notches. Maybe the end of the ejector rod doesn't fit the forward locking lug deep enough.
Something like this is a process of elimination until you find the exact cause. If the gun seems ok with range rods as you hold down the trigger, then I'd not worry about it. Very few S&Ws are "vault like" tight. I have in my personal collection a 66-2 that is extremely tight when you hold down the trigger, yet it doesn't bind. The cylinder doesn't move in any direction. This is a rarity to say the least!