Originally posted by Pisgah:
Almost guaranteed, it is your trigger technique plus a small flinch in anticipation of recoil.
Dryfiring is your diagnostic friend and cure. With an empty gun, pick a clear target, take careful aim, and pull the trigger just as if you were firing a shot. Keep your eye firmly welded on the front sight. Likely you will see it move off during the trigger pull/hammer fall. Now, check your trigger finger. Are you pulling the trigger with the tip, the pad, or the first joint? Whichever it is, try moving things around a bit. With most guns I get best results with the trigger firmly centered in the pad of the finger, but for you it may be one of the other two options.
Once you see your sight staying centered, continue to dryfire practice, always concentrating on the front sight and keeping it steady. The next time you go to live fire, you will see a difference.