I'm old enough to have seen the "correct" part of the trigger digit move all over.

Where the trigger is in relation to your finger depends upon your hand size, reach to the trigger and the configuration of the grip. It can change between different firearms. What you need to do is find out where your finger needs to be to move the trigger straight back. It's rather obvious you're either pushing or losing patience and finishing off with a yank that pushes the sights off target.

Dry fire is your friend. Or, as suggested, ball and dummy. BUT with ball & dummy you need someone to watch the gun for movement. You may also be flinching (closing eyes as the shot gets close).

It's not unknown for sights to be off from the factory, but until you're positive it's the sights leave them alone. Get someone else to shoot the guns too.