I've come to the conclusion after owning several adjustable sight revolvers that few of them come from the factory zeroed exactly to point of aim. I find that I frequently need to make minor adjustments to in order to hit exactly where I'm shooting. Sometimes, the gun shoots a little low, or a little to the left or a little to the right. But, the emphasis is on the word "little." Those things can be corrected, usually with a minor sight adjustment. But, if the gun is shooting WAY low, and/or WAY left or right, that suggests two possible problems: (1) the shooter is flinching; or (2) there is an error, probably in the barrel alignment or the sight installation, that needs to be corrected.
If I were you I'd have someone who's really experienced at sighting guns shoot your gun off a rest. That ought to establish very quickly if the gun is off or whether the shooters, reacting to high power magnum rounds, are pushing the gun as they fire.
Btw, I've pretty much given up shooting high power magnum rounds single action because, try as I may, I tend to push the gun down in anticpation of recoil. For whatever reason, the problem disappears when I fire the same gun and round double action, so that's how I shoot. We have an enormous influence on how our guns perform, in other words.