Every load is different, whether it's loaded by you or the factory. When you find one that your pistol likes, it may be that the bullet is a little different, or the brass is sized differently, or the velocity and pressure curve is different, the list goes on of possibilities.
Generally, lighter bullets will hit lower on target compared to heavier bullets in pistols. This is due to the recoil and an effect called barrel dwell time. As for lateral drift, this one is hard to pin down. I don't see it much in pistols but occasionally a load will tend to print slightly left or right. It's seen more in rifles, because when a bullet goes down a long rifle barrel, the barrel whips. This is called barrel harmonics, not unlike the harmonics of a musical instrument. One load may release the bullet in "mid whip", while another will release the bullet at the top or bottom of a whip. Thus you will get a slight change in point of impact.