Not quite. Remember that sights are regulated by humans, meaning that the POA-POI relationship takes recoil into account. I'll use a .38 Special snubnose revolver as an example. Typically, they're zeroed for 158gr loads at 7 yards. That means that when you're sights are on a point at 7 yards, the bullet will strike that point.
Now consider the time it takes between primer detonation and the bullet leaving the muzzle. The gun is already starting to recoil before the bullet leaves the muzzle, but the sights take this into account when they're zeroed.
Now take a 110gr bullet, fired from the same gun, at the same point, at the same distance. Let's also assume that the velocity is high enough that it generates the same muzzle energy as the previous 158gr load. To do so, the bullet has to travel at a higher velocity than the 158gr load. This means that it spends less time moving through the barrel, which in turn means that by the time the light bullet exits the muzzle, the gun has moved less vertical distance through the arc of recoil.
The result? The lighter bullet hits the target lower than the heavier bullet, all else being equal.
Clear as mud?