You will definitely notice the muzzle heavy aspect with the 5-inch gun more than the 4-inch, unless you have arms like a gorilla.
If you’re just banging away, it’s not so much of an issue for me, but if you’re doing precision shooting that’s another thing.