Geeze Louise, that's a tough one! I can see why you're looking for advice.
If it were me, I'd go straight to the 45. Reasons stated above point directly to the reasons. Bigger loads moved more efficiently and ultimately bigger loads due to dimensions. Something in a 275g or 300g bullet for instance.
Moderate velocity with big chunks of lead are effective and in this case a 4" sidearm for packing in the woods screams .45 Colt, at least to me.
This comes from a guy that's a dyed in the wool .44 guy. I have... many 44's in both magnum and special.
If you want something accurate and hard hitting without the need for velocity much over 1000fps so it shoots easy, the Colt is your huckleberry.
That said, why not go on a mac and cheese diet for a while and get both?