I've never tried any of the factory "hot" loads for any of my firearms, and in the last 15 years I might have purchased 5 boxes of any type factory ammo, not counting rimfire stuff.
Loading your own gives you the luxury of making it as hot or as mild as you want. My experience is, the really hot stuff is hard on the equipment and any improvement in performance over "standard" loads is incremental and not worth it. The human factor aside, I find that accuracy suffers when you get in the upper ranges of hot loads. Adding in the human factor, accuracy gets worse. My theory is, if you want more power, move up in caliber, and even then be moderate in how you load.
I've always liked having many different calibers to shoot, and I load for ten different handgun and three rifle calibers. I tend to keep all my loads in the mid-range of available load data. I did work on my two long range rifle loads to find the sweet spot in velocity and accuracy.