No, absolutely not. You will have to decide what you are comfortable with, but when I got started carrying, it was a 4" 586 IWB year round. Then I moved up to a 6" 686. Eventually I started carrying a 5" full underlug 629 (read 3 lb. gun) IWB. Keep in mind that as the gun gets bigger, your set up (belt, clothing holster, ect.) gets more and more critical. I'm 5'7" 200 lbs. and lately I've been carrying the 629 to church under untucked button down shirts with a busy print and a full power .357 in the weakside front pocket. As Evan Marshall says, "You can carry whatever you want, just dress around the gun". I say carry the guns that you would want with you if the SHTF.
Oh, and the holster I prefer is a FIST 1A IWB kydex/leather hybrid with a body shield.