Even the experts are divided on this one.
I own Two 9mms, a .40 caliber, and a .45 acp and I shoot them all.
People will tell you that 9mm is a sissy round, and that feds and soldiers have moved away from them, which is true. Feds needed a round that could get through windshields and keep integrity better than the 9mm, hence the .40 caliber was developed.
For civilian CCW use there is great 9mm ammunition available and will guard you well. I have a Sig-Sauer P226 in 9mm, and straight out of the box, the gun is a champion. Recoil is extremely manageable, and it shoots great groups. 15, 18 and 20 round magazines are available. You can also put 200 rounds through it at the range, and not even think about it.
The .40 cal S&W is probably the best all-around defense round in terms of a combination of performance and capacity. I have a Sig Sauer P229 and with a high capacity magazine you can have very similar performance and energy to a .357 magnum with 12+1 capacity. A Glock and some other Autos will give you even more capacity.
I have a Kimber 5" 1911 in .45acp, all steel. Very Nice Gun, but it is a lot of gun. The .45 acp is a great defense round, but you will sacrifice capacity.
Bear in mind, in a smaller gun, both the .40 and the .45 will have a lot of recoil. Even my all-steel Kimber is much snappier than my 226. I'll put 50 rounds through it, and I am done.
In the end, I would suggest that you shoot some of your friends guns and see what you like.
Another option? Get a .357 Wheel gun that you can carry .357, but practice mainly with .38spcl. This way you can shoot and carry. There seems to be dozens of options just from Smith and Wesson on .357 Wheel guns.
An old School .357 3 " Smith will be my next purchase.
One other piece of advice, if you will indulge me:
Find a good local dealer and let them know what you are looking to do, ask them for a high quality USED gun as a starting point, and see if you like it. Also ask about their exchange policy, in case it is not quite what you had in mind.