Bottom line is you have to make a decision, carry or not.
Sometimes a small firearm is all that you can carry. Firearms aren't the end all. I feel there are three things that contribute to defending oneself properly.
1. First is your brain. You need to keep situational awareness. Remove yourself from potentially bad situations before they happen. Do not frequent and or travel in high risk areas, and think before you act. Make smart quick decisions. Keep yourself alert and out of harms way. If you have to go into high crime areas, maybe a j-frame or pocket .380 is not a good weapon selection. Dress to your situation.
2. Second is your body. You need to be physically fit and train your body as well as you train with your firearm. You may need to fend off an attacker before you can draw and defend yourself. You will more than likely have to be reactive unlike law enforcement and military which is proactive. Go take a few force on force classes to get a real wakeup call about how proper grip, stance, and sight picture may not always be possible. You may actually need to use those eyes, legs, lungs, and heart to move while shooting, and even those muscles to grapple and defend as you draw and fire.
3. Finally, it comes to the defensive weapon. There are three criteria I use for me.
1. Access. Can I carry it concealed in my attire in the environment? Can I have access to it at all times or will I have to leave it locked in a building or a car? I need to have access to it at all times. I am not going to bring my 4" model 19 on me to the beach, nor am I going to bring my keltec .380 with me to the city.
2. Proficiency. Can I shoot this weapon accurately in the following situation: One handed, on the move, at human sized targets. If I cannot, it can be a 50 caliber and be worthless to me. I train with what I carry and I carry what I can train with accurately.
3. Penetration. I am not as concerned about expansion as I am about being able to make deep holes in things. Does the caliber/bullet I am carrying work reliably in my weapon and meet certain terminal ballistics requirements. Namely for me, the 12" penetration through 4 layer denim and ballistic gel. I also factor in seasons. Luckily I live in a area that goes from warm to hot to oh my god hot back to warm again. But if I lived in a cold climate, I would have to reassess my selections based upon multiple thick layers of clothing.
After I have satisfied all of those requirements, then I make my selections. So sum up a long drawn out answer for a short question, I feel that in my climate , my .380 chronoing 90 grain buffalo bore speer gold dots at 1080FPS is more than adequate in certain situations. of course not all.