How drastic a difference is there going from .230 to .200 or even .185? Or is this a "depends on the pistol/revolver/user" kind of thing?

I'm just been conditioned to shooting .230s because that's what I've always had/purchased, just because.

I may even have an old box of Winchester Silvertip Hollowpoints in .185 somewhere. I've just kept them on hand for defense, but never really shot them.