I have the same problem with my 340PD...probably the same stock grips. Someone on this forum (can't remember who, or even in what thread I found it) was raving about the Hogue grips (see link below). I have wondered if these Hogue grips might work better with the speed loaders. Is anyone familiar with the compatibility of the Hogue Bantam grips (for j-frames) with speed loaders? I'm curious, too, how tough are these Hogue Bantams. I managed to tear the edge of my stock grips with my wedding ring while firing .357s off-handed (one-handed left-hand). (Yes, I know that I need to work on my off-hand firing, but, in my defense, the 340PD is a bear to shoot with .357s.)

Precision Molded Soft Rubber : Hogue Inc., The Best Value In Handgun Grips, Rifle Stocks and Accessories - Bar None!