Wearing a t-shirt underneath is the common approach for this. I also sometimes smooth the grip out with something like crocus cloth. It's like a really fine sandpaper. You could even use burlap, denim, etc. It's not really sanding large amount of material off, just knocking off the pointy bits.