I had one like this years ago. What got my attention is you say it gets worse after the first cylinder full is fired. Unlike you, I didn't slug mine so I can't say if my chambers were too narrow. I'm not saying my cure will work for yours and I certainly advise you to listen to more knowledgeable people here or a reputable gun smith if you can find one. I bought a chamber polishing system from Brownell's. These come with bore mops that look like little polishing balls on the ends of the bristles. You chuck them up in your drill and, using the honing oil (sold by Brownells also), you polish the chambers keeping the brush moving (up and down). I have done this to several of my guns and the polishing doesn't seem to damage the metal by removing but makes the chambers smooth. Of course, maybe I merely thoroughly cleaned the chambers by doing this... dunno.
As I said, talk to someone who knows. Could it also be the extractor star?
I'm almost as curious as you to hear what others say. Good luck