I'll be the first to say it. If there is an actual active rust patch as you say that is that hard to remove, there is probably some light peppery pitting already there. Even after killing the active (red) rust and / or removing it, chances are that the surface will remain "frosted" or somewhat dull. If it shoots OK and doesn't lead excessively don't worry about it.
If it bothers you get some abrasive paste as others have suggested. You can use J & B, Flitz, Autosol Chrome polish, put it on a snug fitting patch and polish it up.

__________________
Sceva
OGCA SWCA NRA