"Passivation" what is done to machined stainless steel articles. It's a pickling process involving various acid solutions, including of all things, CITRIC ACID!
The purpose of passivation is to remove microcopic points where the tooling has "contaminated" the machined surface, and to a lesser extent, provide a neutral surface on the stainless that makes it much less likely for electrolytic corrosion to start. Whether rust is visible or not, hand polishing, or buffing a previously passivated surface destroys that protection. For most applications that stainless handguns are put to, surface rust is not a problem. For severe environments (in a fishing tackle box in a boat at sea) rust can make for some horrible pitting and corrosion.
You will find that passivated finishes are always MIL-SPEC on long life expectancy or harsh environmental applications, such as aerospace, marine applications, etc.