In basic terms, ANYTHING AT ALL that changes the original specifications like porting, polishing, bobbing the hammer, changing grips, replacing sights, refinish, etc. etc. will decrease the value if and when you want to sell it.
That said, it is YOUR gun and you should be able to do what you want to it to make it fit your needs. You can't always worry about loosing a few bucks down the road IF & WHEN you ever decide to sell it and you need to have it perform and look the way you want.
It's very rare for me, but I have refinished a few guns when they have gotten to the point that they bother me by being so ugly, or when more of the finish is gone than is left. If I care enough about a gun to refinish it, I am obviously not looking to sell it anytime soon and therefore don't really care about devaluing it to others - just what it does for me personally.
All I would suggest is to really think (before acting) about changing a gun to the point where it can not be reversed such as Magnaporting, or cutting metal off the gun. What you did by sanding out a few scratches and making he finish even again by polishing will technically reduce the collectors value, but like it was stated above your gun is neither rare or worth thousands of dollars so you might as well have it look good when you shoot it. Hope this helps.
