I've got thee or four with the lock now. I've owned others in the past. It worries me about as much as the chance of being hit by a steamship in the parking lot this morning. I keep a key on my keyring, right next to a P-38 can opener. I've never used either one. I don't even think they're any uglier than any other screw, or pin in the side of the gun. They've been there so long, I just expect them to be. As an aside...There are old S&W's that have a pin in just about the same place.
On the other hand if you don't like it, take it out.