Pretty much anything that isn't original to the firearm will detract from the value, even if it makes absolutely no sense whatsoever like a harmless, easily removed/replaced aftermarket addition will be used as an excuse for lowball bids on otherwise "collectible" firearms.
"Those grip screws aren't original to the firearm so it's now worth 25% less."
"Uh-oh! The original owner put red nail polish on the front sight, so now it's only worth whatever I have in my back pocket. Fortunately, you're in luck as I happen to a Quarter, a Paperclip, a Rubber Band, and a Coupon for a Free Small Milkshake at Arby's with purchase of a regular priced Roast Beef Sandwich! You're not going to get a better offer than that!"
"What?! You had it Factory Restored?! Well, I'm afraid that reduces its collector's value by 75%, so you should have left it in its original tarnished, corroded, eyesore condition, as then it would have been worth more. Nobody would have bought it anyway in such poor condition, but it would have been arbitrarily worth more in its original undesirable condition simply because it was original."
"Egad! That cleaning solution you used isn't original to the firearm! You have to lubricate them with whale oil like they did back in the day! It's completely useless now!"
Heck, I sincerely believe that if you put an original 1873 Colt Single Action Army with grips autographed by John Wayne on an auction site, you'd get at least a few messages from folks informing you that the autographed grips have effectively destroyed its collector's value and now its worth no more than a Heritage Arms Rough Rider .22.
Collectors tend to be obsessive when it comes to originality, even when it comes to otherwise unobtainable pieces like these imported Israeli S&W 3rd Gen, they'd rather go without or spend an absurd amount of money in the pursuit of an unmarked example than compromise, which often times is a poor decision as the guns they snub inevitably become sought after collectibles once the import markings become an accepted part of their history.
So put it this way, they will most likely become collectible someday, but not likely within your lifetime, so don't buy one if you want to flip it for a profit.