Thank you all for your shared experiences!
Chief 38 I gave that a try, and here's what i discovered at least on my safety:
The spring and retaining screw have very easy access on the outside of the frame just by removing the grip. I quickly realized my detent was so low already, the spring is actually riding on the safety lever surface outside of the detent milled curve for about 180 degrees in addition to contacting the detent in the center of the curve (in my case). And since the 1/2 round shaped curve of the rear end of the spring is too large for the milled out curve of the detent in the safety lever, it drags.
Loosening the screw solved the problem, and is as far as one needs to go since the grip captures the screw and if it works itself out, it can't be lost. Or leave out the screw and spring. My safety has enough friction that it doesn't need the spring because the lever's pin also serves as the hammer pivot and a snug fit especially with the hammer spring tension on the hammer.
However for a more professional cure I will grind off about 1/32" of the springs rear semi circle and it will fit the milled curve on the safety lever w/o rubbing. Id would work as designed with the screw fully tightened but with contact only on the detent.
The screw hole in the spring is actually elongated horizontally from the factory (bluing is intact) for some adjustment so when pushed to the front as far as possible and tightened with the screw, it relieves a little tension but not enough in my case.