I also like the High Standard .22 magnum for South Florida carry. Your derringer looks a lot like this inside except a plunger goes through a small metal plate located on the frame recess about 1/2" above the bottom of the grip. If the plunger is rough and/or rubbing against the plate, it can increase the trigger pull.
I replaced the factory plunger, plate and spring with a shortened 1911 spring and have a uniform trigger pull that is approximately 2 1/2 pounds less than stock.
Polishing any metal to metal contact areas is the best way that I know of to smooth out a High Standard Derringer with all stock parts.
Mark