Most likely a problem with the ejector depressor plunger and/or spring in the slide, directly above the ejector mag depressor lever. These plungers are plastic and can get chewed up, deformed or worn down over time. Another remote possibility is that the ejector lever is bent or damaged.
Check to see if the plunger is in-tact and plunges up and down properly/fully. The plunger pushes the ejector lever down when the magazine is removed, interrupting the drawbar from catching the DA hooks on the hammer if and when the trigger is pulled.
Replacement involves removal of the rear sight and installation of the new part and spring. Before replacing the rear sight, while I'm at it I usually clean and inspect the firing pin safety lever plunger and spring and it's cavity in the slide.
If it's the ejector lever, it is a drop in....no fitting required.