I had a similar experience when I first started shooting my 4006 -- it hit significantly low and left. At least part of the problem stemmed from the fact that my grip left too little finger contact on the trigger, and all of my hand-muscle motions acting together were pushing me off POA. Adjusting the axis of the gun in relation to the long bones in the palm of my hand helped quite a bit, but there is still some improvement to be made.
I would agree that smoothing out a gritty trigger could help the situation.
I don't shoot semiautos a lot, as I prefer revolvers. But my experience has been that any new auto requires a slightly different grip from what I have used before on other models. That is a much less severe problem with revolvers, for some reason.