Seem like an easy fix...don't hold it by the magazine. Get a vertical grip and locate it wherever feels comfortable. You can also get a shorter 10-round mag for when you are bench shooting.

If you are pulling the magazine back to keep the rifle firmly against your shoulder (which you probably are) I could see this happening. I'd be weary of doing this as it could cause malfunctions and possible damage. The ejector is a sensitive part. How much lower are we talking about when this happens?