The XD has a better trigger out of the box. The M&P has better ergos with backstraps that also adjust for width, not just depth of the grip. The grip safety is something people either don't mind or despise. I honestly don't mind it at all, so firing an XD or XDm is pretty enjoyable for me too.
It really boils down to personal preference.