XM designation does not necessarily indicate failed or rejected ammo. This is a pervasive myth. Sometimes federal ends up making more than is needed because a contract ends, and sometimes a lot might "fail" to meet contract specifications (for example, having an average velocity of 854fps instead of 880fps) which technically "fails" but is still perfectly serviceable, as individual guns will show as much variation in velocity. Even if a lot does technically "fail," it is still absolutely safe ammo that meets SAAMI specifications--the liability of selling out of spec ammo would be astronomical.
Some XM ammo, like the popular XM193 or XM855 most likely is made on the same equipment as military contract ammo.
In any case, the sheer volume of XM ammo available for purchase means it simply cannot all be rejected ammo.
I would feel perfectly at ease stoking my self defense guns with XM ammo, provided it proves reliable in my gun after at least 100 rounds with no malfunctions.
It should also be noted, that the ammo at the center of this discussion, 38G, does not have an XM designation.