The feed lips on the .40/357 magazine are slightly farther apart than on the 9mm. (Just checked that today when I finally got hold of a .40 mag.) Whether that's enough to cause a potential problem feeding, I don't know.

I'd suggest using the correct mag for the caliber if at all possible. Yeah, I KNOW they're hard to find. I'm guessing the only reason the store I found one at today had a few 40s was that they'd jacked the price way up. They were out of the 9mm. (But at least I'll be able to shoot the 40 once I finish assembling the new slide and barrel.)
============

Just did a quick check. I'd have to say DON'T try to run 9mm out of the .40 mag even with the 9mm follower. The 9mm rounds ride high and are barely retained, tilting upward at the front, which is also why the follower comes up so high with one round remaining. And if you try to put .40S&W in the 9mm mag, at least with what I have, the bullet case tends to catch on the front edge of the magazine.