This only covers the 4506 magazines with the plastic followers, I do not have info on the all metal 645 magazines which are compatible with the 4506.
The first incarnation of the 4506 magazine was the style with the orange followers, these magazines sometimes had feeding issues with the round at the top getting vibrated into a bad position with the nose of the round pointing up and causing a failure to feed. My orange magazine has done this from time to time and is now retired.
S&W modified the magazines with an indent near the top rear of the feed lips to help hold the cartridge in place and keep it from tipping nose up, these were adorned with yellow followers that had a cut out to slide past the indent. The magazines were marked to use only yellow followers as the orange ones bind at the cartridge retaining indent. I have heard that these magazines would sometimes deform at the feed lips causing failures to feed. My yellow magazine has never had any issues.
Sometime later S&W again redesigned the magazines with large gusset with multiple bends to give the feed lips added strength due to reports of deformation and feeding failures, these magazines use the black follower. The black followers have a large cutout on the sides to accommodate the gusset in the feedlips. Neither the orange or yellow followers work in these magazines, but the black followers are 100% backwards compatible with the older style magazine bodies.