Personally I do not own any 44Mag's because I see no real purpose for ME to. If I lived in the woods where there were Bears all over or I was a hand gun hunter (or needed it for back-up) I would keep the .44 as it is much better than a .357 for that purpose. Strictly as a defensive hand gun the .44 Mag is a bit too much in a residential setting or populated area.
Other than that, if you are not a Hunter and do not live in Bear Country, I think the .38 Special / .357 would be of more use to you. To just go to the Range and shoot .44 Mags to wake up the dead is fun for a few minutes, but I'd rather have the versatility of the .357. You can use it to shoot .38's all day long for not a lot of money and you don't get beat up. I suppose you could shoot .44 Specials from the .44 Mag also so at the end of the day it's YOUR call. You also have to ask yourself if you want to start loading another caliber or buy .44 ammo if you do not reload. Since you already own .38/357's there is no ammo problem.
Oh and by the way, if I ever did buy a .44 Magnum it would have to be the 6" barreled version. Too much loss of velocity and energy in the 2" shorter barrel IMHO.