Peter,
I don't have the collection of loading manuals you have, but I'll try to keep up with you. I'll address your post topic by topic. I may sound a little erratic since I'm afraid of taking too long with the post and having it thrown out, which has already happened once. Even though windows 8 might have something to do with that since it wasn't that long.
I don't know if there was a velocity war back then, at least not like there is now. It's difficult to compare bullet data from one manufacturer to another because they make their bullets for certain reasons and design them to operate at different velocities. Speer 8 mentions they only tested the loads for velocity over a chronograph, however some of those loads were obviously developed earlier using pressure test equipment, and re-chronographed for an apples to apples comparison. But the reason Speer seemed hotter overall might have been due to the fact that they had to be to achieve optimum performance. Like I said before, not all of their data is overly excessive. It's also good to keep in mind that while bullets may look the same over the years, it's construction might have changed several times.
I quit reading gun/ammo magazines a long time ago. They are the ones that create the markets and usually it's for the wrong reason. They like to drill the aspect of "faster is better" into the readers minds, but that isn't usually the case. Many loading sources cut their data off if the load becomes so inaccurate it is basically useless.
I couldn't find the load that you referenced in Speer 6. There is a lot of talk on the net that powders have changed over the years due to velocities going up or charges going down. It's not so much the powders as it is the primers, they have been improved to give hotter and longer flames to ignite all the designer powders we have now. Primers make a lot of difference in a load. Also there is the normal lot to lot changes of powders and primers that will make a difference in load data, along with a long list of other reasons. But take a look at Lyman manuals and you will often see the exact same load posted for many years. It is obvious if you see identical velocities and pressures for a particular load as that just doesn't happen. They know that there will be little changes and re-testing loads for every edition is just going to confuse everyone.
Also as to your question of Speer 6 to modern manuals and what changed, first there is the SAAMI which did reduce the max. limits for .38 spls. in the early 70's. Also Speer no longer lists pressure data and they explain why in their newer manuals, which may have a lot to do with it.
I heard, and I don't know how true it is, that the CIP has different PSI limits because of the way they test their pressures, much like the difference between our commercial and military ammo.
1) Speer 8 was a product of the times and should be considered as such
2) SAAMI lowered the pressure level for the 38 special and 357 magnum around the same time Speer 8 came out (causality?) and we have dealt with lower pressure limits since.
3) One might be able to load to the 38/44 power levels with suitable powders and based upon the evidence presented in the early 70's but different manufacturers 4756 is a reasonable choice to do so.
Well, I can't really argue with most of this. I don't think no. 8 is the jewel many seem to think it is, actually it's more of a liability. I also don't feel that SR4756 is a good choice for the .38 spl/.38 spl +P/.38 heavy duty/.357 magnum family, but that is a personal choice. From what I'd read 2400 was the powder first used to make .38 heavy duty ammo and it appears to have been a good choice. But I don't own one so I don't worry about it all that much. I have nothing against anyone that shoots max. loads as long as they know how to do it. Today we have too many people that thinks the only way to go is to find the highest velocity and load it without working up to it.