There is no real reason you have to start loading on a single stage press. If you are going to eventually want a progressive, by all means get one to start with. If you are going to be concentrating on one pistol caliber, I suggest the Dillon Square Deal, or perhaps the RL550B. The Square Deal automatically indexes, which the 550 does not. The Square Deal uses propriatary dies, the 550 uses the same 7/8" dies most other presses use. The Square Deal is not adaptable to rifle calibers, the 550 is. Dillon does not make the only progressive presses, but they make some of the best. The Lee progressive is much cheaper, but apparently takes some fiddling to keep it running. The Lee Turret is auto indexing, but you still have to pull the handle 4 times to get one loaded cartridge. Some owners claim it to be a "progressive" press due to the auto index feature, but I would disagree. Also the powder metering system is a bit more aggrivating to my way of thinking.
If you want to start out really inexpensively, Lee makes a no frills C press that sells for around $25, add a set of Lee dies for another 25, and a priming tool. That plus your supplies, such as bullets, powder and brass, would get you started. When you graduate to something else, the little single stage press could be usefull for bullet sizing, decapping, etc. or if you were loading a few rifle rounds for hunting season, it would likely suffice.
Hornady also makes a progressive which gets good reviews, as do RCBS and Lyman. The RCBS and Lyman progressives don't have much of a following, but their single stage presses set the standard.
Take a look at eBay. There are lots of presses available, and many come with a lot of accessories that you would eventually want to add to a basic press.
