Well, first start with some good books.
Get the Lyman cast bullet book.
It is a great reference book to start with, and is indispensable.
A good Lee bottom pour furnace (the high one is better, IMHO) to start with is a great price and starting point. RCBS has gone crazy on their pricing and I just can't see paying what they now want for their pots.

Pick up a good used pot in good condition if possible- the Lee and Saeco pot thermostats, heating elements and cords can be replaced. I know that some of the oldest Lyman pots are very hard to get thermostats for...so if looking at a used pot at a range swap meet or gunshow, be aware that it may need a new cord, thermostat or element. Also, look for zinc contamination in the pot too (this is a recent development).
Moulds...
Aluminum moulds such as Lee's are the most affordable and easiest to start working with and make great bullets- also they take less use and care than iron/steel moulds. Many folks say they don't last as long as a steel mould, but if treated with care I don't know if you really could wear one out before you wore out the gun for which you were casting the bullets!
Iron and steel moulds take a bit to get used to using- it is an art that takes a bit of time to "learn" the correct temperature of the mould to keep it either hot, or from getting too hot. Also, you need to keep iron moulds from rusting which is something that isn't an issue on aluminum moulds.
Bronze moulds are the best, but are the most expensive and are usually just custom undertakings.
It is also likely that you will need a lubrisizer, especially if you are going to cast for cartridge rifles and handguns. It is easier to start casting muzzleloading balls and bullets which rarely have to be sized just to get the knack for the casting process.
Just my two-cents.