Initially it started taking the gyroscopes out of Nintendo Wii controllers and lots of soldering. But eventually a whole industry grew around it.
So yes you can build your own or buy whole kits. getfpv.com is where my parts came from, its a company based in Florida.
They offer you RTF / ready to fly kits, so you will receive a quadcopter ready to go.
I am using a flight controller from openpilot.com, a open source community based project with an awesome support forum and great software to change settings. Other controllers like MultiWii are less entry friendly
The initial cost really depend on how much you want to do on your own. Remotes can get really expensive unless you take the challenge and buy a Turnigy 9x which is a cheap one, but it comes with many bugs. The community took the challenge, hacked the remote and wrote new software for it. So if you do those steps you will have a remote for $80 with the functionality of a remote in the price range of $1000.
Besides the copter you will require batteries which range about $10-$70 depending on size and model, charging station will run you around $50.
And then of course you will need propellers, many of them
the CC3D controller from opencopter offers you basic stabilization, so if you release the controls the copter goes into a hover but won't hold position.
Now for $30 more you can get their Revolution controller which is able to handle altitude and GPS holds. With their software you could build a drone which will follow waypoints or you can use it as a safety net if you lose control. You can always tell the quadcopter to return home, hold position etc.
Other possibilites are FPV, so the pilot wears goggles and flies the copter like he is sitting in it.
https://vimeo.com/61994834
