Several of us are debating this at the moment. We all agree an on target hit of #3 Buck will do the job. But what do you lose on periphery hits, extremities? Barrier penetration etc.
Periphery hits, extremities and barriers are still getting multiple buckshot hits at velocities identical to a 12 gauge. You lose a couple pellets, not much more.
Whereas #1 buck is arguably the ideal payload, and I'd rather move to 00 or 000 buck from there, #4 is generally considered sufficient in energy and penetration to do a comparable job, so #3 will, too.
Bottom line: #3 from a 20 gauge is a deadly serious defense set-up that will do the job.
Also, does the better control and follow-up outweigh the additional payload of the 12GA?
Using a 20 gauge doesn't automatically equate to better control and follow-up; depending on load, there may actually be equal to or more recoil in the lighter 20 than the 12, which being heavier can absorb more recoil.
It really comes down to weight of the weapon, how many ounces the shot, and what kind of charge it has behind it.
You can put together a 12 gauge set-up with low-recoil 00 buck that shoots lighter and gets you back on target faster than a 20 with standard 00 buck. Because of this, I actually think the 12 is the better choice, but again, you can't go wrong with 20.
(But then, I'll champion .410 bore from a full-sized shotgun for defensive purposes, too.

)