I don't always carry while bicycling but when I have, I've used several different methods. How I carry depends on what I'm wearing and the type of riding I'm doing. Most of my current riding is recreational and I keep the distance to under 20 miles, both on and off road.
In cooler weather, when I'm wearing jeans (yes, I bike in jeans) and a coat or sweat shirt, I'll wear a small revolver or pistol in a belt slide holster or an OWB paddle holster. Both of those holsters are leather and have a thumb break and are easily concealed by the jacket I wear. Sometimes, when I'm feeling the need to carry a full size semi-automatic, I'll use my Vedder IWB Light Tuck which also stays well concealed with a jacket. The IWB holster tends to cause more sweating than the OWB holsters so if I'm going on a ride for "training" I usually will not use the belt holsters.
When I'm riding with clothes without a belt but still wearing a jacket, I'll use a pocket holster in the jacket pocket. When I'm not wearing a jacket, I'll use a fanny pack that has two pockets. One pocket is for the bike tools, extra tube, etc. while the other pocket is just for the pistol I've decided to carry.
I tried using my vertical shoulder rig a few times but that is too uncomfortable for me. I do not have a chest holster, but now since I've purchased a Governor, I am looking into getting one. I think that will be the best way to carry the full sized revolver.