If using a stone, you should use a round one, not a square one. It will fit the contour a lot better. Personally I'd suggest using an Arkansas Stone instead of an India stone as they cut slower - you can always take metal off, but you can't add it back! slow, smooth and steady!
IMHO, I'd just use sandpaper wrapped around a round dowel of the appropriate size and shape. The grit should not be too aggressive! You can always increase or decrease grit as needed. You can easily and frequently change paper and grits and no special stone would be required. You can mask off the areas close to where you are working so abrasive dust does not scratch the gun in places you aren't working on. I would not take any more off than the minimum to smooth out the sharp edge. Pay attention to your angle of attack too - just blend it in carefully. Should take a minute or two-that's it. You do not want it to be obvious ruin the gun.
If you decide to go the stone route, you can get a set of 4 Arkansas white "files" (stones) that are about 1/4" - 3'8" in diameter and come in a boxed set of 4 different shapes. IMHO I much prefer the Arkansas stones over India stones for most Gun Smithing because they are harder and cut slower. Lowers your chance for an oppps!