Nylon or rubber? Use a brass punch? Heck, give that thing a whack with a 16oz rip hammer.
In all seriousness, you don't need to hit it hard. Metal on metal will deliver the force better than a nylon or brass surface will. Place the muzzle on a 2x4 and tap the block toward the muzzle with the hammer. Regardless of how you get it started, once it moves about 1/16" it should just slide off as you rotate it to break the friction. You won't need to hammer it all the way off.
Are you trying to preserve the old gas block or do you just want it off? Used it might sell for $5 so, not worth saving in my book.