All fixed. Thanks to this guy's video:
Whatever lube was in there from S&W 15 years ago dried to hard tar like substance. Some had to be scraped off. Brass wire brush, lots of cleaning, and new gun oil. It's working like a champ.
My prying with a screwdriver, I laid a cloth wrapped screwdriver sideways so I was slightly prying away from the "ear" behind the cylinder. I wasn't going crazy screwdriver prying mode.
I guess these things need to be stripped down and cleaned every once in a while. It only has a few hundred rounds through it over 15 years.