Are the sights loose?

The rear sight assembly as a whole, or any single part off it.
Make sure the elevation screw is engaged and not stripped at it's current setting. The sight will just jump around at that point if it is and not hold a steady setting. Even if you have to lock it all the way down in elevation just to check and see if the gun will group well at that setting, then you'll know it's the elevation screw or it's post.
The windage screws get snapped off occasionally or stripped. The blade just drifts a bit on it's own from side to side as you shoot. Not good for fine accuracy and groups.

Even the front sight on these target Smiths can be loose. Allowed to move a couple .000 between shots will give you extra inches in your group diameter. Add in ammo and operator error in the group size and it can be tough to hit a beach ball at 20yrds.

I'd double check for leading. Not just a visual check which can be deceiving and look OK. But a good scrubbing w/a lead remover chemical or scrub mesh.

Check the bbl and make sure it is torqued in tight into the frame. Give it a really good side to side and up and down as well as trying to unscrew it check.
If the front sight is pointing to a different spot each time you pull the trigger, even if only by a matter of a couple .000 again,,there's your shotgun pattern groups.