I have owned a good number of S&W J frame rimfires, and a few Charter Arms Pathfinders. I was never impressed by their accuracy, but I got best results with standard velocity ammo.
I currently own a snubbie 317 AirLite, a 4" 63, a 650, and a 4" K22. As stated, the 317 is difficult to shoot. I put a set of Herrett stocks on the 63 and factory banana style stocks on the 650 and can do right well with either one. My K22 was disappointing until I stuck a set of Rogers's stocks on it and tried a box of CCI Mini-Mags. What a difference!
A slightly less expensive option would be a S&W 2213/2214 .22 pistol.
Try it, you will like it.