I had the same experience with my 15-22.
But it's so handy and fun to shoot, I just couldn't stop chasing accuracy.
Every rifle is different, so You'll need to find what works best for yours.
But here's where my efforts went...
The 1st thing I did was take the cosmetic round insert at the end of the foregrip out. That actually helped a little, especially with varying barrel temps. I tossed it after verifying improvement.
Next I followed the advice of some folks that said the barrel/handguard nut could be loose. I checked it. It wasn't loose. I reseated it to spec anyway, just to check that box off. No improvement.
Next, I replaced the trigger group. I think the first one was CMMC brand. Later I installed a Rise trigger group, but only because I wanted adjustable trigger pull weight. Both were big improvements over the stock trigger.
Next, I purchased 20+ brands and types of 22lr ammo. Before this I'd had my most consistent results from CCI Standard Velocity.
At the end of controlled testing (cold barrel, warm, hot, clean barrel, dirty barrel, hot days, cold days, sub-sonic, Mini Mag, and on, and on, and on...) I landed back on the same ammo, CCI Standard Velocity.
To be fair, 22lr just doesn't have the quality and consistency a marksman is looking for. More money usually gets you better quality in this caliber, but no guarantee. I'm just not willing to spend 20+ cents a round for my varmint hunting and plinking.
After all that I was still getting inconsistency with the first 1-4 rounds. Then it would settle in for about 20 to 40 rounds. Then it would drop off again.
I worked diligently on my cleaning routine, trying to find the sweet spot for accuracy. I tried full wet swabbing with various cleaners, to just using a bore brush without any solvent. Dry swabbing every 10 rounds, to brush only every 10-15 rounds. And I also tried various levels of cleaning for the bolt face and barrel face (where the bolt meets the barrel).
No matter what, I still had inconsistency with the first 1-5 rounds, then good for 20-40, and then back to inconsistency and flyers.
It seemed to be related to barrel temp. Since this a squirrel/rabbit getter, firing a few "seasoning rounds" before aiming at my quarry really wasn't an option. I wanted to just grab the rifle, step outside, and take my shot. Or walk through the woods and take a shot when the opportunity arose.
By this point I was 7,500 rounds into ownership. I knew the rifle. I knew it's behavior. I knew I couldn't rely on an inch left on first round, inch up, inch down... You get the idea. Just inconsistent until 3+ rounds went through the cold barrel.
I really enjoy the light weight and easy handling of the 15-22 for my wanders in the woods and varmint control around the house. I was researching other platforms to replace the 15-22 when I decided I'd take one more radical step, before retiring it to plinking duty.
I purchased a Volquartsen replacement barrel at a reasonable price (much less than a replacement rifle). That was the magic!!!
I fired 75 rounds to season the barrel, using 5 rounds each of 15 different ammos. I cleaned the barrel and settled in to run the 20+ brands/types to find which worked best.
Here's the joke... CCI Standard Velocity was still the most consistent! But now I was getting consistent 3/4 MOA (really!). No more flyers! No more cold/warm barrel variances! And far less variance between ammo brands/types. (Disclaimer: 22lr is still inexpensive, low-quality, plinking ammo. So flyers are going to happen)
So, I'll second what others have said. Out of the box, the 15-22 is simply incapable of consistency and <MOA groups. Even with the usual modifications, it still won't group reliably. I know that folks are going to weigh-in with what a "tack driver" theirs is at 100 yards. I call BS on 98% of them!
It's a shame that such an expensive option is the only way to get it to hit consistently because it's really such a wonderful plinker and little game getter. But I'm happy with it!
So, there's my experience and assessment. I understand the OP's frustration. It's real.
Carry on...