The active ingredient in most primers is lead styphnate. It is a high explosive with a wavefront velocity on the order of 16,000 feet per second.

A small piece of media stuck in the flash hole will not stop the explosion from reaching the powder and unless you're part of the "five shots-one hole at 1,000 yards" crowd, it will not have a noticeable effect on the performance of the ammunition.

If you need to prove this to yourself, prime a case with a clear flash hole and one with an obstructed flash hole. No powder, no bullet, just a primed case. Darken the room. Manually load and fire each one. Compare the resulting light show coming out of the muzzle and see how much of an obstruction a piece of corn cob really isn't.

All of that said, when I tumble brass, I always inspect the cases afterward and use a toothpick to knock out any media in the flash hole. This is not because I think the media might cause a misfire, but because it seems to me to be leaving the job incomplete to not knock it out.