Annealing after every loading is unnecessary unless you're really chasing the smallest groups possible or shooting F Class or something.

That said, I am a metallurgist and I would recommend annealing post sizing as that will remove any work hardening and internal stress that is imparted on the brass by the sizing operation.