I am by far not an expert, but I have shot black powder for years, and as a serious competitor for the last 11. I have on occasion done a quick poor job of cleaning at the range, and then forgotten to do a better job when I got home. That means my 1855 rifle has sat uncleaned for a month or so, and although it develops a light rust in the bore, a brushing and swab with oil takes it out. (It is hard to hurt these guns, and no, it is not original, I built it myself)

On the other hand, I forgot to swab out the bore of my nagant rifle after a trip to range shooting corrosive primer ammo. I found it after a short period of time and the corrosion was a much more serious issue and took a lot of work to clean up.

So, this not so scientific example makes me think the primers are at least as bad if not a worse culprit than the black powder, or maybe a faster acting one. Either way, clean up after you shoot.