FWIW, I thought paying estimated taxes was optional. This is payment in advance. (If not, why are they "estimated.") The concept of paying by the quarter for the current year, as I understand it, is to make the bitter pill easier to swallow when it comes due as a whopping total on 15 April of the next.
I did that a couple of times, but I found in my case I preferred to just write one check when due.
It seems to me that at least some of you are talking about paying taxes that are already due from a previous year(s) spread out over time which explains the late penalties due. I've done that, too, when I got arrears due to a divorce.