I built my last house 17 years ago. Cost me 65% of what it appraised for. Took a 15 year fixed rate mortgage and paid it off in 7 years. What was being spent on house payments went into a money market account, used to pay property taxes, homeowners insurance, and any major maintenance expenses. The excess went into retirement accounts every year.
Recently purchased a smaller home for retirement (patio home community with no exterior maintenance, lawn care, etc). Paid cash for that. Sold the old house for more than double what I spent to build it, and put that money into retirement investments.
One reason why many financial advisors recommend not paying off a mortgage is because they want to help you manage the money you would use to pay it off, then there would be no vig for them.
Being debt free gives a whole new flavor to life.
Assuming a $100K mortgage at 3.5% the mortgage interest write-off is $3500 per year. Assuming a 20% tax bracket that saves $700 per year in income taxes. Net cost is $2800, or 2.8% of that $100K. You might be able to get better than 2.8% earnings on that money, but not in CD's or money market accounts, only in something with some risk attached (stocks, bonds, etc).
In my opinion the only times it makes more sense to use borrowed money is (A) when you don't have the cash and need a mortgage to make a home happen, or (B) when you are talking about investment property, with leverage being the name of the game.
Unless you own your home you never have any control over your housing expenses, and you never truly own that home as long as someone is holding a mortgage on it.
My $0.02 worth.