I'm also debt averse after 20 years of paying off student loans, car loans, and periodic credit card debt. At this point, if I had enough money in my brokerage account to pay off my mortgage, I would probably do so, even though the interest rate is low.

Also, be sure to include taxes when comparing rates. For example, last year I took a loan to buy a car at 5% interest and was also receiving 5% interest on cash in my brokerage account. It looks like a wash, but I have to pay tax on the the money market interest, so I was actually better off paying off the loan (which I did last week).