Lee, that's why I started with "Depends on your plan". If you want to build wealth slowly and not planning on quitting your job, paying off the mortgage (even though it will slow down your acquisition) is a good strategy.
The way to mitigate the risk of debt is education, buying right and having multiple exit strategies. Having leverage on a cash flowing rental property in a good location is not a bad debt. If you are an experienced landlord (education), you have plenty of equity in the property (buying it right), and you can rent, wholesale or retail the property if need be (multiple exit strategies) you're covered. The leverage on real estate is going to build your wealth. How many people became millionaires via stock market vs real estate? I have been in the stock market for years and have not experienced the returns I get from real estate currently. I still have money in the market but that's not my focus.
You sound like you have a good head on your shoulders. Delve more into real estate. Learn how to do it well and you will see what I'm talking about. 100% cash on cash return is not unheard of. You can't do that in the stock market.
Here is why I like real estate over stocks.
The monthly Income - cash flow (taxed at the lowest level)
Annual depreciation - can't do that with stocks
Equity build - as tenants pay off your mortgage,
Appreciation - initial forced appreciation, then annual appreciation
Leverage - you can borrow against it. Can you borrow money from a bank to buy stocks?
Plus you can insure it - can't do that with stocks.
Income, Depreciation, Equity, Appreciation, Leverage makes it IDEAL.
Take care,