Debt acquisition/pay-down peak?

[Aaron Hollingshead]

So, there's an idea I've been toying with for a few months now.  I know @Brandon Turner loves to talk about funnels...but what about this idea of a mountain or a peak or a tipping point?  I pride myself on making my own decisions with my money so while I know that conventional wisdom says I shouldn't pay down mortgages, I just don't necessarily buy that!  It doesn't account for different goals...for instance, paying off rentals would provide stability, peace of mind, higher cashflow, something tangible to pass on when I die, and a myriad of other benefits (yes, I know, with a myriad of drawbacks too).  

So here's my latest hypothesis...there must be a peak or a tipping point where it makes sense to pay them off.  For example, I'm 36 now and have a net worth of around 350,000 with 800,000 worth of real estate.  Last year I made around 15k in rental income.  If I paid off my rentals in 5 years (which I think I could do), I'd be netting around 50k a year...might not make sense now.  But, say I spent 3-4 years growing that inventory to more lieke 1.5-2 million, then spent 6-7 years paying it off (10 years total instead of 5)...then I'd be in a position where I was making, hypothetically of course, more like 100k a year and could go work at starbucks (something I would actually love to do, lol) and just enjoy the rest of my life...

So in sum, should this real estate game be looked at from two perspectives - accumulation and pay-down?  I feel like so many people are just stratified on either end of full on leverage or pay all for cash, and I wonder if this is an alternative people should consider?