The original note owner also had a schedule C for the remaining payments they would have at the end.

The structure seems bulletproof, but I wanted to run it by y'all to see if I'm missing something - thanks for any feedback you can offer.I bought my primary residence for $168K, and paid the mortgage down like a mad dog. 5 years later, I did a cash out refi (30 yr fixed, 3.9%) for $120K, which paid off the $34K left on the note, and used the dosh to buy an $82K rental duplex.

Yes sometimes it was messy but usually between the choice of 1 or 2 apartments a new person would get the idea that yes it was floor plan they were considering and were able to overlook the sometimes messy aspects.I did run across an occasional unit that was not able to be shown until vacated, due to terrible housekeeping but this was rare, and my owner knew then that I would have a vacancy until turned and ready to show.  

So as an investor, am I able to negotiate down the 2nd with that lender, before I purchase the property directly from the owner?

As a business owner I had the joy of this process. 

The current owners have been in it for 16 years and took great care of it.

Remember, if they have leases then you can't force them out just because you purchased; leases run with the land so you are bound just like the previous owner.

Its a single owner property.

In chatting with a few mortgage brokers, I appear to be limited to 65% LTV, as a "cash out refinance" for a non-owner occupied property.   

I'm sure these properties will cashflow, but will provide some serious headache to the owner, even if the property is not self managed.