My biggest concern is the size difference.

In this city, the occupancy limit for duplexes is 6 people per unit, and zoning told me they don't concern themselves with the number of leases at a property.

What are the key characteristics that a buyer should be concerned with when shopping ?

The only thing that would concern me would be if you were related parties. 

I also only accept cert check though so it's not a concern.  

You and your partner should discuss your situation/concern with your CPA. 

Essentially it was a 2-4 unit property as far as the lender was concerned.

He is a high income business owner so he can handle the financial risk of a shortened vacation season, but he's also concerned.

Your question needs to be about what properties give you the most profit and not about the insignificant difference concerning the amount of time and labor in regards to larger or smaller properties.As for turnkey properties, that is a huge subject and don't ever trust someone else to do your math or manage your properties.

Aside from the normal concerns of letting a stranger stay with you… business people are busy people or they’re done with being busy and value their freedom.