Vacation / Second Home Financing on Small Multi-Family

[Brant Jones]

Has anyone worked with a lender that offers second/vacation home financing on small multi-family properties (2-4 units)?

We've been pre-approved for a second home with Navy Fed but they only lend on SFRs for vacation / second homes. We found a great opportunity to purchase a duplex, and get an "almost free" vacation property by renting out the other unit, but have been unable to find a lender that has the right financing product, i.e. not a conventional 20% down investment property loan.

Appreciate all the help in advance!