Multi-Family Loan in Louisville, Kentucky

[Samuel Ault]

I am looking to purchase a tri-plex or four plex in the Louisville, KY area. I want to live in one unit. I am a first-time home buyer. I want to obtain conventional financing. 

The only person I've talked to so far was a mortgage broker referred to me by my realtor. She said that the most she could lend on a tri/quad plex is 75% due to Fannie Mae. She also said that this would likely be the case with most banks that I went to. 

I have already reached out to several bankers that I know (Chase, Republic, YCB), but I am thinking that surely someone on the site has figured out a way to do better than what I have described above. I have a decent income and good credit for what it's worth. 

Please let me know of any strategies to get my down payment below 20%. Thanks for reading!