Buying 2nd property next to the 1st property

[Rob In]

I am in the process of settlement on my 1st property in 1 week. i have a 5% down conv loan thru a credit union bank. this a duplex and is a row house. i plan to live in one unit and rent out the other. the seller is also selling the duplex property next to it. I am interested in buying this property as well as it shares a backyard. I can put max 10% down on this property but the bank lenders are saying this would be a investment property and so I would need 20-25% down. is there a way to buy the 2nd property with max 10% down?