Let me clarify one thing on the LLC for @Rob Beland and @Christian Carson I was suggesting to set up an LLC for his business not to place the duplex into, yes placing the duplex into the LLC would negate any tax benefits that you gain by owning your own home.
I guess the last sentence in @Rob Beland's post says it all. "You are their neighbor" and therein lies the problem. It is going to be a challenge being their neighbor and their landlord at the same time. Especially for your first RE deal.
Because the first time the rent is late, the excuses start, and then they will say, "Hey our kids play together", "Our wifes' are friends', "We have beers together," etc. Then you agree and let it slide, but down inside you are thinking I still have to pay the mortgage.
Then you start thinking that this RE stuff doesn't work, it is too hard, people suck, whatever. But RE does work and it is challenging at times but let me tell you from 20+ years of experience with dealing with tenants - Your not their friends, sure you need to be friendly, but you are not friends. You may think of it that way, but they don't, trust me. Don't put yourself in a position that you are going to regret later. Ask me how I know?
Anyway, I guess to solve this issue my suggestion would be to buy the duplex and place two paying tenants into both units and get an apartment which will be paid for by your 2 paying duplex tenants. Then go get another duplex, and another and another. Rinse and repeat!
Work on your business not in your business. I have bought a lot of properties from "Tired landlords".