Civil Engineer by day, FIRE (financial independence retire early) seeker by night. 9 month reader and first time poster. 

Or is there a better way (other than saving over time). 

I have a duplex that is basically breaking even but we will save $550 by going from a 5.5% fixed down to 3.5% on the remaining principal (no cash out).

So, if they are now retired and in a lower tax bracket than you and your brother, they may pay a lower effective tax rate on any taxable income (if any) from the property in your proposed scenario, than you and your brother would if you just bought it and let them live there.

I have saved $20,000 so far and I would like to invest it wisely.

I'm currently living in a duplex I own, work full time, but my employer doesn't offer a retirement plan.

When a tax exempt entity like a retirement plan engages in a trade or business on a regular or repeated basis, then UBIT applies.  

I have a decent amount in savings, but I would like to build additional investment capital before I purchase my first single family home.  

I was in love with micro lending until the account hit 5 digits and I started to get nervous, so I opened a Prosper account to build that and when that hit 5 figures I started saving for my first rental. 

To be honest, I am currently in the saving / researching phase right now.