Also the fact that we don't live in Hawaii I see as major downside.

The downside is that I have to run a lot of the HOA duties or they probably won't get done. 

@Nick Edwards, a potential downside to borrowing from a 401(k) is that if your job changes - you quit or get fired - you have to pay back the loan in full within 60 days or you are hit with early withdrawal penalties and are taxed at your current tax rate.

Sure, we both benefit from the upside, but only one of us would fear the downside.

So, then you'd have your downside case at least be known.

Hi @Connor Simonds I agree with @Chris Simmons   and @Judah Hoover  - limit your downside risk as much as possible while you're starting out.

One downside of the 401K option,With the SDIRA, I thought I read where i have to have a custodian?

If it's a short term hold, I would factor in flood insurance to mitigate the downside.

I've done a lot of reading here but one thing I can't find is the doom and gloom (other than the people posting you could lose all your money) I can't find any tangible downside. 

I'm writing this post as a follow-up to a post I made over on the investor psychology forum about the downside, isolation and anxiety resulting from achieving wealth - and I was really grateful to see an outpouring of support and great advice (rather than insults, which I was anticipating).I said I'd write another post with the story of how I and my family got to a substantial net worth (by mom and pop standards) over the years.