I’d struggle to come up with the 3.5-5% down on an fha or other low down options. 

Not ideal, but I imagine this situation must drive you bonkers.

If I had to do it all over again, I'd buy a primary residence with 3-5% down, maybe house hack, then buy a new primary every time I've saved up enough for a new down payment leaving the former primary as a rental.There's no telling when interest rates will come down.

I'd say the question is better suited for your financial advisor, who can determine the best use of the money.

Don't have the cash or equity at this point to do the cashout refi, otherwise that's the route I'd prefer to go. 

Had I hired that Builder or someone else to build it 4 years ago I'd have save 62% as well as a huge amount of time being the GC.So, this build isn't just 30% more than that duplex I mentioned. 

Unless you are sure you can replace your income with real estate, I'd be reluctant to quit your day job.

Not sure if I'd do it in a rental but this ones going on the market, so I trusted their judgement.

I'd recommend talking to a mortgage broker who is licensed in your state. 

I'd sell and move into a SFH or MF that you will have more freedom.