We are in a position where we could afford to pay for both my primary residence and this short term rental if things didn't work out.

And if one roommate leaves, I want the rental to be affordable for remaining people, which is important because roommate situations fall apart all the time. 

Unfortunately, for my clients with less purchasing power the current climate is making it extremely difficult for them to purchase a home that they can afford and at best, "like."

Wish I could have afforded to get a couple of them 

I am currently in the furniture industry and I have rental properties as well.

Additionally, you are only able to reap the rewards of one deal...and that is a lot of risk in one deal.That same 100k, you can keep in the bank (if your private lender agrees to $0 down) and flip as many properties as the lender can afford to fund (and as many as you can manage).Or, you could leverage it on 10 different properties at 10k down payment each (if that is what you and the lender agreed to).You and your private lender can agree to any terms you want. 

I will say, if you're seeking "affordable housing" then generally you're midwest and south east states are a good starting point.

I think getting actual quotes would help me determine what I can afford to withdraw from the whole life policy if I go that route. 

If you want to really dive in and do more deals, then pull out the money so long as you can afford the extra monthly.

What can I afford?