Cash Out Refinance on Primary Residence

[Gabe Bouldien]

Backstory- I bought my current house in 2015 when the market was really soft in my area and I got it for a really good deal (195k). Over the first few years of ownership my wife and I rehabbed the house completely. Now the market is red hot and houses in my neighborhood are selling for 325-350. I have 105k left on the mortgage. Rents in my area for SFH range from 2k-2.5k/month as the area is very desirable and the schools are some of the best in the state. I was thinking about cash out refinancing and pulling $ out for a down payment and moving to a new primary residence and turning my current house into a rental. Estimated cash flow would be +$500-$1000 per month depending on final rent amount. Are there any negatives that I am forgetting about to keep me from doing this? Everybody that I mention this to gives me a look like I am crazy, genius, or a combo of both.

[Gabe Bouldien]

@Jason Ramage- I would be holding indefinitely. I currently have a 15yr at 3% with 9.5 years remaining. I just received a cash out refi loan quote for a 30yr at 2.9% with only $500 in closing as the lender is giving me a huge lender credit. I would be pulling ~175k out. My thought process is trying to keep this money liquid enough where I can use it for my next primary, but smartly "investing" to make some money on the cash out money. I have options to hard money lend in my area with reputable clients at 2% origination, 12% interest for 3-6 month terms. 

I have researched HELOCs too. 

Feels like no matter which way I go I will win. I just like to hear other people's thoughts.