Possibly break loan covenants and can't cure default, lender takes back property.  

We've utilized the VA loan for our current home, and this strategy has worked well so far.

And if it's a high amount, would you be better off just using that as a downpayment and get a standard new investment loan?

A non-recourse loan is an option if your SDIRA doesn't have enough funds.

There was also a loan grant available from the state of NH for first time home buyers for 3% down payment assistance.

More doors the better if you are using an FHA type of loan.

The current lender refuses to do a loan assumption. 

Hi Treza,Not sure where you are in the process there, but when your ready to exercise financing options please reach out, it's very difficult to find a ground up program for inexperienced borrowers without having an experienced GC sign on the loan with you which in turn is not easy to find. 

This term replaced "subprime" and looks a lot different than subprime loans of the past.

As far as credit, you would want a score above 680 ideally to get the highest loan to value on a cash out refinance, but I would aim higher at 720+ as that comes with a significant rate drop.