Owner Occupied- Want to Move Early Due to Drug Dealing + New Baby

[Rob Roy]

Hi Everyone!

My wife and I bought a townhouse two months ago using a 5% down conventional loan.  We had the intention of living there for a year, turning it into a rental and then going to buy another property in the area to rinse and repeat.  However in that time, I've seen and have had others tell me they've seen drug dealing happing a few doors down, and we found out my wife is pregnant with our first child - yay!  It appeared to be a nice townhouse community when we would come by to look at the property before, but after actually living here it's not that great and bringing a baby home to it makes me nervous.

If we were to move out early, I would still want to rent the townhouse out and go buy another house using a VA loan, but am afraid this constitutes as mortgage fraud before the 12 month mark. Is it better to just ride out the remaining 10 months, or does having a baby and wanting a bigger living space preclude me from following the one year owner occupied agreement?