Owner-Occupied Investment Property Question

[Austin Dame]

I'm looking to move out of my current house and buy a multi-family property where I would occupy one unit. I'm worried however because my house is a very strange layout. What if it doesn't sell and I can't owner occupy my investment property in a timely manner? What if none of the tenants will accept a buy-out and insist on staying there until their leases are up? Will the bank try to charge me with mortgage fraud or demand that the loan be paid back immediately?

I also run a business from home. I'd love to buy an investment property a little outside of town, but what if I move and no one will come to me because I'm too far away? Would that be an "unforeseen circumstance" that the bank would accept for me to not owner occupy if I try it and fail and have to buy/rent something else?

My mortgage broker says that it's fine if I fail to owner-occupy so long as I actually have the intention, but I don't know if I trust them. I don't want my credit ruined or for this deal to blow up in my face here. I'd be sinking everything I have into this deal because I can't afford a 20% down payment on an expensive multi-family property. I also must move from my current house due to health issues (it has way too many stairs) so I thought I'd kill two birds with one stone here and just owner-occupy a multi-family investment property. Thoughts?