rent out right after buying house for primary residence

[Yoon Seokwoo]

Say I meant to buy a house for my primary residence and told lender about it. The lender gives me a loan for conventional mortage with 20% down on primary residence (lower rate than when buying investment property) But something unexpected happens after closing date and I can't live in it. Then, can I just rent it out without any panelty?  - differently used than my original intention (living there as primary residence) 

Anything I should be wary of?