Vacation home VS rental property

[Adam Craig]

I am looking to buy a vacation house near Fort Myers FL. How strict are the rules to get owner financing. For instance - I purchase the house as a vacation home (second home), then we realize we are not using it as much as we though - could we then rent it out part time without the loan holders fussing.

Owner financing would obviously be ideal. I just dont know how "flexible" the rules are. What trouble am I looking at if they deem it a rental.