Unfortunately our rural towns have been dying since the 50s/60s - which is sad as there are lots of cool and fascinating places - but it all happens slowly - first a large employer leaves, or maybe there's a farming crisis, tax revenue decreases, then folks with younger kids move away or the younger kids grow up then they immediately leave then never come back, then the schools start to die, then the schools close, then no one wants to live there, then the local hospitals or clinics lose fed or state funds then they close, their workers leave etc. 

So if this tenant leaves "legally" at the end of the leave, you're could lose 13 days rent.

I feel that would leave me with no debt and in a position of flexibility. 

The terms with the seller is he is selling at 280k with 25k down (less than 10% down) leaving me with 255 left at a 5.5% 20 year pay down term.

Until you build a track record with a private lender they will surely want you to have some skin in the game so there's that side of things you have to resolve with them.

The snowbirds leave by April / May and don’t return until Oct / Nov.

Try calling the health department, but that may result in everyone having to leave while the house gets treated.

They’re almost certainly going to require a down payment as they want their borrowers to have some “skin in the game”. 

We also made the mistake of moving into a bigger home that we really liked and didn't want to leave.

That only leaves 8 feet (4 in front, 4 in back), but that is NOT the problem.