Myers varies based on seasonality, local demand, and competition.

I recommend you discuss and plan this in depth with a local lender who knows your market well. 

Quote from @Devin Peterson: Hey Michael,Local to you in Middletown here I would love to connect and chat more about this one. 

If you prefer to stay local and build equity, house hacking a duplex or triplex in Fort Lauderdale could work, especially if you can add an STR component to boost income.

I'm planning on hanging on to my primary and renting it out (medium or long term due to local rental laws).

There are people located all over who could help you locally. 

Why not talk to a local agent or 5 and see what they think?

I think their logic was that a family with 12 people (parents and 10 boy children) would have difficulty finding a suitable house with say 6 bedrooms.In addition to the Federal guideline, some local ordinances address occupants both tenants and owners. 

But for the last 3-4 years local residents have decided it was cheaper to just take the groceries. 

I am a local realtor in the Boone, Blowing Rock, Banner Elk area and would love to help out if you need any assistance.