Sit back , let him build the house , and when its done , say " Thanks " You should check with a local land attorney and see what your options are 

Maybe being a limited partner with some of the really good syndicators out there is an option to this route. 

In addition to conventional loans and FHA loans, if either you or your wife are veterans, the VA also has a rehab program.I'd be happy to discuss my rehab experience both as an investor and as a mortgage broker, and at least help you think through your options. 

Instead of 40 houses cash flowing $200 each or $8000/month, you buy 10 houses and pay them off for a cash flow of $10,000/month with 1/4th the headache.People don't understand the amount of time, energy, and stress involved with managing a lot of doors, especially if they are lower-class properties typically needed to generate cash flow.

These solutions can make a huge difference, and the long-term option will really help set your STR up for success.

Offer aid and housing options if you're able -- don't try to profit. 

Has anybody found a workaround for this or would the seller simply have to hold the note long-term without the option of selling it?

Consider house hacking by purchasing a multi-family property with an FHA loan, or explore creative financing options like seller financing or partnering with private lenders.

HUD targets average rents for an area, so just like average school grades, this typically translates to Class C property rents.In our Metro Detroit market, the MAJORITY of the claims about getting higher than market rents can be traced to a Class D Neighborhood/Property. 

I have warned countless people that the "new", "Cheapest", and "shaking up the insurance market" carriers typically come into the market under priced, have a wave of claims, then they either jack up premiums to survive or fire their staff to keep heads above water.