Should we challenge it?

just remembered a few other stuff:-the local HUD/section 8 may have classes for vendors/owners/landlords; may also have a website with all documents/criterias posted-may have excellence programs-inspection: will be more through if children, especially under 6 are in the house.

Finding a contractor to move in mid job has been a challenge and so far I have lost about 3 weeks in the rehab process. 

I'm not trying to minimize your efforts but to alert you to the challenges of marketing in most every community where there are so many buyers with money chasing so few opportunities at the the moment.

I'm happily married and have two young children.

While you can't advertise the property by saying things like, No Children, or anything, you do have more leeway under Fair Housing rules than professionals like property managers or real estate agents.  

We plan on keeping it indefinitely and passing our property to our children.  

Go to school, get good grades, go to college, get a good job when you graduate, save for retirement, work 40 years for a company, you'll be okay.You're challenging that. 

I want to move out in 5-6 years once I have children.

I have a good grasp on what I can get siding, windows, trim, interior/exterior doors, decks, guttering roofing, flooring, landscaping and concrete done for and who to use/where to source materials, along with electrician/plumbing -- lacking site prep, foundation/framing/roof trusses cost -- I could get the cost of the rough in from a buddy who runs a plumbing co.My challenges are A - not being able to be there all day to run things, and B - still missing some of the key contractor connections.