In Europe the pipes are considered an aesthetic element and often left exposed outside the building, so I hear, and I think you've caught that spirit; I love it that you've left the stack out the open. 

@Sonia Spangenberg If the city is giving you issues, you could ask them if you gave them an environmental report showing that the land is safe and added septics is doable and won't harm the water table that might get them to calm down. 

Theres also a large element of skill involved... buying turn key?

Ranch land, farm land, land with preservation easements or environmental easements, land for wildlife, water, air, mineral rights of land, land for development and subdivision, and general land use of all kinds.

(*Note: we waver on the slightly neurotic side when it comes to environmental health - I'm sure most people would be totally fine living in this county, plus our soil tests showed totally normal levels of heavy metal exposure.  

Even though I firmly believe this is all vastly overblown especially compared against past flu variations in the last two decades, it is hard to ignore the human element and the fact that some people are legitimately scared right now.

The other tenant is paying $700/month so I can probably get the same amount in rent without doing much, but Im looking to get that extra 100$ in rent, if possible, but sprucing up various elements in the house, including the bathroom (in the area rent for a 2/1 duplex unit can be as high as 850).

At that price point, tenants are grateful to have a landlord who actually takes care of the place proactively, refreshes elements of the property before they're decrepit, etc. 

This would be the starting point to resolve this, and in future leases (if your lease is silent on this element) you will likely make sure this is clearly addressed!  

Also, if you intend to put as much cash down as possible, you don't want an FHA anyway because the most desirable element of an FHA loan is you can have a very low down payment.