Question regarding my right to rent as a landlord

[David He]

Dear bigger pockets, 

My wife and I are both new to multi family investing and I posted two months ago about a "failure to obtain permits for a duplex" citation for a new multi-family property we just acquired. 

You can read more about here. But the tldr version is that the seller had turned a single family home into a duplex which they didn't get permits for and now as the new owners, we are on the hook for the missing permits. 

We currently live in one unit while renting the other. The officer's main issue is not with us renting out the property, but with work that was done and no permits were obtained. So I asked him - if we bought a house that was not a duplex but also had a lot of work done without permits, would we still be required to get permits for all the work that was done, and he said yes. 

But my real estate agent believes that we are only responsible for getting permits for any new work that we do and that the county cannot force us to get permits for work that was done before and suggested that if we just ignore the county, this will just blow over.

I'm reluctant to take this approach and I would appreciate everyone's advice here. I don't care about the legal designation of the property as a duplex and just want a property where I can rent out both sides. What the previous owner did was just partitioning the house into two sides by installing a board. The house is still single family in every other respect.

I'm not sure why I would have to turn it into duplex in order to rent it. The county said that if I don't turn it into a duplex, I need to turn it back into a single family and get permits for it. It's my property, and I would think I have the right to partition the house how I want without the county telling me how it should be. 

I'm meeting with the county supervisor tomorrow and I plan to prepare as much as I can. If anyone has any suggestion or knows of any statues regarding a homeowner's right of being able to modify his property and rent it, or being required to get permits for all past work under the previous owner, I would greatly appreciate it!