Fannie and Freddie Won't fund a "legal non-conforming"

[Michael West]

So, I have an interesting situation here that I've never come across before and could use some additional insight from other's experiences.  I have a client who is buying a Duplex with intentions to make his primary residence, live in one side, rent the other.  He's fully approved as a buyer, however, it appears that the property itself is in an R1 zoned area and is "grandfathered" in as a duplex.  So, on the appraisal it is "legal non-conforming (grandfathered).  This then caused a question to be raised by the lender asking the city's zoning department "If a catastrophic event occurred that required a complete rebuild of the property, would they allow for the zoning variance to continue to allow it to be rebuilt the same as it is - a duplex."  The city said "absolutely not.  It must be compliant with the current zoning."  This shut the deal dead in the water. I've been told that this is a Fannie/Freddie requirement, yet not a single other person or lender has ever experienced such a thing.  To make it worse, we found another property in a completely different city to move onto and its going to be the same exact situation. My buyer is currently homeless after selling his former residence a few months back (with a different agent). 

I don't know that I'm fully believing what I'm being told and looking for someone with some experience in this to confirm or deny this as true.  What other options might we have.  Portfolio, hard/private money options are not within my buyers means.  Seller's are not willing to provide seller financing.