Bought a conversion without permits- expensive mistake

[Elizabeth Read]

I bought a 3br/3 bath house 2200sq ft in Altamonte Springs FL this Nov. purchased for 105k using hard money, put about 25k into it and got it under contract in Jan for 173k- all was well, passed inspection with no notable concerns, headed toward a late Feb closing. Then my buyer's appraisal for the bank blew up when she couldn't find permits for conversions made several years ago by previous owner- they had converted the garage into a master converting a 1600 sqft home into 2200 heated. Did great work, never pulled the permits.
I was unhappy- the appraiser said she'd finish the report up using 2200 sq ft, but they'd need the permits (which don't exist)-- we're now talking to the county about getting the house inspected and getting it squared away but it's turning into a nightmare. The first person we talked to said it can't be done, can't sell the house. My realtor's boss is saying the last thing we want is the inspectors going thru the house with a magnifying glass.
You've heard it before, but let me remind you-- if you buy a house that has had a conversion- make SURE it's a legal conversion with permits on record. I'm thinking I'm gonna have to buy this house--- which puts me out of the rehab business for a while.
[b]