Trouble refi out of hard money loan

[Carlos Handler]

Hi everyone! I purchased a home with a hard money loan. I'm trying to refinance to conventional loan. The property is a single family residence 5 bedroom, 3 bath. The previous owner split the 2 story home into two units. Upstairs 2 bedroom/1 bathroom & downstairs 3 bedroom/2 bathroom. The appraiser I had said the upstairs is an ADU but isn't permitted. What ideas do you guys have? Is there lenders out there that will refinance? Thank you!