HELP! How to work with sellers with no agents

[Kayla Heine]

Hello everyone! I need advice, I am attempting to buy a 3 bed 2.5 home (My very first home) from a older gentleman with;

no representation

who is trying to move out of state ASAP

Who refuses to pay a broker fee (because he absolutely does not like agents)  

and he also DIY his home by adding the extra rooms and bathrooms. The problem is that permits were NEVER pulled for the additional work, so on paper the house is considered a 2 bed 1 bath. He is VERY firm on the asking price, and is very proud of his work. And wants to walk away with full asking price, he says that if I offer over asking price, that can pay my broker fee.

My agent and I are worried that if I go through the inspection period, and appraisal period, which will cost me $1,500 roughly, the appraisal will not count the additional added space, it will only be considered a 2 bed 1 bath, which is great for me because then the price can be lowered! BUT we are worried he will not drop the price because he feels it is worth more. 
He is going to have this issue with all potential buyers, the fact that permits weren’t pulled will be an issue. 

So do I take the risk of losing $1,500 in hopes that he drops the price, or do we explain all of this to him and negotiate beforehand? 
Asking price is $295,000 but in my market a 2 bed 1 bath is around $250,000 leaving me enough money to pull permits and bring the house up to code. 
Right now he countered offered with paying $3,000 of closing cost. That’s where we are at. HELP