Help! Hard Money headache

[Robert Kriedermann]

I took out a hard money loan for a rehab project. Purchase price was $51,000 and the rehab escrow was $60,000. I've worked with this lender before and the fee is always 5 points plus $2500 with 1% of the loan to be paid as an interest only payment each month. The escrow is dispersed as follows: 20% at the start, 20% after 2/3rds of the project is complete, and the last 20% when the project is finished.

I get my first $20,000 and do 90% of the work before asking for the next $20,000. The lender can only provide another $10,000 because that is all he has at the time. I'm assuming he keeps the money earning interest somewhere until the last possible moment before he has to pay me. That way he earns interest in 2 places on the same money. So I accept what he has and tell him to get the remaining amount ready because the project will be done in a few days and I will be asking for the rest.

I complete the project and ask for the remaining $30,000 of the escrow. The lender doesn't have it. Weeks pass. He calls me and says he has $15,000 of it now and he will have the other $15,000 sometime in a few weeks. Meanwhile, I have been paying interest on money that I don't have in my hands and he can't give me. In a few weeks I will no longer need the money because I have a different rehab project being sold and I will have cash on hand. On the phone, I ask him to keep the $15,000 he can't give me and take that amount out of the loan as well as the interest and fees associated with that amount. He agrees and tells me his assistant will write it up.

I did not hear anything so I email his assistant to find out what my adjusted payment is. His assistant tells me they will credit the $15,000 to my loan and credit me the $150 each month I had payed as interest on that money. I ask about taking off the fees associated with the $15,000 and they refuse.

$15,000 x .05 = $750 and $7.50 per month in interest. I feel that money should be credited to me. One might see him creating a loan based on money he doesn't have as an invalid mortgage. Is it legal to charge someone fees and interest on money that isn't available? Not giving me the escrow as specified in the contract is breach of contract. I feel like just asking for a refund on the fees associated with the money he can't give me is actually being nice about it but he isn't even willing to do that. He says he gives me a discounted rate compared to other people so he isn't willing to give any fees back. Why should I be punished with higher fees because he can't complete his side of the agreement?

What should I do?