Problem with a contractor - advice needed!

[Zach Amit]

Hello.

I recently purchased a multifamily property in upstate NY.
I hired a contractor to renovate a part of the property, we signed a contract and i did my part in paying him on time. 
The process of working with this contractor has been troubling, we signed the contract in early march and only now he is finishing the work. all in all there were about 10 days of work done.
In the contract he put a clause about delays etc that protects him - so he is protected, although he said all the work will be done by the end of March. 

It's been a nightmare communicating with this contractor. he took forever to answer questions, would never pick up the phone or call back. when i asked repeatedly for him to call me back, he said i was annoying.

The work is now almost done and he sent me a revised "bill" that is 1900$ higher than the specified contract that we signed. (entire budget was 19K). the contract specifically says that any additional work / expenses need to be added in a separate contract and that would need to be signed.

He said he made an "aesthetic" decision and that cost more (without consulting me) and that the bill he got from the electrician was higher than expected (although i can't see why - both the electrician and the contractor came multiple times to make assessments before starting the work, and I have already upgraded the electric board prior to this work) again, without letting me know or asking my approval.

the point where i felt something was not right (besides the bad communication - very frustrating when you have tenants) is when i got a text from him last week saying that the work was done. upon inspection i found that he didn't do part of the job (insulation of a big part of the work) . when i sent him a list of corrections (including the insulation) he called me and said of course he still needs to do that. which left the impression that he was hoping i wouldn't notice. 
on top of it all, he is very reluctant to make any corrections to some of the work that was done poorly.

He is now calling me every day. I still have to pay him his last payment. at the point of writing this, i still haven't seen the final end of the work. (our contract stipulates he will get paid once seen and approved) 

I'm not ok with an extra 1,900$ charge for expenses i didnt approve.

Any advice?

thanks in advance, 
Zach