Looks like he was running a game on you, in my humble opinion. He either needs to perform and or you will have to sue him. In CA, performing work unlicensed or while your license in suspended is a CRIME. I am sure it is there too if a licensed was required for specific types of work,,, but agin, I don't live there.
Sorry you are getting "dirtied" up on this deal. He knew what he was doing and that seems to be obvious.
Personally, I would calculate what is the best financial move for you to make. If he has not performed to the tune of the money you have paid him then you also have legal cause in that manner.
He either needs to figure out a way to perform or face legal ramifications. If he won't cooperate, get a new contractor in to finish the work BUT document WELL what was done with pictures and also what was not done. Establishing a solid evidentiary case against him is paramount as he will deny anything you can't prove. Might be a good idea before any other contractor comes in to have a Licensed Home Inspector come in and document as a 3rd party the condition of the home.
I really hope this doesn't hurt you too bad in the end. Sounds like a lot of costly lessons learned. I hd to learn them also,,, and still learning!
Side Note: I would record any and all contact with him that you can. See if you can get him to admit failures on a covert digital recorder.... That is good evidence..... So during your next conversations and when you start talking legal moves, that would be a wise move.
Good luck and I am open to anything else you would like to inquire about.
My Best, Rob