Need some contractor advice!

[James Defazio]

hired a contractor to remodel two bathrooms in our rental. Bid came out to $4600 for everything so we signed a detailed sow contract and he got to work. I told him I had to have it done by Feb 1st because I had tenants moving in and he said he'd have it done way before then. He was supposed to start on Jan. 10th. Every time I ask him if he's there working he's always "on his way" or "has his kids" and won't be there today. One time last week we went 4 straight days (Sunday through Wednesday) where we didn't hear from him at all. He didn't return phone calls or text messages and it took me threatening to tow his nice expensive trailer to them impoundment place for hi man to call me back. He told me his fiancée broke up with him and he has been sick (apparently enough to not answer phone calls or texts for 4 days) 

Long story short, he was in tears and apologized so I decided to let him finish the project. He is working on it today but we are not going to make our deadline and unfortunately I didn't have that in the sow. He also asked me for an extra 1000 on top of the 50% I paid him up front a few days into the work because of money trouble. Of course, me being a nice guy, decided to help him out and do that for him. I can see how that has worked out for me. 

My question is, do I have to pay him the full amount he charges me since he has repeatedly taken advantage of my kindness, patience, and abused our timeline? Thanks in advance for the help.