What to do if contractor delays work?

[Ming Liu]

I paid my contractor $5k to start a heating system job. I knew him through a friend and he did two big jobs for me with no problem, so we didn't have a formal contract and proposal in place yet when I paid him. He promised to finished on Dec 19th, but didn't do anything, when being asked, he said that he will be done by Dec 31st, and only showed up one day, now I'm checking on him everyday and most time he promised to work but didn't show. Now it's been more than a month with only three day's work done. I cannot rent out the unit due to the delay and downstairs tenant started to complain as well because they have no idea when the contractor is going to show up do the work. Can I have other contractor finishes it and ask for the money back from the other contractor? What's the best way to do this? Thank you!