Contractor doing work without authorization or contract then wanting paid

[Paul Baker]

Just got a phone call from a local fencing contractor telling me that he just finished installing a fence at one of my rentals.  Problem is i never told him to do the job.  I got a quote from the guy back in December.  I sent him a text on December 19th telling him i would probably use him but had to wait until new siding and guttering was put up so those guys weren't working around a fence.  He never responded to the text.  Not a peep from him.  No acknowledgement of even receiving the text.  New tentants moved in around the middle of January, they asked that a fence not be installed as they had no pets and they wanted their daughter to be able to ride her bike in the backyard.  

Fast forward to today, April 24th.  Guy says he just finished up the fence and wants paid.  Again, never received a phone call or text from the guy to confirm anything.  Four months has gone by and I had honestly even forgotten about the guy even giving me an estimate.  I asked him who told him to install the fence and he said i had called him and told him to put it in, which I obviously hadn't.  

Can I be held legally responsible for paying this guy?  I'm not saying the guy is trying to scam me, but he definitely dosn't run a good business if you dont even call to schedule a job or have any written contract for the work.  I'll probably just end up paying the goofball so I dont have to deal with him, its only a couple grand, but its still really annoying and a simple phone call or text would have prevented this.  Also, what's stopping him from going to another of my houses and installing another fence and then putting a mechanics lean on the property?

Thanks!