After that he said "you will pay" to me and threatened to sue me due to my neighbor smoking. 

In your situation, I would probably consider hiring an attorney.If he is violent or threatening, report it to the police immediately and get it documented.

@Russell Brazil he is threatening to have me arrested.

The carpets will never be the same.I made a good faith effort to pay Javier half because I didn’t want to sour things with my realtor but he then threatened to put a lien on my place.I’ve now retained a lawyer and I’m going to sue him for damages.

HD will threaten to enforce unless you pay them to remove restrictions...Its all just leverage....to be used or not.If you'd like leverage, buy about 3-5% of HD stock and now the shoe is on the other foot!

They are threatened a lot and when I visited as a 10 year old my cousin was actually kidnapped for ransom!

In the future, add a no commercial use clause on your lease, something like:COMMERCIAL ACTIVITY:TENANT(s) agrees and understands that LANDLORD does not permit commercial/business activities on the premises being rented AND LANDLORD may, upon service of a 5-day notice to TENANT, terminate the tenancy of TENANT, without giving TENANT an opportunity to remedy the default, if TENANT, a member of TENANT household, or a guest or other invitee of TENANT or of a member of TENANT household engages in any commercial activity that increases traffic to the property and/or threatens the health or safety of, or right to peaceful enjoyment of the premises by, other TENANT(s).So, doesn't matter what causes the traffic, the traffic itself is a lease violation.

Upon our return the next day, the snow had melted and turned our giant visqueen into a "stairwell water balloon" which was threatening to burst.  

My spouse had to contend with a re-roofing of the new home after they threatened Lennar with court (by one of my few neighbors who was not in the 30k club) as this happened to every home in the neighborhood.

I pissed and moaned, threatened to just get a new loan somewhere else blah blah.. they eventually asked me for who my appraiser of choice was, I got a name of a real person with a brain in my town, they ordered one from her... problem solved.Either you overpaid, or the appraiser is a tool.