I found one lot that has this possibility but the owner didn't get permits.

At most do some painting, maybe refinish the floors, change out the locks and call it a day.The first investment really is like your learners permit. 

I’m concerned about rehab headaches in dealing with the city, pulling permits and funding it.

Ducts, permits, wiring etc etc.

You can't just get a bulldozer and start knocking down trees and move dirt around.Usually, before a person buys land they make an Offer To Purchase and the closing of escrow is contingent upon getting building permits and your building department's approval to develop the property.

Not to mention things like utilities, permits, materials and people not showing up.

Depends on:- Location- Types of contractors you use- How the project is managed (GC or Self)- Level of Finishes- Time of Year- Permit and Impact Fees- Your Negotiating SkillsLooks like you're in Atlanta -- you're likely somewhere between $80/sf and $150/sf, depending on the things above. 

Bit of background - The property is in a commercial area and I believe at one point in time was a hair salon.Given this is my first rental and my inexperience I reached out to a couple contractors and one of them told me to start at the permitting and zoning office.

NJ uses the same codes across every town and the paper work is pretty much the same for permits etc.

Ironically, it actually discouraged me from building a permitted ADU and opting for a nonpermitted basement ADU for au-pair quarters and later on airbnb (STR not allowed for permitted ADUs/duplexes etc) which seems to be a better return on investment vs. long term renter if living close-in.