For your first commercial deal, your best home is probably going to be with a local community bank or credit union. 

Credit unions often have great programs, so hopefully this is one of them!

In some cases they cannot pull the money out because it is now (Non-owner occupant) an investment and their credit union or bank may cap them at 70% or 75% MAX LTV.Nothing wrong wiht putting 10% down on a primary but look at the rates on both 5% and 10% usually when you have PMI built in the rates stay the same.

One is fourplex near downtown of Union city new jersey and the other a duplex in jersey city heights neighborhood.

Small banks and credit unions focused on farm/agriculture tend to be the most aggressive with land/lot loans.

I have some prior experience house hacking my condo in union city, but outside of that I am generally new to real estate investing.

I am above 50% on standard Visa/MC, below 50% on Amex, and about 0% with anything that is like a local credit unions. 

I'm currently biased towards Indianapolis based on its job growth and affordability, but you could honestly find deals in any of the markets you mentioned (and if your heart is set on the Atlanta metro area, I'd strongly suggest the towns Union City and Fairburn based on their growth, appreciation, and price -- I almost closed on a good BRRRR deal in Fairburn but the rehab was just out of my budget). 

This was my first investment property and I used a loan from my 401k for the down payment and also used a local credit union for a 10% down payment on it when I bought it.

I am finding that most credit unions will only do helocs on primary residence.