Is it smart to build a multi-family as a first time home buyer to take advantage of the benefits?

A smart agent is going to protect the seller and not fall for unlicensed broker tricks.

I thought it was a smart investment in a nice neighborhood in a nice town. 

At least some of them probably still run (or the seller was smart enough to take the window A/C units out of the windows for the pictures), but you might end up buying some A/C units soon.

So my question is, would it be smart to go this route?

You will need to buy and rehab smartly so you not only not leave money in the home but hopefully pull more than invested out.

just like zillow zestimate is BS, I don't think any AI can accurately tell you what your rent should be.AI still isn't smart enough to look at your photos and location and give you good advice.When I listed my rental last month, the zillow rentzestimate was changing daily.

I'm headed out of this State to the next one.Not to sound like a smart assy...I told the agent what strategy I suggest he use going in, and he declined what I said, which is what I've learned from here, Chris Voss, Meet Kevin, Robert K...etc.

absolutely you can combine FHA with grant programs in NJ - good for 2-4 unitshttps://www.fha.com/grants/new-jersey-hmfa-smart-start

Wealth creation, smart tax benefits, the physical characteristics of a building and how there is a range of possible ways to go (not one path for all) works for me.