Using FHA to buy a second investment property

[L. Brown]

Hey BP,

I bought my first 2-family home (owner occupied) in August 2020 with a conventional loan. I am working on buying another multi-family (2-4 unit, owner occupied). I spoke to the lender I used to buy my current property about buying another property and he said the following:

"For a conventional loan the minimum down payment for a 4 family is 20% vs 15% for a 2 family. Since you already own a 2 family it will be very difficult to buy another multi family using a FHA loan since FHA has a guideline that does not allow the program to be used to acquire additional investment properties."

One of the reasons I went with the conventional loan is so that could keep the FHA in my back pocket if I wanted a home I wanted to put a lower down payment on. Is he correct that I can't use the FHA loan to buy my second investment property?