Help me sort through the misinformation

[Kim Freeman]

I am in New York and bought my first home, a 3 family a year ago with an FHA. It's cash flowing nicely and it's what inspired me to want to do it again. I went to my RE Agent to discuss another multi-family purchase (which I intend to move into) and he informs me it's not possible. The bank will not allow me to use 'primary residence' as a reason for low money down because I already used it, and have a primary residence. Now , I am restricted to a 20% down conventional loan unless I find more creative ways to finance. I asked my first banker and he said 'yup true'. I'm still doubtful but I hesitate to go directly to another bank to ask, because I'm not believing anyone at this point since I hear this advice given all the time on the bigger pockets podcast! It CANT be true cant it??. looking for a general consensus I think.