FHA denied the mortgage from 2-family Multi to 3-family Multi

[Peace Lily]

Dear BP Members,

I have a question and hope to get your advice. I lived in a 2-family house for one year on conventional financing as my primary residence. I recently just moved out and rented out the whole house. The clause with my existing mortgage bank is that I can move out after one year of occupancy which I had fulfill. I saw a 3-family house and planned to buy and move in as my primary residence and applied for FHA or Fannie Mae financing but I tried with a few mortgage companies and all got rejected so far. FHA said that I cannot have both multi family on low interest rate and they said I was trying to use 3-family as an investment property even though I would be living there. The mortgage broker recommended me to refinance my existing 2-family house (not to take out equity) so that it becomes my investment property but I asked my existing mortgage bank and they said that I don't have to refinance it even though I am using it as an investment property. They don't mind. I am puzzled. I have heard other investors moving from one multi family to another multi family using FHA financing. Is that correct or am I missing something? What other options would I have to qualify low down payment? Many thanks for your advice in advance! - Ling