You will have to pay mortgage insurance on the FHA loan as well as your monthly payment so be sure to factor that into your expenses.
Also, FHA appraisals are a bit different than conventional mortgage appraisals. The appraiser is required to do a more detailed inspection to ensure that the property is safe, sanitary, and habitable. They will look at the roof, the foundation, etc. With conventional banks their goal is to close the deal and sell the loan ASAP. With FHA their concern is that the person they lend to is living in a safe environment. See the difference? If there are multiple offers on the table you might miss out because it might take longer for the loan to go through.
I'm not sure how to answer the last parts of your question about the fourplex so I'll leave that to someone more experienced. I would think having two cashflowing 2-4 family properties between the two of you would be better than one fourplex, but I don't know enough about your market to really say. It also depends on your budget for the down payment and what you can currently afford.
Good luck to you both! :)