I understand that it starts getting difficult after 4 conventional mortgages, and you’re capped at 10.

In more desirable cities conventional mortgage is preferred by sellers because FHA requires clearance like termite(chk see if your area has that requirement). 

Should I pay the 20% out of pocket and obtain a conventional loan? 

All of my properties were either purchased using conventional financing or cash.

But of course, to play both sides...in reality if I do refinance, I would actually get worse interest rates from the bank since it's currently under a 4.25% rate that was originally designed for conventional primary residence home loan, no? 

The two month buy out is just a convention or starting point. 

I really won't be limited by my lender much at all, I'm pre-approved for a higher amount that I can afford a down payment for (when also considering closing cost, inspections etc...) even with doing FHA or a low down payment conventional loan.I haven't gotten a credit card yet because I'm just starting to analyze properties and don't want to make any changes that would effect my pre-approval. 

I would re-run your numbers with you living there and see what's what.Lastly, if you can afford 5% down (and assuming you have good credit) look into a low-down payment conventional loan.

I would like to know if conventional loan would work or I have to get 203k.I would appreciate your advice if you can confirm if conventional loan would work.

It would be a lot easier to purchase a "turnkey" property for $475-500k.I always recommend Conventional over FHA, assuming you qualify.