I financed this property using down payment funds from 403B and a local commercial bank.

What would stop him from just not coming back to finish once he gets the last payment from you?

The cost associated with insurance for not having 20% is peanuts compared to what you potentially can do leveraging your down payment. 

A single-family home or small multi-unit property can be a great way to dip your toes in the water.Build a Team: Even if you’re planning to handle most things yourself, having a good real estate agent, property manager (if needed), and a reliable handyman can make life so much easier.Learn the Local Laws: Pensacola is pretty investor-friendly, but it’s always good to know local zoning laws and short-term rental regulations if you’re thinking about Airbnb or VRBO.Run the Numbers: Make sure you’re clear on the property’s cash flow potential.

Low or no down, seller financing and low or no interest with a balloon payment later is the holy grail (location, building condition, market also matter but it sounds like you have all that).

If you happened to be in Jimmy’s part of Florida, and he was home, you could go over to his house, where you’d have a beer with him and personally learn “working a note” , “using recreational lots as down payments” and “real estate exchanging” right at the “masters” feet.  

I think it was for down payment and closing costs.

In this type of situation, they want low interest rate with low down payment also.

If the other two parties want to flip a house maybe they can do just that as partners and then sell you the house which you can buy with a primary home loan with low down payment.

To qualify for a loan, focus on having a credit score of at least 620 (or 580 for FHA loans), keeping your total monthly debt under 43% of your income, and saving enough for a down payment and closing costs—about 5-10% of the home's price in total.