The handyman gave me a quote for $5000 and $300 for materials.

You need a good lease agreement, landlord forms library, educational material, and landlord attorney to guide you. 

The quote was $5352.00 ($3,650 for labor and $1,702 for materials).

That property was run down, smelled funny, and it looked like the cheapest materials were used in there which are just going to fall apart year after year. 

What if the new owner backs out of the job and gives it to their "regular guy", will the original contractor come after me for any expenses/materials spent?

Contractors don't make a living off marking up material prices.

The material is interesting, there is no upsell, and you do get a fancy certificate by email.

@Bernard Reisz To clarify on your comment regarding rental property ownership and income not qualifying for a Solo 401k, is this the case even if you qualify for "material participation" in your properties?

Your mortgage payments may be fixed, but your taxes and utilities will increase, your maintenance material costs and labor rates for repairs have gone up, and your property is wearing out (appliances, flooring, water heaters, furnaces, roof, etc. all have a lifespan and will eventually need replaced). 

If you have given the contracdtor a deposit and that deposit has been used to purchase materials, & those materials are in your property I would;1.