Using the contractor you work for to build real estate?

[Preston Painter]

Hi first time posting here. New to real estate investing but I've been learning for about a year now and have an LLC formed.
   I’m curious if anyone has any advice on my specific situation as I feel it’s very niche. My partner and I have secured funding for a 20% Construction loan and land. We are looking to build two duplexes on adjacent lots. As it turns out the contractor we want to use we both work for as our W2 job. We could potentially save a lot of money by doing most of the smaller aspects of building by ourselves. How will this affect the loan if we come under budget? Is there a conflict of interest here? Or is this a great opportunity?