Maybe if you had a really large down payment and a free and clear primary etc. 

I borrow from private lenders all the time and its my primary source of capital for the rentals to do.

If you are looking to make a sound investment, which is what most people want when they come here, then house hacking is your best option for what you are asking since you may want to use it as a primary.

Lenders want the Title search done right away, as that is they primary concern (ie. out of their control). 

If the home is paid off in CO I would leave that alone unless you were to sell and reinvest into something else, especially since you are buying primary residence to live in. 

For example, if you're interested in flipping, you might try buying a fixer-upper as your primary residence and trying to renovate in a timely manner.

I have yet to purchase a primary residence so I can do 3.5% down with fha but then there are closing costs.

Buy your next primary home conventional and it won't matter that you have an FHA mortgage.

So, no one group is the primary focus of the laws. 

I am now looking to purchase a 2-4 unit property as a primary residence in NNJ.