If you don't want to pay anything anymore, you probably at that time want to lower the death benefit so your cost will be approximately 0.25% a year (much less than a lot of mutual fund and financial advisor fee).Yes you can borrow on margin on a brokerage account, however you can not borrow up to 90% of it, and you can get a margin call with catastrophic consequences.

Additionally, you never know when paths will cross again maybe not in a working relationship but otherwise and they will remember and try to "settle a score" if there is nothing catastrophic I would chalk it up to a bump you climb on and move on.   

I also have a $175,000 line of credit at the ready if something catastrophic happened.

Some investment may have a high chance of loosing a little money  and some a low chance of problems but the rare problem could be catastrophic.

Violating those rules has catastrophic tax consequences, so you just don't want to go there.

Biggest challenge in my view is going to be managing the valuation side of the equation, lots of range of opinion on the property values, from flat to catastrophically down and a lot of secondary market buyers are skittish on valuations, especially on cash-out refinances

If something catastrophic ever happens, and say you have a fire, and it was found that someone died because you had illegal space in there etc etc, you would be put in a world of hurt.

I will "self insure" on the small stuff (hence the high deductible) but if something catastrophic happens, I don't want to be worrying about the economics as well as the rest of the mess.

And while sales are down, it's not catastrophically down. 

TLDR: Insurance is supposed to be for catastrophic lapse, use it that way.