I feel like by the time I can withdraw without penalties, I should have enough cash flow and appreciation that my "retirement" years should be taken care of.I'll have to see what happens when it comes to lenders, I guess.

Even then, the IRS may scrutinize the transaction if it appears the intent was to immediately convert the property into a personal residence.Additionally, if you eventually sell the property after converting it to a primary residence, you may qualify for partial benefits under the Section 121 exclusion for primary residence gains.

Even after moving 2.5 years ago, I continued to self-manage because I had a solid network of connections that made it pretty easy to handle remotely.That said, it eventually came down to wanting my time back.

Over time, they get that its not easy and a lot of hard work and eventually quit.   

Just withdraw it—it’s like a checking account.The Risks (Depends Who You Ask):A lawyer says everything can go wrong.An insurance agent says the same (so they can charge more).An engineer?

When the house eventually sold, I contacted the buyers agent and explained it to him so he could be the hero with his clients and show them how they could save a bundle on their property taxes by appealing and getting that corrected!

My dad used to manage other people's properties but eventually quit because the money wasn't worth it and he always ended up prioritizing their properties so as not to get any hassle from them.Another way to approach this might be to go all in on one project yourself and see how far that money gets you. 

They eventually declared bankruptcy.

This leads to state funded insurance which will be very costly because only those at highest risk will participate in it and if the losses continue, they eventually will need to get the money from someone (rest of people in california?) 

Value and rents should eventually increase to turn cashflow positive and build your wealth.You're next learning experience will be, now that you are out of day-to-day management, how will you be able to handle the inevitable expense challenges?