This is a really great thread, because there are so many perspectives and if you just pull the *facts* from those perspectives, you can learn a ton and apply that knowledge to your situation. Your goals, your vision for your life, your risk tolerance, how active you want to be with your investments, and so on, and so on. Look at YOU first, then look at the numbers.
For me, I was really down on my career, burnt out, and I started to read about real estate and it just clicked. I wanted financial freedom. I wanted to invest for the cash flow, but also for the fun of buying the assets, managing the improvement and value-add of those assets, and the rush of finding the next deal. For that, I needed cash. I borrowed the max from my 401k and used it for the down payment on my first rental property. It was a good BRRRR deal. After fixing it up, renting it out, and refinancing, I have less than $10k invested and earning a cash-on-cash return of 22% after expenses. The cash that I pulled out on the refi led to 3 more properties. I'm still paying 7% on my 401k loan, but making far higher rate of return on my real estate and I'm enjoying the hell out of it. I also stopped contributing (I got no match) so I could put the extra money into my real estate funds.
So I was willing to take a moderate risk with the 401k loan because I wanted to jump in to real estate and do the work and find the deals and learn how to rehab. I wanted to fully-immerse myself because I believe in it and I'm passionate about it.
Would I recommend everyone do what I did? Absolutely not. It would be terrible advice for someone who really doesn't want to spend time actively managing their assets. Like, if I didn't want to fully commit to being an active real-estate investor AND property manager, taking $ out of my 401k and abandoning it might have been a terrible decision.
Figure out what you really want, read up on how to get it, and the rest will follow.