Jonathan - certainly different perspectives on this. One school of thought says that you should never put your retirement savings at risk and the other camp argues that investing in yourself with higher returns makes more sense. I've always been pretty financially conservative myself but as I've gained experience in real estate I've shifted to the latter camp and have taken numerous loans from my 401k to fund real estate deals.
I think it all comes down to your personal risk tolerance and life situation. If you're young with few responsibilities/obligations, you may be willing to take more of a chance with your savings. If you are closer to retirement and can't afford a potential loss to your savings, you may not want to go this route.
I've been investing in real estate for ~8 years so I have built up a level of experience that gives me confidence in my ability to far outperform the stock market through real estate investments so I take as much as I'm allowed in 401k loans. Think of it in these terms - I can borrow money from myself at 3.75% or I can borrow it from a hard money lender at 10%+points. Far cheaper for me to borrow it from myself.
Just make sure you distinguish between a 401k loan and a disbursement. If you withdraw money from your 401k you will be hit with an early withdrawal fee (10%, I think) and it will also be taxed at your personal income rate. For me that would translate to something like a 40% hit on my funds. (Finance guys, chime in here if I'm off in my assessment)
I haven't been able to convince myself to take that kind of hit yet even though I can get 100%+ ROIs on my BRRRRs. I'm still a little conservative, I guess. There are a few other considerations as well. First, you can only borrow a certain percentage of your 401k (10%, I believe). Second, if you leave your job you will be forced to repay the loan within a certain period of time or it will be taxed as a withdraw. Don't quote me on this but I think it is 60-90 days to repay. This is just something to keep in mind because once you become financially independent through real estate investing and you want to leave your W2 job, you will just need to set aside some money to repay the 401k loan.
I hope this helps. Good luck!