Not CAN, but SHOULD I use my 401K?

[Bill Brady]

I've tried to find the answer to this question over the past few days and have been unsuccessful. Everywhere I've read, including on BP, it's a "YES!!!...you CAN use your 401k to invest in RE" (rollover SEP-IRA, borrow, cash out, etc.) but nowhere can I find out if you SHOULD use it. I'm curious as to whether the longterm impact on taking money out of your 401k would somehow negate any return on investment you receive from a cash-flow positive rental.

If all numbers work out with a 15% CAP rate, $1000 cash-flow, 40% cash on cash return, etc....is there a true downside to using your 401k as a down-payment on a property? Are their any horror stories out there?