Robert,
I know EXACTLY how you feel. I was there myself about 7 years ago. I was a full-time employee at ExxonMobil and spent my evenings and weekends trying (unsuccessfully) to invest in real estate part-time.
I'd say that I was putting in about 10 hours per week reading online articles and attending local real estate meetings and another 10 hours per week putting out We Buy Houses signs, fielding calls, meeting with sellers, and making offers.
I did this for 8 months without buying a single property. Seems like every deal fell through for a different reason. I was extremely discouraged. But I didn't give up. Instead, I did something that many would think is crazy. I quit my full-time job and went full-time as an investor. Over the next 2 months, I spent 60+ hours/week pursuing real estate investing. During those 2 months, I purchased 6 properties and eventually made over $100k from those deals.
I'm not suggesting that you quit your job and go full time without ever having done a deal. But I AM saying that it worked for me. It was "sink or swim". I swam, and I think there's a good chance that you'll swim, too.
The reason I was able to quit my FT job is because I lived very frugally. I was making $55k/year at ExxonMobil and only spent about $15k/year. I had $60k in the bank from several years of eating Ramen Noodels and living in a tiny apartment. I knew that I could live for 4 years without earning a penny.
So, if you do decide to go my route, you'll need to dramatically slash your expenses in the short-term so that you and your family can survive for at least 3 or 4 years with no income. That'll give you the breathing room, and peace of mind, you need to make things work. In the end, the sacrifices you make today will likely pay off in spades.
I'm sure my approach isn't customary, but it worked. I haven't had a J.O.B. for 6 years now. I don't need one to fund my existence. And don't plan on ever getting one again.
Hope that helps!
Doug