Joe, I was able to build a nice portfolio without too much money down in my late 20's. This has all been written by others elsewhere, but here is what I did: buy a single family home as your primary residence with a 5% down conventional loan. With this loan you're required to move in and stay there for at least a year. If you don't mind having roommates, get them in there and they'll cover your whole mortgage payment while you save up for the next down payment.
After one year, buy house #2 the same way and move in for 12 months. Get roommates if at all possible. Keep repeating! If you buy houses in good condition, you should be able to manage them without hiring a manager, and you'll learn some handyman skills as you go along with the help of Home Depot and Youtube.
There's much more to it than that, but that's the basic roadmap. It's a bit of a pain, but it's the best way I know of to get into the game with little money. My roommates who thought I was crazy for taking the smallest basement bedroom in the house to save up the next down payment are still renting.