@Robert KhederianI think 3-units to start, even 4, is pretty manageable if you're buying in a B or better neighborhood and B or better condition property. Pay attention to the tenants you'll be inheriting. For me the tenants are just as important as the property. Do they take care of the property? Do they seem to be on your side, or do they try to take advantage of every situation, aka "professional tenants." Stay away from professional tenants.
Also keep in mind that multifamily properties cost not a whole lot more than a single family homes. Often the case, multi-families cost less per square foot (but by definition multi-families aren't in prime A neighborhoods either) However, by buying single family first, it takes a lot longer to acquire your second property than if you bought multi-family first.
To help put your mind at ease,perhaps you could read one of the many landlording books out there. While they won't go into as much detail as the info you can find on BP, they can give you a general picture of the kinds of issues that landlords sometimes have to deal with. (I like The Landlord's Survival Guide by Jefferey Taylor) You can always get a PM, but in my opinion, they're not worth it if you plan to live at the property. I'd also recommend sitting down with a landlord or two, especially those who manage their own properties. If you pick your property right, with the right tenants, the right lease, and with good systems in place, it's not really that bad at all. There will be times you will need to take a day off to take care issues. But in my experience it hasn't been very bad at all. I've had to take a half day about twice a year, mostly because the vendor wouldn't agree to come out on a weekend.
Lastly, I'd recommend having a handyman ready and if he can do plumbing even better. Also, have a few thousand in reserves. Money cures all, including stopped up toilets, or whatever, in the middle of the night.
