I don't know WHAT some of you guys are talking about. Make no mistake - during the crash RENT WENT UP. The % of people renting... WENT UP. Where do you think all the people that lost their homes went??? To live on the moon? Come on...
Regarding risk. Provided you are not holding properties in an area with insanely overkill tenant-favoring laws (NYC for example) and you do your numbers correctly, I don't think you have to worry. If you have properties in a normal area with reasonable landlord-tenant laws, and you have to evict someone, it isn't going to bankrupt you. (But in NYC it could take 1 or 2 years to get them out, so yeah in THOSE SELECT AREAS you could have a major problem.) But I absolutely do not suggest investing in those kinds of areas so to me that's an irrelevant concern.
Bank calling the note due because you lost your tenants? I am going to have to call BS on this one. Whoever told you that is lying or misinformed... Please show me the page of a mortgage agreement where it says they can call the loan due for this!
If you do your analysis and numbers correctly before buying, and you aren't basing them on some pipe dream of a rent figure (ie expecting to get 1000 a month for a unit that will really rent for 600) then you are not in some hugely dangerous, risky situation.
Once in a while, you'll have an unexpected repair. Once in a while, you'll have a vacancy for a month or two. But unless you are doing something VERY wrong, you aren't just going to suddenly lose all your tenants and not be able to get new ones for 6 months. That's absolutely insane, and whoever is leading you to believe this is just trying to discourage you.
If you lose a tenant and the place doesn't rent right away, you can hire a broker to help you rent it, lower the rent a small amount, etc - seriously just keep the property in good condition, make sure you don't overpay when you are buying it, and make sure your numbers WORK. There should be plenty of money left over after paying the building's expenses every month, for you to save some toward future repairs and pocket some as profit. If that's not the case, IT'S NOT A GOOD DEAL.
Also, having more units (ie several houses, or a multifamily, etc) spreads your risk around. If you have one SFH worth 500k and it becomes vacant for 2 months, obviously you will feel it more than if you had 5 100k SFH's and 1 of the 5 became vacant for two months... something to consider since you seem risk-conscious. With a good duplex or triplex, you can have a single unit be vacant for a little while and still break even as the other tenant's rent should cover the mortgage, tax, and insurance.
Bottom line is, honestly you have to be careful who you take advice from. A lot of people will try to discourage you from investing or taking risks, and it's usually NOT because they are more well-informed than you. It's either because they are brainwashed into being terrified of debt, or they are jealous of the possibility of you making a ton of money and getting ahead of them in the world. It's sad but it's human nature - read a few books on human evolution and psychology. We are basically wired to try to hold everyone else back and just get a little bit ahead of the pack by any means necessary. When someone steps up, a lot of people in the crowd will try to pull them back. I'm sure you've encountered this in the office too, if you've been working a while.
*I am not a lawyer or financial advisor etc, obviously these are just my opinions and you need to make your own decisions.*