So, I want to own 500 rentals, but how do I get there?

[Brandon Schlichter]

Since getting in real estate 11 years ago, I knew I wanted to buy rental properties. Then in 2013 I finally got the shot to do it after finally finding a private money partner to go in with and start buying. Since July 2013 through today, I've purchased 51 total rentals, all 1-4 units mostly (About 50% SFDs, the rest mostly 2 units). 

I'm now in contract for more rentals to take me up to #54.

The problem I've had since the start, and continue to have is funding - Banks really are skittish on dealing with me on the lower-end properties that I tend to buy. I think our average property value is in the $75k range. After 2-3 years now of seasoning, a few lenders are starting to warm up to me, but the ones that do want to lend also want to limit the total number/amounts of properties they'll loan to. So, I feel like I'm in a never-ending cycle of trying to find more money to buy with. The management aspect isn't too difficult, dealing with contractors and labor isn't too bad, but the money situation is the one that always bothers me.

I know the traditional answer is "Why the heck would you want to buy small stuff when you could buy ONE apartment complex and do it the easy way" - My answer is that cap rates for that kind of stuff is less than half of what I'm making on 1-4 units in my area. At some point I realize I'll have to hire management help, but my cap rate still is superior to a larger apartment complex,  so to me this route does make a great deal of sense.

All my deals sans one has been just cash purchase. I've been trying the BRRR route since day #1, but the refinance aspect has been terribly difficult, out of all my rentals I have been able to refinance 6 rentals with another ~8 in the process of being refinanced right now. So, I continue to chug along with more private money which I pay closer to hard money rates on. The flexibility with cash is awesome and I get smoking hot deals all the time. I realize the option here would be take money, and shoot for a boatload of seller finance deals, but I don't find those as much as I do distressed sellers needing cash ASAP.

So, in my shoes, what would you consider? Chugging along slowly refinancing properties over time? Some magic source of cheaper money? Or would you bite the bullet, give up the dream and buy larger commercial-type apartment complexes where commercial loans are much easier to obtain?