Does buying houses NOW make any sense still given lower returns?

[Jack B.]

When I was able to make 50K a year between cash flow, appreciation and principal paydown on a 60K down payment, it absolutely made sense.

Now that the down payment required is 150K a year and I'm looking at 50K a year returns on that dp, it's less attractive. I'm basically making 33% on my money, whereas I could get that in an index fund with less hassle and no effort. 

Yes, the house will be paid off at some point, etc. I've been buying up like crazy lately due to what REI did for me over the past 10 years. But the returns are much lower now that I have to put in so much money each time.

When it get's to the point I have to put a 200K down payment in to make 50K a year returns, it doesn't make any sense anymore.