It is hard to say if paying points makes sense in every scenario. But just off of averages I would like to help people understand if it makes sense to buy down a rate.
The average person in the U.S. owns a home for about 10 years. That average changes all over the U.S. If you can recoup the cost of buying down the rate in 5 years, I think that it is always worth a look.
If you are deciding whether you should buy points or use the money as a down payment, that too is about your situation. If you plan on doing a cash out refinance later on down the road, a down payment would get you closer to the 75 - 80% LTV's that you need to do a cash out refi.
Cash out refinances usually have a rate hit as well. So getting a lower interest rate in the beginning, only to have it jump a point after doing a Cash out refinance would be counterproductive. A larger down payment would be more helpful than buying down the interest rate in that case.
But if you want your property to cash flow as much as possible in the very beginning, then a rate buy down would produce that result better than a larger down payment in most cases.