Basically when you look into home warranties you want to check their fine print. Are they a exclusion, or inclusion type company. Exclusion referring to coverage excluding the following items, or coverage for or including the following items. Understanding what is likely to fail, your access to reliable contractors, and your risk tolerance are the factors that help drive the decision. Its insurance, if you get stuck with out it, it burns, if you have it and don't use it...welp its there just in case.
I definitely think that its a benefit, I'd make sure that you express to your tenants that you are using a home warranty company and that it may delay how fast and the process for repairs. Also how repair deductibles are paid and contact with the contracted companies is managed for access to homes.'
I did get burnt recently on a rental where the water heater went out and the plumber that they sent out said the the failure occured before the home warranty was purchased. I ended up having to spend money to purchase and install my own water heater, and still have a $600 home warranty on the home. :/