The risk rises the more people do it as most residents don't want to share walls with vacationers, college kids of spring break, etc.

You probably won't be able to rent to vacationers or families but if someone is coming for a temporary assignment and isn't to squeamish I'm sure you could do it that way.

We have planned for a 50% vacancy in our vacation rental here, which seems to be very conservative, as we live within 30 mins of major ski areas and in the summer these mountains are filled with vacationers from NY, MA, CT and RI.  

The difference in our situation was that we bought and sold in a place where the locals either don't have money or interest in doing this type of work and many properties are owned by vacationers who don't want to do work either!

I was fascinated when he talked about renting to vacationers, businessmen, military, etc with his property in Manassas, VA.

It matters to plenty of vacationers, so keep that in mind.  

A lot of vacationers want amenities and the condos that come with those amenities typically (over-generalizing here) have higher HOAs.

And we converted the top unit into an Airbnb...so far we have not had any trouble getting vacationers..that being said we are marketing the unit as a fully furnished all inclusive trying to get a longer term tenant in there. 

@Nathan Gornto the Asbury Park market is doing phenomenal; the town continues to flourish and attract attention from both investors and vacationers.  

Most vacationers are coming out for weekend trips.