If it is to have a vacation home that someone else helps pay for then maybe.  

The best thing to do is approach them in person and inform them you are the new owner and that you will need them to vacate.

@Ryan LeonardYour lead flow may have slowed down due to summer coming to an end, people on vacation and back to school.I think your best ROI is to create and maintain relationships with brokers.

Please note, units have to be registered (then renewal annually) with the Department of Planning and Permitting in order to be operated as legal vacation rentals.  

We couldn't do a traditional second/vacation home loan on the unit because the condo is non-warrantable.

And please note, you'll also have to pay GET, TAT, OTAT taxes if operating a vacation rental.  

My realtor told me that I could negotiate into my offer for the seller to buy them off or give them a vacate notice...interesting. 

(Yes, Colorado Springs also allows them in non-owner occupied homes but with limitations that in practice make it nearly impossible.)I still think this change is good policy, but for anyone wanting a vacation rental in municipalities that have a primary residence rule, this doesn't change anything.

I may need to pick your brain about the best places in Japan to visit for a family vacation. 

I originally invested strictly as a vacation home, with home of price appreciation.