Secondary home on a duplex

[Kao Nou Xiong]

With secondary homes, banks usually require a 10-15% down payment with a requirement of living there for at least X weeks each year. 
With that said, has anyone done this before? Getting a secondary home with a 10-15% downy payment is ideal, and live there temp or visit the location for X weeks while renting the other side. 

Any thoughts on if this is possible?

Thank you.