Tenant occupant of a house refuses inspection access

[Tom Shi]

My offer to purchase a house has been accepted, but as more information have emerged about the house and its occupants, I have major concerns now.  This is a tenants occupied house that I intent to use as my primary residence.  

There are multiple unrelated adult people living in the house but only one person is on the "lease." Quotations marks is there because I am not sure if there is an official lease contract or just a informal arrangement between the seller and that individual.  The people occupying the house has been there for years, and it is my understanding that would qualify them as tenants legally even if there are no paper contracts.

The offer contract I signed states that the seller is responsible for vacating the property before closing and there is specific language there indicating that closing is dependent on proof of an vacant property.  

In Portland, Oregon where the house is located, there needs to be 90 days notice of termination to tenants when there is a sale of the house. In order to meet the closing date, we heard through the listing agent that the seller is planning to offer cash-for-keys to the occupants of the house to ensure early move out.

Now I am finding out that the people that live there maybe reluctant to leave and it is unclear whether they would be receptive to cash-for-key. Also the inspection is scheduled in a few days, and I heard that some of them may refuses access to their bedrooms despite 48 hours advance notice.  

The more I learn about the situation the more messy it is.  My question is that if some of the occupants refuses inspection access to parts of the house, is this ground for terminating the contract due to inability to perform full inspection?  I know that there is a good chance that the seller would be unable to vacate the property before the closing and honestly if that's the case I am comfortable just let the house go.  The only reason I made an offer is that is a cheap place in a good neighborhood and I have been trying to buy in a red hot housing market for almost a year. I am exhausted and just want the process to be over with.  

Anything else I should be aware of to protect myself in this situation? 

Thanks