As @Erik Baumer is saying, how do you know she doesn't want to talk to you?
If he isn't the owner, it's very logical that he won't allow you to talk to the actual owner and will say she doesn't want to talk to you...
It sounds like this guy is one of those who just need an ax to grind and an insecure ego with a need to assert himself as someone important...
This guy can't sue you for something he is claiming you did to his property because it's simply not his property. The owner is the one who would have to initiate legal action if she has a problem with your actions.
I would circumvent him and go straight to the owner. Until you actually speak to the owner, you don't really know if a lawsuit is a actually a real possibility. Maybe she doesn't even know what's going on in the first place.
You shouldn't even be discussing it with him and tell him so because he isn't the owner.
Just go behind his back and get through to the principal here first so we know were things actually stand here