Earnest money deposits

[Thai Nguyen]

So I am selling an investment property in Sacramento and after about 3 weeks, the buyer fell though. We had another back up offer so we decided to send the original buyer a cancellation stating we are going to keep their deposit because they wrote the offer as an all cash offer but did not state it was contingent on a sale of their own home. Upon sending the cancellation, the agent threatened to hold it up in court and then title said we can not move on with the sale to the new buyer until the matter is resolved.

My question is, how frequent is it that seller keep deposits without having to fight for it? What is the process say we did decide to fight to keep it or should I just give the money back and move on with the sale to the new buyer?