It was a deposit for the appraisal. By giving them my credit card info over the phone they also locked the rate. They assured me on the phone that if the purchase did not happen I would get the deposit back as long as the appraisal was not done.
Here is what the deposit agreement says:
The deposit agreement says it is a good faith deposit. "Your deposit to cover the costs to process your application and pay for services from outside companies, such as your credit report, appraisal and title work. In order to process your application quickly and efficiently, this work (and the expenses) will begin immediately after receiving your application. In doing so, we rely on your commitment to continue the loan process. Therefore, you cannot cancel or rescind this deposit agreement.
Your deposit may or may not be refundable, depending on the status of your loan.
When you close your loan with us, your deposit is fully credited to your closing costs. The credit will be reflected on your closing statement.
If your application is denied or withdrawn, for any reason, we will refund your deposit minus the actual cost of your appraisal and credit report"