Hi there. I work as an appraiser in the Bay Area, CA. I don't know your market but I will try to help. First, you said that you had two separate quotes on your appraisal fee. If you went with a conventional lender most likely they use an AMC (Appraisal Management Company). The AMC's act as a middle man between the lender and the appraiser. They provide the firewall but take a hefty fee. So what you pay for the appraisal fee is not necessarily the same as what the appraiser actually makes. I hear some struggling appraisers only make $150 per report. The appraiser, in most cases especially in Ca, is not even allowed to attach an invoice to the appraisal to show what he or she was paid.
Secondly, $450 or $500+ is not a typical exterior drive by fee. An exterior appraisal will normally be done on a 2055 form as opposed to a 1004 form for a full blown report with interior inspection. If it was done as an exterior appraisal you should ask them to upgrade it to the full inspection (they will charge you more for this). Even if it was done as an exterior the appraiser has to verify where his data is coming from. It would normally come from public records, local MLS data, google maps, and the homeowner (you). I always call the homeowner before I do an exterior and ask them to give me detailed information about their property. If they went above and beyond what is typical I always recommend that they upgrade to a full blown appraisal. Were you not there during the inspection? Normally the appraiser will call to set the appointment with the borrower and ask for access.
A couple other things to note. Some lenders will question the appraisers value when they first receive the report. They will sometimes ask appraisers to look at certain comps that may or may not be relevant. If most of the recent comps are in the $30k range then they will want to see that or have an explanation why yours should be valued above it.
0.7 miles can change values a lot. In my area, you can cross into another neighborhood just by crossing a street where home values double and school ratings change dramatically.
If you feel that the appraiser overlooked comps more similar to your property you should compile a list and send it to your lender. You can always challenge the appraisal. Tell the lender why the comps the appraiser chose were wrong and why yours are more relevant. Make sure the lender is using a local appraiser, not somebody who is coming in from another area who doesn't know your market.
Sorry for the long post. Just giving you my perspective. I wish you luck and hope you get things straightened out.