Agent said to skip asking for the permit

[Quynh-Chi Nguyen]

I am currently under contract for a single home. It will be my primary resident. The property does have a one car garage that is not listed on the property card. My lender requires the seller to provide permit and have it up to code with the town sign off. Both listing and buying agent insist that I can still close without a permit. I said I want the permit because I do not want the town to come after me after the sale. Is it a valid argument? Can the seller walk away from this contract? I know I can but I have spent $1000 for inspection and appraisal already. Maybe I should? Any thoughts here? Thanks -Chi