Do you know why this seller is selling? The seller's motivation might help explain their reluctance. Some people, rightfully or not, have an extremely pessimistic opinion of real estate brokers due to a past negative experience (I admit, in my market a good agent is hard to find). Other people are hiding something that a trained professional would spot from a mile away i.e. deferred maintenance, title issues, etc.
I would ask this seller what prompts their misgivings about dealing with realtors. Are they willing to let you engage your own appraiser, home inspector, and title insurance company as part of your due diligence and it's only realtors that they're skeptical of? If the answer is no and they just want to sell fast buyer beware I personally would not want to move forward but it's not my money.
I would say the next step is to continue a dialogue with the seller and try to understand their perspective and lay your own concerns on the table. Be patient with the seller trust your gut. If something feels off don't feel obligated to move forward. Good luck!