Not sure about Connecticut. But In California it should have been on the Seller Property Questionnaire page 3 line 16 - Neighborhood noise and nuisances. It has multiple explanations of what would qualify and odors is one of them. If Connecticut has similar contracts it should have been disclosed by the seller. From the conversation you had with your Realtor sounds CT likely does. However, the seller can claim they did not ever notice the issue so my advice is talk to your other neighbors to see if they ever heard of your sellers complaining about the issue. That could very well be your smoking gun and grounds for a lawsuit. Also make sure it is something that should have been covered in the seller disclosures.
California law is very buyer protective and Connecticut very well could be the same. You may have an argument where the neighbors smoke makes your home less desirable etc. You would need to most likely consult a real estate attorney but I have heard of buyers being compensated for much less of a non disclosure issue. Call me bias but I cannot stand the smell and I would be pretty upset as well in your shoes.
I am not a lawyer and not licensed to practice real estate Connecticut. This is not legal advice please consult the appropriate professional. Good luck.