As most have already said, as long as you haven't accepted them yet, you should be fine as long as your decision follows the Fair Housing Act (meaning you're declining of their application is not based on Race, Color, National Origin, Religion, Sex (including gender identity and sexual orientation), Familial Status, or Disability).