Why accept colleague requests?

[Eric M.]

I get alot of these and many without any message and I don't know who they are or what they want. I am not sure why are requesting. (I guess I am not much of a "networker")
My email address is in my sig and I would much rather them send me an email than a PM.
Should I accept? Is there a "risk" to accepting? I don't want to offend by denying or ignoring. Do they get a notice that I have denied them?

Thanks