Robert:
Here are my thoughts, there is "being right", there is "doing the right thing", and then there is the in between; let me explain.
If there is no express provision within your lease agreement and no express law stating not to count holidays, then clearly you are "right" to charge a late fee.
Then there is the "doing the right thing" where they are long term tenants, never have been late, are going to be tenants for years, and have apologized for their tardiness profusely; then maybe the right thing is to let it slide this one time. Its tougher to decide what to do when the tenant is new but again apologetic to you for their tardiness, yet you see them as potentially as a long term tenant.
So the question then to ask yourself is are you willing to be unforgiving about a "late fee" of $25, $30, or $50? Does you being right mean you need or are squeezing every dollar out of the tenant so that they are out in a few months when the lease ends? (What are you repair/clean up costs, holding costs for a new tenant, advertising etc.. going to be once they move out? Because lets face it there is always a cost when the tenant moves out.)
So here is what I would suggest if the tenant is relatively good and might be a long term tenant, I give them the
"As you are aware, your lease provides that rent is due on __ and is late on ___. This done in conformance with State and local statutes (Not sure if you have these). I did not receive rent within the required time frame therefore your payment on ___ was late and you have been assessed a late fee. However, so that we will continue to have a strong landlord and tenant relationship, I am going to waive the late fee for this occurrence only. This is not to be interrupted as any sort of mortification or change in the terms of our lease agreement; rent will continue to be due on _____ and considered late on ______ and it will be assessed moving forward. Should you have any questions, please let know. "
Obviously, if this is just another occurrence and they are playing games them hit them with it - I would.
No matter what type of tenant they are, I would send something to the tenant regarding your assessment of the late fee or waiver so they are clear that you are the Sheriff in town and wont take to such shenanigans.
Good luck
Steve