You need to assess the extent of the problem: a mouse, a mouse family, a mouse army? The smaller scenarios are usually easiest dealt with by traps and bait. The latter may require an exterminator, sealing of the unit and a larger expense. Of course, all this is in addition to addressing the tenant's hygiene as mentioned.
I spent about $1,200 on Terminix over the past 12 months or so on one property getting the house sealed and doing treatments. While there were gaps for mice to get in, the tenant was definitely a major factor and still is. I just ended the ongoing service and told her mice are her responsibility, as per the lease. Unfortunately, she is terrified of them, so I don't know how this is going to turn out. It seems she was in the "small family" category as the technicians never saw evidence of anything near the volume she was reporting.
I did learn the Teriminix $800 house sealing treatment can be had for much less if you get the Home Depot squirt gun and caulk-like sealer, then patiently explore every crack in your foundation. Terminix did little more, but they had larger, fancier squirt guns and a professional grade caulk with poison embedded in it. The lead tech recommended I go with the Home Depot treatment in the future to save money.
I am interested in advice as well as I don't know who to address the "whole problem" efficiently when tenant lifestyle is a contributing factor. In my case, even though the tenant had the "call anytime" service, she never called them; but did regularly complain to me about mice.