It depends on what kind of loud it is, what hours it is happening, and to some extent how well the building buffers for noise.
If there is noise from children you can't evict for that. The law recognizes that kids make noise and there isn't much to be done about that. If there isn't much to insulate the noise from one apartment to the other than it may genuinely be a case that the tenant isn't playing overly loud music. I have been in places where I was able to hear the next door neighbor having a normal volume conversation through the wall. If that is the case I would think about getting some insulation blown in.
Then there is the time. If the music is happening during the day and evening, and not going past say 9 pm, that seems pretty normal.
Also, the leaving tenant could just be hypersensitive. Some people think they are entitled to silence. Good luck to them.
I wouldn't necessarily penalize a good tenant without exploring all of the options.