Are these GSE loans (Fannie or Freddie or USDA)? If was originated as owner occupied in the past and no longer is owner, if you have change of name on deed or insurance, if impound had a dispute, claims - are a couple reasons why the servicing changes. Some servicers label your loan as troubled and they sell it off ASAP. Servicers have merged and operate on thin margins.
Refinancing with a credit union or bank doesn't solve the problem. Most lenders do not service their own loans and reserve the right in the future to sell it- even credit unions and 99.9% of banks.
How to avoid BEFORE this happens: pay a day early, don't raise issues, get loans that aren't backed by US Government that the note states they don't sell the servicing.
How to avoid the late payment- pay all accounts from the same bank account on the same day of the month. Monitor the account that all x number of payments went out. Keep one mailing address and opt to receive digital notice to one same email account that you monitor. IF you made payment to old servicer new one cannot show you late they have to transfer the funds along- there is a 60 day grace period. Fix it before the next month.