I dunno, first thing I can think of is the chief stepping down after some change in the rest of the town's authorities would have forced him into unethical or illegal acts if he stayed on, and the rest followed him for the same reasons. Which is a sorry future for the town.
A chief stepping down by himself would not be weird. every officer following is the weird part, so it can't be something minor like him aging out or whatever.
The flip side is worse, but optimistic; where that chief had to step down because he'd been found out having carried out bad acts, and the entire force was involved. This would mean that there had been years of fuckery involved.
It could be some kind of union like issue where the police involved were bargaining collectively for something that isn't public knowledge, and weren't bluffing about quitting if they didn't get that "thing".