It could just be delay from the overload, but it could also be a side effect of federation. Federation is a voluntary relation. There's a solid chance that set of federated instances is different for lemmy.world than it is for lemmy.ml. It might seem like that doesn't matter so long as they're federated with one another, and you're looking at a post from one or the other. The catch is that, if they aren't federated to all the same instances, lemmy.world may not see all of the votes that lemmy.ml sees—specifically, the votes from servers not federated with lemmy.world. And that would explain the divergent results between the way the two servers rank posts.
This is due largely to the way that ActivityPub passes interactions, and where those interactions are stored, and how. There's analogous behavior on other ActivityPub services, like Mastodon, where like totals differ depending on which instance you view a post from. And there are probably ways to fix it, but they would either requires some degree of centralization, or major changes to the protocol.