This is one of the common issues with federation yes.
First please check if the comment (tree) was maybe done by an instance that we defederated with. In that case it is expected behaviour.
Otherwise it is likely an issue with the instances that are pushing the new comments to our instance. Normally they retry a few times if there is a problem delivering the comments to the shared inbox, but there is currently an issue in the Lemmy codebase (with a work in progress fix) that if the instance backend is restarted it forgets all the planned pushes and retries and thus some comments are never send to other instances.
However, I cannot fully rule out that it is a problem with our instance being unresponsive as I noticed that there is a small memory leak that causes a lot of memory swapping after a few days uptime and thus a noticable slow-down. But I still think it is more likely that the problem is on the other end.
If you notice a comment missing you can try to manually pull it by copying the full URL of the post (linked in that colorful Fediverse icon) and pasting it in the search bar here. That will usually make it available for everyone and you can reply.
Edit: also if you are the first person subscribing to a remote community from here, the old comments that predate that first ever subscription are not pulled automatically.