sh.itjust.works Main Community

7584 readers

11 users here now

Home of the sh.itjust.works instance.

founded 1 year ago

MODERATORS

[email protected]

Anyone else notice a long delay for comments/posts from lemmy.world? (sh.itjust.works)

submitted 7 months ago* (last edited 7 months ago) by [email protected] to c/[email protected]

11 comments fedilink hide all child comments

It might not just be lemmy.world, but several times recently I've made posts (usually to !helldivers2.lemmy.ca) that I check throughout the day. Then like 8 hours after I've made the post, I'll see comments appear from people on lemmy.world that are several hours old.

It seems like there can be a queue that gets build up and then is suddenly flushed.

I see it happening the other way as well. This morning I made a post from my sh.itjust.works account and then switched over to a lemmy.world account and can't see it. So the delay is going both ways.

top 11 comments

sorted by: hot top controversial new old

[–] [email protected] 4 points 7 months ago (1 children)

I'd like to celebrate early and possibly jinx myself in the process but the line is going down.

[–] [email protected] 2 points 7 months ago

Lol the way that's dropping I'd say a celebration is in order!

[–] [email protected] 2 points 7 months ago (1 children)

We are currently seeing a backlog in outbound federation towards lemmy.world.
Really, this graph says it all... 📈

I've restarted the containers, hoping it starts catching up.

[–] [email protected] 0 points 7 months ago (1 children)

Oh yeah, that looks pretty telling. Also, just wanted to say thanks for how awesome you guys are on the admin team. The work you guys do and how responsive you are is freaking impressive. Thanks!

[–] [email protected] 1 points 7 months ago (2 children)

Sadly that line keeps going up.
I don't have much of an update but it's really specific to outbound federation and just with lemmy.world.
ie: inbound works fine, outbound with other instances works fine.

[–] [email protected] 1 points 7 months ago

Hmm sounds like it's a problem on the lemmy.world side then.

[–] [email protected] 1 points 7 months ago

https://github.com/LemmyNet/lemmy/issues/4529

What's your ping with Lemmy.world?

[–] [email protected] 0 points 7 months ago* (last edited 7 months ago) (1 children)

I'm experiencing the same and only noticed it when posting to lemmy.world communities. I'd navigate to their instance directly and see both comments that didn't federate yet and my comment missing. I haven't observed this on other instances though, lends me to believe this might be just a lemmy.world problem?

Edit- have an example here (open in browser) https://lemmy.world/post/13836315 https://sh.itjust.works/post/17236782

Note my comment still missing from their instance nearly 24h later

[–] [email protected] 0 points 7 months ago (1 children)

I think I saw somewhere that lemmy.world has grown way bigger than most other instances - probably because the name is less confusing to new users. I'm wondering if they're running into scaling issues as a result.

[–] [email protected] 0 points 7 months ago* (last edited 7 months ago) (1 children)

Some of the instance admins have been poking at this lagging federation issue for a few weeks now, trying to figure it out.

The Reddthat admin noticed that Lemmy's federation process can't seem to meet demand in some cases. Reddthat has had trouble staying current with lemmy.world due to the network latency between Europe and Australia: https://sh.itjust.works/comment/9807807. I understand that lemmy.nz has seen this, too.

Another thing that has been noticed is spamming of actions from kbin instances. It looks like some process gets stuck in a loop on the kbin side. https://lemmy.world/comment/8961882

I'm sure there are other contributing problems that still aren't well understood. This software is a work in progress, after all.

[–] [email protected] 1 points 7 months ago

These posts are enlightening, I think reddthat is right lemmy will need to using batch messages on longer intervals soon, that's pretty unsustainable in current form. What an interesting scaling issue to encounter.