this post was submitted on 12 Jun 2023
418 points (99.5% liked)

Lemmy.World Announcements

29163 readers
149 users here now

This Community is intended for posts about the Lemmy.world server by the admins.

Follow us for server news ๐Ÿ˜

Outages ๐Ÿ”ฅ

https://status.lemmy.world

For support with issues at Lemmy.world, go to the Lemmy.world Support community.

Support e-mail

Any support requests are best sent to [email protected] e-mail.

Report contact

Donations ๐Ÿ’—

If you would like to make a donation to support the cost of running this platform, please do so at the following donation URLs.

If you can, please use / switch to Ko-Fi, it has the lowest fees for us

Ko-Fi (Donate)

Bunq (Donate)

Open Collective backers and sponsors

Patreon

Join the team

founded 2 years ago
MODERATORS
ruud
lwadmin
lwCET
jelloeater85
Serinus
lw_mod_notification
418
How is lemmyworld so stable? (monero.house)
submitted 2 years ago by [email protected] to c/lemmyworld
116 comments fedilink hide all child comments
 

At the time of writing, Lemmyworld has the second highest number of active users (compared to all lemmy instances)

Also at the time of writing, Lemmyworld has >99% uptime.

By comparison, other lemmy instances with as many users as Lemmyworld keep going down.

What optimizations has Lemmyworld made to their hosting configuration that has made it more resilient than other instances' hosting configurations?

See also Does Lemmy cache the frontpage by default (read-only)? on [email protected]

you are viewing a single comment's thread
view the rest of the comments
[โ€“] [email protected] 16 points 2 years ago (1 children)

Likely experience and knowledge improving the quality of deployment. Most instances are likely underspecced, are on hosts that aren't easy to scale up with, or are maxed out in their current offering tier (lemmy.ml comes to mind there)

[โ€“] [email protected] 6 points 2 years ago (1 children)

I wouldn't be surprised if it has more to do with caching than throwing hardware at it.

[โ€“] [email protected] 10 points 2 years ago* (last edited 2 years ago) (1 children)

Looking at ruud's post, he moved the instance to a pretty beefy server - it sounds like a large part of the stability is coming from overestimating performance requirements.

[โ€“] mo_ztt 9 points 2 years ago (1 children)

* correctly estimating

๐Ÿ™‚

[โ€“] 314xel 2 points 2 years ago (1 children)

Correct. Lemmy is a monolithic application, so there's only so much a server upgrade can do.

[โ€“] PriorProject 11 points 2 years ago (1 children)

Lemmy is a monolithic application, so there's only so much a server upgrade can do.

This is sort of true, but not really true. The default docker setup is comprised of 4 containers. I've seen admins report that two of those containers (lemmy and lemmy-ui) can be horizontally scaled just fine. The pict-rs and postgres containers can currently only be vertically scaled, but Postgres natively supports scaling read load.at least through read-replicas, and there's an incomplete proposal to support scaling reads through separate db connections.

All of which is to say, it's possible to throw 4-6 machines at a Lemmy install. It's not truly a single-procees monolith. Would the Lemmy code be able to productively use all that hardware? I dunno. It's scaled better tombig hardware on lemmy.world than I would have predicted last week, maybe it can fully utilize a 6 machine setup, or maybe the db falls over first and you need to fix performance bugs because sn instance can scale to the user counts necessary to support bigger hardware setups.

[โ€“] [email protected] 2 points 2 years ago (1 children)

I wonder if you could replace Postgres with CockroachDB in this instance to scale out.

What is Pict-rs?

[โ€“] PriorProject 4 points 2 years ago

What is Pict-rs?

The image-hosting component: https://crates.io/crates/pict-rs

I wonder if you could replace Postgres with CockroachDB in this instance to scale out.

It's not a crazy idea, but it's not obviously a panacea either.

  • Cockroach is harder to admin and would only provide benefit on giant instances. It would be worse for small/beginner instances, which is the majority of them right now.
  • Cockroach doesn't support the full PG protocol, there may be porting issues.
  • Cockroach can have poor performance on some query types. I don't have deep knowledge of what kinds or if Lemmy would be impacted, but when I read about cockroach migrations it's common to find some performance footgun people had to work around. This can also add to the porting effort and could make it harder to develop for Lemmy on cockroach.
  • There's also AWS Aurora, for distributed PG-compatible dbs.
  • Finally Postgres scales pretty big. You can't run reddit or Facebook on a single-write leader with a bunch of read replicas, but you can run a pretty big website that way.

All of which is to say, maybe there's something there... but often distributed databases have fewer features and have bigger footguns than rdbms's. If you want to try cockroach or aurora, you don't need the devs help. You can stay to stand up a Lemmy instance that points at them. If the compatibility is really good enough, it will "just work" and you can try some performance testing. If it doesn't, then you have your answer than porting effort is requires for speculative benefit.