this post was submitted on 12 Jun 2023
142 points (98.6% liked)

Lemmy.World Announcements

29034 readers
2 users here now

This Community is intended for posts about the Lemmy.world server by the admins.

Follow us for server news 🐘

Outages πŸ”₯

https://status.lemmy.world

For support with issues at Lemmy.world, go to the Lemmy.world Support community.

Support e-mail

Any support requests are best sent to [email protected] e-mail.

Report contact

Donations πŸ’—

If you would like to make a donation to support the cost of running this platform, please do so at the following donation URLs.

If you can, please use / switch to Ko-Fi, it has the lowest fees for us

Ko-Fi (Donate)

Bunq (Donate)

Open Collective backers and sponsors

Patreon

Join the team

founded 1 year ago
MODERATORS
 

Thoughts? Seems like we are one of the only bigger instances with open sign ups still as well.

Edit: didn’t mean to start any instance tribalism. Like others have said, its not a competition, and its better that users are spread across instances than piling into one. Just think it’s interesting to watch the different instances grow and change and see where people end up congregating.

you are viewing a single comment's thread
view the rest of the comments
[–] deadcyclo 4 points 1 year ago (2 children)

Yeah. But horizontal scaling (well horizontal scaling in a system like this where you need clustering so the instances talk to each other) is hard. And I think there are a lot of other things that need to be polished, added and worked on before that. It would probably also need somebody with knowledge of clustering to start contributing. I think step 1 needs to be that the dev team needs more help properly tuning the database use. The database is very inefficient, and they lack the skill to improve it:

We are in desperate need of SQL experts, as my SQL skills are very mediocre. ^https://github.com/LemmyNet/lemmy/issues/2877^

So getting help improving the database is probably the #1 thing that can be done to deal with the scaling problem.

[–] lp0101 3 points 1 year ago* (last edited 1 year ago) (1 children)

I fully agree - there's no excuse why the DB should be falling over when I've seen a single postgres instance (with a read replica, granted) handle >1M users just fine.

Unfortunately my SQL skills haven't improved since my DB class in university, so I won't be much help. I'll be keeping an eye on the repo of course, and I can give some consulting/guidance or even open some PRs myself when they decide to implement horizontal scaling.

[–] deadcyclo 2 points 1 year ago

Yeah. I'm in the same boat. My SQL skills aren't impressive either since there are other people at work that handle optimization. Haven't used rust either (yet) so cannot really contribute there either. Though I'm considering potentially starting work on a cross platform mobile app. I haven't worked with mobile apps for a good six or seven years, so I feel like it's high time I get back up to speed. (But knowing me, I'll end up making something half finished and the start procrastinating)

[–] [email protected] 1 points 1 year ago* (last edited 1 year ago)

Yeah, I think a single postgres cluster with read replicas should be able to handle the needs of a single instance just fine. You can then horizontally scale the backend and frontend to keep up using containers.

Ideally, that instance can scale up to a few million users, and then federation can provide the horizontal scaling that takes the lemmyverse up to Reddit scale.

The backend just needs to handle databases better. Adding support for read replicas, making it more efficient, etc.

Not sure how well pict-rs scales, but that's probably pretty light already, vertical scaling might be good enough that it'll always be limited by the DB.

But yeah, I guess the worst case scenario is that Postgres doesn't scale enough and we need to switch to something like cockroachdb. Or go for snowflake uuids and noSQL like Twitter did back in the day.