this post was submitted on 01 Jul 2023
86 points (97.8% liked)

Asklemmy

43974 readers
2021 users here now

A loosely moderated place to ask open-ended questions

Search asklemmy 🔍

If your post meets the following criteria, it's welcome here!

  1. Open-ended question
  2. Not offensive: at this point, we do not have the bandwidth to moderate overtly political discussions. Assume best intent and be excellent to each other.
  3. Not regarding using or support for Lemmy: context, see the list of support communities and tools for finding communities below
  4. Not ad nauseam inducing: please make sure it is a question that would be new to most members
  5. An actual topic of discussion

Looking for support?

Looking for a community?

~Icon~ ~by~ ~@Double_[email protected]~

founded 5 years ago
MODERATORS
 

I already get rate-limited like crazy on lemmy and there are only like 60,000 users on my instance. Is each instance really just one server or are there multiple containers running across several hosts? I’m concerned that federation will mean an inconsistent user experience. Some instances many be beefy, others will be under resourced… so the average person might think Lemmy overall is slow or error-prone.

Reddit has millions of users. How the hell is this going to scale? Does anyone have any information about Lemmy’s DB and architecture?

I found this post about Reddit’s DB from 2012. Not sure if Lemmy has a similar approach to ensure speed and reliability as the user base and traffic grows.

https://kevin.burke.dev/kevin/reddits-database-has-two-tables/

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 25 points 1 year ago (5 children)

Poorly. Lemmy will scale poorly.

I won't be surprised if the larger instances start locking down more as a way to sustain themselves, like restricting communities or only allowing text posts.

[–] nyakojiru 8 points 1 year ago (1 children)

Sometimes you have just to accommodate to the situation and keep going until it settles down. The error I think here is thinking something can’t have flaws and issues, even more if it’s not behind a corporations. And no one wants corporations.

[–] [email protected] 9 points 1 year ago (2 children)

It isn't about accommodating to the situation, but planning for long term growth.

Right now, instances of Lemmy don't have any way to fund server costs other than asking for donations. Outside of Wikipedia, that isn't a sustainable business model. How is Lemmy supposed to survive if, every time a sub gains critical mass, it shuts down?

[–] ritswd 5 points 1 year ago* (last edited 1 year ago) (2 children)

planning for long-term growth

Which is part of any scaling effort, and you can’t really guess through predicting and resolving bottlenecks, it takes some serious expertise. And as far as I know, the Lemmy devs have never built a high-scale service before, and I think that is possibly the single biggest risk to the growth and success of the Lemmy project in general.

Source: that’s my job, I’ve been doing that for some of the most high-scale services in the world for about a decade. I absolutely could help, actually I’d love to, but I definitely won’t under current Lemmy leadership, for reasons: https://lemmy.world/comment/596235

[–] [email protected] 4 points 1 year ago (1 children)

I get the feeling that the development leadership is going to have to change, with either the current developers bringing on leadership that can manage the growth or someone forks Lemmy and that becomes the default. The current developer model, like the current admin monetization model, can't stand as is in its current form.

[–] ritswd 1 points 1 year ago

That is what I’m rooting for. Alas I can imagine how they might cling to it, even if at the detriment of the project. Or not, I don’t particularly have any information. My fingers are crossed…

[–] V4uban 3 points 1 year ago (1 children)
[–] ritswd 0 points 1 year ago (2 children)

I think Kbin is something good being built by good people, I get what they’re trying to do, but unfortunately I don’t have a lot of faith that it will turn out to be a successful project.

In terms of technical scaling, I’m puzzled that they went with an interpreted language if the goal is scale. I get that the basic usage of Kbin’s features may not require a ton of CPU-heavy operations, or a fine handling of the memory; but once it meets sufficient scale, there will have to be some scale edge-case bottlenecks where you’ll want to step out of the beaten path and get lower-level, so I’m a bit confused about why they chose a technology that will make those harder to get past rather than easier. PHP is great for rapid prototyping, but I’d argue that’s not what the vision should be here.

About community scale, I’m not expert, but they seem to really care to offer a karma system; and we’ve seen the karma-farming behavior that this has been incentivizing on Reddit. I don’t see why it would be any different here if enough people end up joining. Lemmy is intentionally not offering a karma system, and it really feels like the healthier move long-term.

I think all it would take would be for the Lemmy devs to admit that they’re in over their heads, and that their political affiliations have been a hindrance to the project, to the point that they transition the governance of it to other people. I really hope they do that. If they do soon enough, they’re so far ahead and built on so much more long-term thinking, that I think it would pretty much make Kbin kinda obsolete. I have no special information about this, so I could be wrong, and I hope for them that I am; but I can see that as a pretty likely outcome.

(That, and on the shorter-term, I wouldn’t contribute to a product I don’t use, and I can’t use it for now because my usage is 100% mobile, and the current lack of API means no native client. I wish the mobile web was better than it is as an application platform…)

[–] V4uban 1 points 1 year ago (1 children)

Your point for Lemmy passing the governance of their project could happen. Or, as the number of other contributors increase, people will think that the impact of the two creators is diluted.

Or, at some point, someone creates a fork of Lemmy still based on ActivityPub, still compatible with both Lemmy and Kbin, but without any of Lemmy's political background '

[–] ritswd 0 points 1 year ago

Exactly, that’s what I’m rooting for.

[–] [email protected] 1 points 1 year ago (1 children)

Have you found that their political leanings have affected you in any way? Just curious if you have some sort of bias that's making you think people on the left can't produce efficient software.

[–] ritswd 1 points 1 year ago* (last edited 1 year ago) (1 children)

It hasn’t. But letting terrible people have power affects the world in normalizing violence and hatred. It’s not about left or right, if they were American racists against Chinese people, I would have the exact same problem. I’m personally quite on the left, but without the hate.

I am living safe and not being targeted with hateful violence like the Uyghurs or North Koreans are, so this is far, far more important than what can affect me.

[–] [email protected] 0 points 1 year ago (1 children)

Sorry I don't want to turn this into a debate but I feel like you're being disingenious; black people are getting killed all around your country because of their skin colour, and you have private prisons that systematically exploit prison labour. How are the social, political, and cultural challenges for your country different than theirs? I live in a country the USA has deemed an autocracy, I'm not apart of the ruling class, but I am living safe and not being targeted with hateful violence. Likewise, are you ignorant to the plight of ethnic minorities in your country and the hateful violence they experience? It just seems so chauvanist of you to say that your country is superior than anothers because you have the privilege of being "safe".

[–] ritswd 1 points 1 year ago

I am not, I actually volunteer 5h a week to teach kids in underprivileged neighborhoods, in an attempt to reduce the racial gap in the United States. There are a lot of racial problems in the US, due to the botched Reconstruction of the 19th century; and I firmly agree that it’s our common responsibility that to act to remediate this. I think education is a powerful weapon to bring opportunities and safety, so that’s where my focus is.

I guess the difference between someone like the Lemmy devs and you, and someone like me, is that I’m fully aware there are problems everywhere, but I won’t automatically associate someone’s nationality/residency to those problems; only what those people explicitly say about it. I initially considered that the issues people were having with the Lemmy devs were because they were not being US-centric, until I was linked to their past hateful posts on Reddit. I can’t believe they actually took questions on r/AskATankie; before that I was annoyed that people were calling them “tankies” because it sounded like a slur, I had no idea they actually identified as such. It’s the same feeling I get when I see Americans carrying a Nazi flag or a Confederate flag. So disheartening.

It is convenient to think of the world as divided in nationalities, a lot of things would be simpler. You need to realize that it’s a bit more complicated than that. China has a lot of successful high-scale services, with TikTok, Weibo, Alibaba, etc. It would be excellent if, for instance, ownership of Lemmy moved to a Chinese engineer (or a team of Chinese engineers of course) who helped with scaling those; provided it’s not someone with hateful view who self-identifies as a tankie. It doesn’t sound like a lot to ask.

Wherever you live, I am glad you are personally safe. I mean it for real. It’s better if you do things around you for those who aren’t, but I don’t resent people who don’t, we all do what we can.

[–] [email protected] 1 points 1 year ago

It is not like any other social network has become sustainable business. Reddit, Twitter, YouTube, FB all are net losers with all trials with and selling user data.

We can safely say that after almost 20 we still don't have sustainable business model for soc networks.

Let's try with donations.

[–] [email protected] 3 points 1 year ago* (last edited 1 year ago) (1 children)

Wouldn't that create a natural balance though? A large instance starts struggling so people are incentivised to move to smaller instances or start new instances and so spread the load more evenly. That's how it would scale. I'm surprised how many of the larger instances haven't closed signups yet but that wouldn't be a bad thing if they did.

[–] [email protected] 4 points 1 year ago (1 children)

The issue isn't on the user end, but the sub end since that is where all the data is stored.

So, according to your proposal, the best thing a sub should do when it is getting popular is to go private with its existing subscribers and any new people who want to participate should go create their own sub in a different instance.

[–] [email protected] 1 points 1 year ago (1 children)

I wasn't talking about subs, I'm talking about when an instance gets too popular. Ideally you'd want lots of small instances, ideally communities should be spread evenly as well and if your users are spread out that should happen more or less naturally.

[–] [email protected] 1 points 1 year ago

Where do you think the main costs of internet traffic and data are?

[–] nyakojiru 1 points 1 year ago

Sometimes you have just to accommodate to the situation and keep going until it settles down. The error I think here is thinking something can’t have flaws and issues, even more if it’s not behind a corporations. And no one wants corporations.

[–] nyakojiru 1 points 1 year ago

Sometimes you have just to accommodate to the situation and keep going until it settles down. The error I think here is thinking something can’t have flaws and issues, even more if it’s not behind a corporations. And no one wants corporations.

[–] [email protected] 1 points 1 year ago (1 children)

When the protocol favors monoliths, we're right back to the Reddit problem

[–] [email protected] 1 points 1 year ago (1 children)

Scalability doesn't mean "favoring monoliths". It's just scalability and honestly, 60k users shouldn't bring a service down. 60k users is not even close to being a monolithic instance.

[–] [email protected] 1 points 1 year ago

Scalability does mean favoring monoliths because it costs money to scale and scaling here isn't proportional to your instance's users, it's proportional to the size of the entire network.

60k users is today, not tomorrow. I'm thinking forward to 6000k users.