this post was submitted on 11 Jun 2023
233 points (98.7% liked)
Asklemmy
42502 readers
1353 users here now
A loosely moderated place to ask open-ended questions
Search asklemmy ๐
If your post meets the following criteria, it's welcome here!
- Open-ended question
- Not offensive: at this point, we do not have the bandwidth to moderate overtly political discussions. Assume best intent and be excellent to each other.
- Not regarding using or support for Lemmy: context, see the list of support communities and tools for finding communities below
- Not ad nauseam inducing: please make sure it is a question that would be new to most members
- An actual topic of discussion
Looking for support?
Looking for a community?
- Lemmyverse: community search
- sub.rehab: maps old subreddits to fediverse options, marks official as such
- [email protected]: a community for finding communities
~Icon~ ~by~ ~@Double_[email protected]~
founded 5 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Is there an approximate specs per number of users guide to size a lemmy instance?
I haven't seen one yet. Disk usage this morning on lemmy.world was reported at about 4GB over 11 days (probably low usage). The 100GB drive would probably fill up in 275 days or so if usage did not increment. If it's not redundant and dies, all that content is lost.
So storage will be a huge issue for lemmy unless I'm missing something.
100GB is practically nothing nowadays.
There are people (myself included, not to brag) running home servers with literally hundreds of terabytes of data. At that ~0.3 GB/day number, I alone could host 3,500 years worth of data. Get some of those r/DataHoarders and r/HomeLab guys on here and Lemmy would never run out of space.
Considering Lemmy is absolutely tiny compared to Reddit, these aren't numbers worth considering. Every single instance needs to mirror data, and I still don't understand how this is supposed to scale to something a fraction the size of Reddit unless the federation is just a few enormous instances that can afford that scale. It's not like everyone pitches in what they can -- every single instance individually needs to be able to support the entire dataset and the associated synchronization traffic (for the portion that its users have requested access to).
What if there are a couple large archive mirrors and the posts on other servers have a life expectancy maybe based on time, but also engagement? Crucial posts could be stickied, but I don't see the need for everyone to hold onto everything forever. Even in the event of a catastrophic loss of the archives, the communities could still live on and rebuild.
Yeah, that's a good point. I imagine there'd have to be some compromise like that for smaller instances. How often do users load up content older than say a couple of weeks or a month? Could be a hinderance on the experience, hard for me to estimate (for myself) how often that happens.
That is just one instance and that is a small amount of users. Reddit has 430 million active monthly users. Let's say 1% move over to Lemmy. According to the 4 GB per 22 days (with ~1k users per instance) for Lemmy.world, that would mean you would need 1.6 Terrabytes of storage per day to support that 1% of users. Of course this would be spread over a number of instances... but you can start to see where the problem lies....
Can content be stored somewhere like S3 instead of spinning disk? It would certainly be more robust and cheaper.
Yeah, I'd love to get an approximate sense of how much these instances cost