this post was submitted on 04 Jun 2023
901 points (98.1% liked)

Lemmy

2172 readers
56 users here now

Everything about Lemmy; bugs, gripes, praises, and advocacy.

For discussion about the lemmy.ml instance, go to [email protected].

founded 4 years ago
MODERATORS
 

This site is currently struggling to handle the amount of new users. I have already upgraded the server, but it will go down regardless if half of Reddit tries to join.

However Lemmy is federated software, meaning you can interact seamlessly with communities on other instances like beehaw.org or lemmy.one. The documentation explains in more detail how this works. Use the instance list to find one where you can register. Then use the Community Browser to find interesting communities. Paste the community url into the search field to follow it.

You can help other Reddit refugees by inviting them to the same Lemmy instance where you joined. This way we can spread the load across many different servers. And users with similar interests will end up together on the same instances. Others on the same instance can also automatically see posts from all the communities that you follow.

Edit: If you moderate a large subreddit, do not link your users directly to lemmy.ml in your announcements. That way the server will only go down sooner.

top 50 comments
sorted by: hot top controversial new old
[–] [email protected] 66 points 2 years ago (4 children)

I'm going to set up a general purpose instance tomorrow with the intention of handling a relatively large number of users. The main problem is choosing a domain!

[–] [email protected] 37 points 2 years ago (8 children)

Naming things is one of the two most difficult issues in IT, alongside cache validation and off-by-one errors.

load more comments (8 replies)
[–] [email protected] 26 points 2 years ago (5 children)

I was also contemplating setting up a new instance for this. I have 100s of gigs of unused ram, CPUs on idle and a 10gbit connection looking for something to do. The only issue I couldn't figure out was the name. I own itjust.works was thinking of something clever subdomain to use with it. I'm glad I'm not the only one with this issue

[–] [email protected] 39 points 2 years ago* (last edited 1 year ago) (6 children)
load more comments (6 replies)
[–] [email protected] 20 points 2 years ago

can't wait for fedd.itjust.works to go online!

load more comments (3 replies)
load more comments (2 replies)
[–] ruud 54 points 2 years ago

Lemmy.world is a new server, accepting signups. You're welcome there.

[–] [email protected] 50 points 2 years ago (5 children)

Sadly, I feel like the Fediverse, based on ActivityPub, was fundamentally designed wrong for scaling potential. I do like Fedi and I like ActivityPub, but I think instances should not have to be responsible for all of this:

  • Owning user accounts
  • Exclusively host communities
  • Serving local and remote users webpages and media
  • Never going down, as this results in users and content becoming unavailable

Because servers "own" the user accounts and communities it's not trivial for users to switch to a different instance, and as instances scale their costs go up slightly exponentially.

I wish the Fediverse from the beginning was a truly distributed content replication platform, usenet-style or Matrix-style, and every instance would add additional capacity to the network instead of hosting specific communities or users.

I guess it's a bit too late for a redesign now... Perhaps decentralized identifiers will take us there in some form in the future.

[–] [email protected] 25 points 2 years ago* (last edited 2 years ago) (1 children)

Not sure why you reference Matrix, which has even worse scaling issues as it indeed tries to replicate nearly the entire network on every server.

The Fediverse is really just working how the general web does, just with some standardized API for websites to interact. It's not perfect, but it works and has proven to be relatively scalable.

It sounds a bit like you had a bit too much of the Bluesky cool-aid, which indeed replicates nearly all of the mistakes of Matrix and makes it impossible to scale via small community owned servers instead of big company owned data-centers (which might be by design?).

[–] [email protected] 19 points 2 years ago* (last edited 2 years ago) (7 children)

Well yeah, point taken that replicating everything everywhere and forever might be impossible. But I do believe at a minimum my identity should be portable and accessing Fedi (ie. in microblogging: posting and viewing a feed of the latest posts of my follows) should be decoupled from which instance I pick to access the Fediverse.

I don't particularly like how owners of instances which grew are now essentially locked in to having to spend 100s or 1000s of dollars a month keeping their now expensive instances running and providing service. This is a bad place to be for a platform ran by volunteers. Letting instance owners scale their service down as well as up would be ideal. But this requires at least decentralized identity, and at best some form of content hosting redundancy...

It's easy to say the current architecture of Fedi works when it's still small. Your instance has 139 users.. That's not intended as a slight. Hosting instances is good and I applaud you for it! But I wish it were easier to more equally share the load once the platform becomes more popular.

load more comments (7 replies)
load more comments (4 replies)
[–] [email protected] 49 points 2 years ago* (last edited 2 years ago) (6 children)

@[email protected] It might be a good idea to default the Communities page to All instead of Local, to help push users into discovering other instances and promote them.

load more comments (6 replies)
[–] [email protected] 46 points 2 years ago (20 children)

I think lemmy will be bitten in the ass by not having considered clustering/horizontal scaling from the start. Federation alone as a scaling mechanism is only feasible for "nerds". But if the network wants to grow, we will need a few scale-able large hosted instances. And if their only choice is to scale vertically, there will be a hard limit (unless we put a good old Mainframe somewhere ^^).

Another downside of this design is: you can't run it with high availability. If there's only one process per instance, updating it will mean the whole instance is down. Sure, if all goes well this downtime is under a second. But if it doesn't go well or if a migration is needed, this might quickly become hours.

[–] PriorProject 19 points 2 years ago (6 children)

I think you probably underestimate how far one can get with "vertical" scaling. Here's the dockerfile: https://raw.githubusercontent.com/LemmyNet/lemmy/release/v0.17/docker/prod/docker-compose.yml

  • It includes 4 different containers... so there's a way to scale out to 4 machines right away. Maybe not every container is doing an equal amount of work... but there's some amount of immediately available machine-splitting.
  • I'm no expert, but I believe that at least the lemmy and lemmy-ui containers are stateless. If so, they're horizontally scalable already.
  • Postgres then would likely be the main bottleneck. But postgres offers read-replicas, so again the write-load and the read-load can be hosted on separate machines. And if there's enough read-load, you can have many replicas.

Other comments from the admins have shown that lemmy.ml today is running on a single eight-core box and it's currently hosting 30k registered users and over 1k active. So how much more compute capacity can we throw at "vertical" scaling on the current software architecture?

  • Just by going to a bigger single box, we can get 128 cores with no problem, a 16x bump in capacity. Does that get us to at least to 300k registered + 10k active?
  • Splitting the containers onto 4 separate machines. Does that get us 2x more?
  • Adding PG read-replicas and additional lemmy/lemm-ui containers would allow us to expand our instance footprint to maybe 6 physical machines should get us another 2x or more in performance.

Conservatively, that's 100x the computing capacity of the current hardware and could potentially support 1m registered users and 50k active. Now, I don't REALLY expect this to be possible today, there will be many software bottlenecks found along the way to scaling a single instance this large. But my point is that there's already a medium amount of horizontal scalability built into lemmy, and if the software doesn't fall over for algorithmic reasons (which is will at first), the current infrastructure architecture allows quite a lot of growth. There's plenty of time between now and a federation of million user instances to adopt a truly distributed storage backend if needed.

load more comments (6 replies)
load more comments (19 replies)
[–] [email protected] 42 points 2 years ago* (last edited 2 years ago) (1 children)

Over at https://join-lemmy.org/ , when someone clicked on "Join a Server", they are presented with a list of instances, it's not that obvious that these are cross-accessible (yes, the homepage mentioned it, but not here), and people are bound to look for one with the most users.

Perhaps, add a simple TLI5 explanation/diagram explaining how Lemmy works on https://join-lemmy.org/instances .

(The documents are also too wordy for most people to care.)

[–] [email protected] 26 points 2 years ago

Pull requests welcome, the code is here: https://github.com/LemmyNet/joinlemmy-site

[–] [email protected] 42 points 2 years ago (1 children)

Point us to where the coin slot is. E.g. Patreon. We insert coin 🪙, you upgrade.

[–] [email protected] 47 points 2 years ago* (last edited 2 years ago) (3 children)

Here they list them:

https://join-lemmy.org/donate

EDIT: shoutout to the madlad who donated $1,000!

load more comments (3 replies)
[–] [email protected] 40 points 2 years ago* (last edited 2 years ago) (11 children)

I have been wondering how cumbersome the Lemmy design will become for some. I love the idea that it is federated and decentralized however these are also major drawbacks for most average users (i.e not multi account users.

Multiple accounts needed for maximum uptime on different instances. What if I really like my username and its taken on another instance? If one instance is down and i comment with my other account will i then need to manage replies etc through different profiles? What happens if something spins up another instance of a similar domain so that they can get a username of someone to imitate them? I am sure these can be blocked after the fact or will other federated instances be automatically blocked.

What happens when someone gets bored of their instance and stops it, or it gets blocked, or they start getting unwanted attention. Does this mean all that content then goes into the ether?

Will this go down the route of whomever provides the instance with the most resources, best load balancing becoming the one, blocking other instances and controlling it as if it were private and independent?

There are a lot wait and see things, but I am excited to help and see what this great project becomes.

[–] [email protected] 22 points 2 years ago* (last edited 2 years ago) (13 children)

I've experienced a taste of this already. I checked the instance list a couple days ago, and didn't see one that stood out for my interests, so I created an account on the main lemmy.ml instance.

I just registered the same username on another but as far as I can tell, there is no way to merge or link these two accounts. So all the setup I've done and all the communities I've subscribed to, I have to do over again.

------

Another "issue" (a bug or feature?) I'm seeing is there are a lot of duplicate communities between the instances. I guess one will eventually "prevail" and become the defacto instance for that community.

load more comments (13 replies)
load more comments (10 replies)
[–] [email protected] 39 points 2 years ago (3 children)

I tried like 4 or 5 instances before coming to lemmy.ml, but none of them were taking applications anymore. Finding even those was a hassle, since all I got was a list of domains without any details as to what the instance is about or if they allowed newcomers.

Now that I've setup everything, Lemmy does seem like nice alternative to Reddit, but as someone from the outside, all of this is daunting.

load more comments (3 replies)
[–] [email protected] 38 points 2 years ago (6 children)

Another thing:

We do need more site admins to help us handle the applications and moderation.

For obvious reasons, we prefer ppl who have been here for a long time, and post / comment consistently. If you'd like to help us out, so that nutomic and I can focus on coding, that would be splendid.

load more comments (6 replies)
[–] [email protected] 31 points 2 years ago (6 children)

I sent my registration yesterday, because I signed in another instance, one from my country, but I couldn't see all the post and no comments from lemmy.ml even thought is supposedly linked, so thank you for approving my account.

Even if I'm a tech savvy person I found the whole experience of joining lemmy pretty bad, I like the concept of federation, but I think it's too confusing to normal people, it really needs to be more seamless if you want to grow, how? idk, I was thinking some sort of replication, when you sign up, you are registered to the main instance (this) and given the choice to select other instances, automatically selecting let's say another 3 based on your location, then your account is synced in all the registered and linked instances, when you login if an instance is experiencing overload then it switches to another one. I don't know if this is realistic or out of the scope of Lemmy, or maybe against the philosophy of it. I'm just rambling.

I'm just glad that there is an open alternative for anonymous social interaction in this day of walled internet services such as discord, twitter, facebook etc. and I wish you all the success.

[–] [email protected] 22 points 2 years ago (6 children)

Agreed, someone needs to create an easy "sign up here" with a default option (maybe just randomize across various instances, not sure)

load more comments (6 replies)
load more comments (5 replies)
[–] [email protected] 30 points 2 years ago (3 children)

If now is struggling then on June 12 will be a nightmare.
Reddit will go dark in protest, many messages to join Lemmy, most instances will be overloaded or even DDoS with so many users, like what happen with Mastodon.

load more comments (3 replies)
[–] [email protected] 29 points 2 years ago (1 children)

I wonder if a longer term solution would be to auto rotate the server list to bump less popular ones.

[–] [email protected] 33 points 2 years ago

Pull requests welcome, here is the repo:

https://github.com/LemmyNet/joinlemmy-site

[–] [email protected] 26 points 2 years ago (1 children)

Is there any way to migrate instance or do I make a new account entirely?

[–] [email protected] 22 points 2 years ago (1 children)

You have to make a new account unfortunately.

[–] [email protected] 18 points 2 years ago (4 children)

Seems like this should be a high priority feature. I did try joining a different instance but at the time only lemmy.ml was functioning and accepting applications. Now that I have subscribed to a bunch of communities starting afresh would be a bit of a hassle.

load more comments (4 replies)
[–] [email protected] 26 points 2 years ago (1 children)

Users are likely going to see this as it's the "official" Lemmy instance when trying to join for the first time.

Any admins of instances that are accepting people, give your best elevator pitch!

load more comments (1 replies)
[–] [email protected] 23 points 2 years ago

I just created https://lemmy.studio/ for music-related discussions.

[–] [email protected] 21 points 2 years ago* (last edited 2 years ago)

It's good to be popular. Congratulations on the growth!

[–] [email protected] 21 points 2 years ago* (last edited 2 years ago) (2 children)

I guess every bigger instance needs to constantly promote a monthly support fee to keep the servers runnin. Something that most people already use and don't have to register first. Like PayPal or Patreon.

Edit: And I support Lemmy now on Patreon.

load more comments (2 replies)
[–] [email protected] 21 points 2 years ago (1 children)

Got my instance running earlier today thanks to some helpful people in the Matrix chat. Intention at the time was to have some friends of mine join but I'm not opposed to having signups as well. Long live open source, federated software!

load more comments (1 replies)
[–] [email protected] 20 points 2 years ago (1 children)

Is there any way to transfer an account to a different instance?

[–] PriorProject 22 points 2 years ago* (last edited 2 years ago)

There is an issue open for this at https://github.com/LemmyNet/lemmy/issues/506, but it's not addressed yet. To migrate, you'd have to:

  • Manually create a new account.
  • Manually copy your settings.
  • Create a post, comment, or set your profile description to point to your new account so people can manually find you at your new account.
  • Your existing comments and posts would stay associated with your current/old account.

I wouldn't sweat it though. It's not currently "broken", and the lemmy.ml admins can shut off new signups if it becomes a major problem. My take is this is more about directing new signups elsewhere. The one instance can't hold every new user that's signing up right now.

[–] [email protected] 20 points 2 years ago (4 children)

If server crashes again the devs should just stop all registrations to this instance, people always follow path of least resistance and the first dev instance is just too easy to be stuck onto

load more comments (4 replies)
[–] [email protected] 19 points 2 years ago* (last edited 2 years ago) (5 children)

First post for me!

Sorry, I applied and got approved here. Still waiting to hear back from beehaw…

I’m really digging this UI compared to Reddit, but I am 99.9% a mobile user via the native Reddit app (don’t @ me!)

I am very tempted to setup my own instance. Wondering what resource usage looks like for an instance.

load more comments (5 replies)
[–] [email protected] 18 points 2 years ago

Sorry for contributing towards this by registering but I'm very appreciative of the work being done to facilitate this community. I hope to see Lemmy grow with the negative direction other platforms are taking.

[–] [email protected] 18 points 2 years ago

I just created https://lemmy.film if that would be useful for anyone.

[–] [email protected] 18 points 2 years ago (3 children)

Is there a feature to send a read-only/static link to Lemmy pages?

I’m envisioning a pre-cached version of the page that is updated hourly or so, rather than querying the database live for every comment on the thread. In a perfect world, these could also be offloaded to a CDN as static pages…

load more comments (3 replies)
load more comments
view more: next ›