this post was submitted on 22 Jun 2023
93 points (97.9% liked)

Selfhosted

37779 readers
287 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
 

I have a domain name that I own but am not making use of and was thinking of setting up my own personal Lemmy instance, partly so I can have a Lemmy id and instance that I can completely control, partly so that I can contribute directly to my hosting cost, and partly because it might be fun to tinker with (or it might just end up being a pain; I'm still trying to figure out which is the case).

However, from this comment it sounds like, rather than contributing to horizontal scaling and easing the load on other servers, I might actually end up increasing the load on other servers by adding yet another server that the other servers have to talk to in order to keep my server updated on the latest comments and posts to which I am subscribed.

So given this, would self-hosting a personal instance actually make things worse for everyone else and thus be an irresponsible action at this time and/or for the foreseeable future? Because the last thing that I want to do is to inadvertently add a burden to the fediverse!

top 24 comments
sorted by: hot top controversial new old
[–] [email protected] 16 points 1 year ago

Oh interesting, I'm curious on the answer to this as well.

[–] [email protected] 13 points 1 year ago* (last edited 1 year ago)

ActivityPub traffics are not that heavy. The activities are queued in the background and processed by a pool of dedicated workers, so no matter how long the queue, they should not bog the webserver down. Each entry in the queue does occupy some memory, but should be negligible.

I would say that an active users subscribing to a community using their own instance is actually generating less server load to the subscribed community's instance compared to a user in that instance actively interacting with the community directly. But, if you subscribe using a personal instance but never actually use it (never logging in, never interacting with the community, never lurking in any discussion, basically forgot it exists), then yeah, you're probably a dead weight, albeit a tiny one.

[–] [email protected] 9 points 1 year ago

The impact either way is minimal, so if it is something you enjoy doing go for it. Maybe it will turn into a slightly larger instance for your friends over time and then it can have a positive impact.

[–] [email protected] 8 points 1 year ago (1 children)

As what I understood, actually it makes things worse. This shouldn't be the case because the concept of fediverse itself is to have a lot of many instances communicating with each other.

The problem is the Lemmy is still a young project and we weren't expecting all this explosion in users. activityPub implementation doesn't scale well for now and so adding a new instance theoretically makes things worse.

But this is something that devs have as a high priority, in my opinion, because is very important to have instances correctly communicating with each other, otherwise the concept of federation falls.

[–] girthero 12 points 1 year ago (2 children)

When I first heard of the fediverse without understanding the architecture I envisioned something like torrent networks, where the larger the network the stronger the network. After learning more I'm not sure that's the case yet. Hopefully that is the endgame.

I would want to be in a place where I could enrich an existing community by self-hosting and synching content of that community and offering my small chunk of bandwidth to that community. I realize there is no community synching between instances, but I feel that's where it should be to prevent corporate control of communities in the future.

[–] [email protected] 3 points 1 year ago* (last edited 1 year ago)

ActivityPub define an inbox forwarding feature. I haven't dug too deep into Lemmy source code to see if Lemmy implement this, but if it does, this feature would allow more efficient propagation of activities (e.g. A -> B -> C instead of B <- A -> C where A is the originator).

[–] [email protected] 2 points 1 year ago

You can read the ActivityPub protocol... I don't think it will become like bittorrent, as there peer-to-peer concept doesn't seem to exist in the spec.

Having said that, while we are running into implementation limitations on the larger instances, the problems are being tackled. A couple of us are chatting on [[email protected]](/c/[email protected]) and [[email protected]](/c/[email protected]) to see if we can come up with good ideas to present to the devs to help Lemmy scale. You're most certainly welcome to join in on the fun!

[–] [email protected] 7 points 1 year ago

If it is a single-user instance and always going to be, then yes what you have heard might be true. But again that is not a lot of load to begin with, and if your instance ends up with more users I don't see how it can put more load on other servers than if all those users are on "other servers" directly.

[–] [email protected] 6 points 1 year ago

Hey that’s me!

If you’re excited, hop on, as have I! The protocol related scaling issue will resolve itself over time; a few of us are throwing ideas out and hoping some will stick with the developers.

But just bear in mind that you’re not adding to the network scaling because federation on the big servers will not be alleviated by you (or me) having an extra instance.

[–] [email protected] 6 points 1 year ago

For my personally, I'm hosting an instance so that I can play with it, but also so that I can test new functionality, and help fix issues.

So I think if you are concerned about being a burden, then actively use your instance for the betterment of lemmy!

[–] [email protected] 5 points 1 year ago

Great question, glad you asked. When I joined Lemmy a week or so ago, a post from the lead developer that was stickied for a while specifically mentioned adding instances was helpful to help take the burden off of the main flagship instances of the time like Lemmy dot ml.

What you can also do to assuage your concerns is invite other users to your instance so you share some economy of scale, and try to spearhead some community you have a personal interest in so you’re also contributing content from your server. We do that with [email protected] for example.

[–] [email protected] 5 points 1 year ago

Whether the comment is true or not, I feel like part of federation is the exercise of this freedom. If Lemmy can't support this paradigm, then the answer is that it needs to adapt and overcome these limitations, not that we need to adapt to its limitations.

Making new instances into the federation will bring this problem further into the attention of the developers and probably pressure them to improve that specific weakness in Lemmy.

At least that's my theory.

[–] [email protected] 4 points 1 year ago

I did just that, set up for me and some friends. Working nicely so far although takes a bit of time to get enough stuff federated!

[–] [email protected] 3 points 1 year ago

It's something I've considered as personal self-hoster. Technically, keeping my instance updated is incurring more traffic on the end server than I ever would individually.

It's got me thinking about opening up my instance to maybe a few hundred people to make the extra load on the other servers worth their while.

[–] [email protected] 2 points 1 year ago

That's a good question. Every federated action (post, comment, votes) is transmitted to every subscribed server in real time. Computers are really fast, and should be able to handle thousands of federated actions every second. I don't think we have a good idea on what number of servers are too many.

[–] [email protected] 2 points 1 year ago (1 children)

Tbh to me it just feels like we are in the early stages of a huge boom of users. I'd assume in a few months most of these growing pain issues will be gone. IF anything, maybe wait until the load balancing is settled and make your instance (but no issues if you want to help stress test the system).

[–] ericjmorey 1 points 1 year ago

I’d assume in a few months most of these growing pain issues will be gone

I'm assuming it will be years before things are really ironed out.

[–] [email protected] 1 points 1 year ago

Having all the servers you want content from pushing duplicate data out just for you would increase load. You as a single user on an instance that already receives that data is a significantly smaller footprint. Only people truly interested in running a public instance and all the associated work that comes with it should really spin up instances. Contributing monetarily to your favorite instance would be much more sustainable.

load more comments
view more: next ›