I get the impression that we're headed for the same issues that pop up when we put all our eggs in one basket with Reddit/FB/whatever. People flock to the largest instance, and someday that instance could go down due to cost or the host losing interest.

I'm wondering whether it would be technically achievable to have servers/instances and federation where the communities are essentially mirrored or have broadly distributed existence - maybe even with user storage a la torrents.

If there's a large [email protected] community and a small [email protected] community, all of the discussion, images, contributions to lemmy.here die if the server goes down for good. Yes, the users can relocate to lemmmy.there - even under the same community name - but it's not the same as having full continuity of a completely mirrored community.

I realize this concept has technical hurdles and would involve a reimagining of how the fediverse works, but I worry we're just setting up for another blowup at some TBD date when individual sysadmins decide they've had enough. If it's not truly distributed and just functions as a series of interconnected fiefdoms, communities and their information won't survive outages, deaths, and power struggles.

you are viewing a single comment's thread
view the rest of the comments

[–] rexxit 6 points 1 year ago (1 children)

I just typed out a whole reply and it got eaten after I clicked reply - no surprise there are still going to be bugs in this implementation, nevermind getting more complex with it.

I'm not technical enough to make anything happen if it involves code, but I'm way psyched you're working on something along these lines. The way I see the problem there are some distinct issues

sharing the load of hosting (i.e. users are also hosts, as with torrents)
fault-tolerance / outage-tolerance
community/data immortality (i.e. discrete communities that are somehow not reliant on any instance)

[–] mo_ztt 7 points 1 year ago

Yah absolutely. A lot of the problems in #1 and #2 are already solved, with technologies like Bittorrent as you mentioned, although doing the work to make it work isn't trivial. #3 can potentially be complex. So at present I'm imagining that someone will need to "pin" on some machine under their control all the data that operates a given community, in order for it to be guaranteed that all that data is available. Someone has to guarantee the storage in order for it to be guaranteed that the appropriate data won't be deleted, and I think it's better to just be explicit with the requirement and let people decide how to configure their software. Then the issue is just that the technology works well enough that that data can be found quickly if it's needed, and cached on other servers well enough that you can run that "pin" on your own home internet or on like a $20/mo Digital Ocean account, as long as you have the disk space and a machine that can be available almost all the time.

That sounds simple, but there are weird little things to worry about -- e.g. what if a stored-in-the-shared-store community is abandoned, and part of it passes out of the shared memory of the system, but not all of it? At what point can someone else create a new community to replace it? What happens to little pieces of shared content that still do exist within the system when that happens? What if at some point the old community reappears? I feel like that can be dealt with reasonably well, but it's clear that there are questions like that that are uncharted territory which always has some surprises.