Lemmy Project Priorities Observations
I've raised my voice loudly on meta communities, github, and created new [email protected] and [email protected] communities.
I feel like the performance problems are being ignored for over 30 days when there are a half-dozen solutions that could be coded in 5 to 10 hours of labor by one person.
I've been developing client/server messaging apps professionally since 1984, and I firmly believe that Lemmy is currently suffering from a lack of testing by the developers and lack of concern for data loss. A basic e-mail MTA in 1993 would send a "did not deliver" message back to message sender, but Lemmy just drops delivery and there is no mention of this in the release notes//introduction on GitHub. I also find that the Lemmy developers do not like to "eat their own dog food" and actually use Lemmy's communities to discuss the ongoing development and priorities of Lemmy coding. They are not testing the code and sampling the data very much, and I am posting here, using Lemmy code, as part of my personal testing! I spent over 100 hours in June 2023 testing Lemmy technical problems, especially with performance and lost data delivery.
I'll toss it into this echo chamber.
view the rest of the comments
May and June 2023... "add instances for scaling"
cross-instance community shock built in
More and more servers were added to the Lemmy network, but the issues that Beehaw pointed out were lurking too. The experience of a new Lemmy instance server is:
Despite the wide variety of hardware and budget choices, there is no install testing to establish that PostgreSQL has enough RAM and disk I/O performance to support Lemmy. Lemmy starts with zero data in the tables, Lemmy starts out empty... where it performs fine. The server can federate and subscribe one community at a time to remote servers on the Lemmy network
Social Experience of a New Instance
Each remote community starts out with with a handfull of posts, but none of the comments, so it is the community in name - but not content. The number of posts, comments, active users of a community - the community_aggregates database table - is not copied from Beehaw. So a new instance will only show a few users, a few posts, and no comments. The data misrepresents Beehaw's community to new members of this remote instance.
This creates a disconnected experience from the lemmy instance, the entire history of comments and posts are not there, so context of what is a repost or duplicate post isn't available to people joining the community on a newly created instance.
Beehaw had been online with content since January 2022, for well over a year before the rumblings of Reddit influx started in May 2023.
So new Lemmy servers that were fired up in June 2023 - by the hundreds - all started with empty copies of Beehaw communities, void of the moderated posts and comments to set an example of what community content looked like. And members of those new servers would just dive in to what they perceived to be an empty community - a place of other Reddit newcomers - just now generating new posts and comments.
The framing and name of remote instance can present the person with a variety of context clues and expectations, when they switch topics to a Beehaw community - it has all be framed by a non-Beehaw Sign Up screening, introduction, and site name. A newcomer can jump right into a conversation topic without having a social sense of what that means...
It was negative experiences with Reddit that were driving the activity, not the seeking of quality content, but a money matter and price increase on the Reddit side. This too set a tone of grievance and desire to find rapid replacement for the feed that people had been accustomed to. There were even bots created to feed Reddit content into Lemmy that went online...
For an established site, Beehaw, with over a year's worth of cultivated content and experience with Lemmy, this was a shock. And the server crashes and overloads by just the site_aggregates UPDATE bug because of new instance rows exploding in the database table - was worsening day by day. That particular bug didn't hit on the new instance servers very hard, they had very few members and were not creating posts and content, but the active server Beehaw with active local users - was carrying all the load of that bug plus having overloads trying to send out federated copies of new content to hundreds of once-eager servers that might be shut down and not even bother to read the content.
Beehaw wasn't the only server active with local users before the mass swarm of over a thousand new instance servers went online...