Lemmy Project Priorities Observations
I've raised my voice loudly on meta communities, github, and created new [email protected] and [email protected] communities.
I feel like the performance problems are being ignored for over 30 days when there are a half-dozen solutions that could be coded in 5 to 10 hours of labor by one person.
I've been developing client/server messaging apps professionally since 1984, and I firmly believe that Lemmy is currently suffering from a lack of testing by the developers and lack of concern for data loss. A basic e-mail MTA in 1993 would send a "did not deliver" message back to message sender, but Lemmy just drops delivery and there is no mention of this in the release notes//introduction on GitHub. I also find that the Lemmy developers do not like to "eat their own dog food" and actually use Lemmy's communities to discuss the ongoing development and priorities of Lemmy coding. They are not testing the code and sampling the data very much, and I am posting here, using Lemmy code, as part of my personal testing! I spent over 100 hours in June 2023 testing Lemmy technical problems, especially with performance and lost data delivery.
I'll toss it into this echo chamber.
view the rest of the comments
Lemmy had been on GitHub for over 4 years when May 2023 rumblings out of Reddit began that by June 1 there was a countdown of 30 days...
What lemmy lacked was data-centered testing. First off, the quantity of data. There was no means in the project to populate even what might be considered a modest amount of data: 500 communities, 10 thousand posts, 20 thousand comments, 5000 users. The testing servers that were in place in June 2023 (enterprise, voyager, etc) were empty. They had some basic scripts to test posts and comments, but feature-focused testing, not tens of thousands.
Lemmy was only tested on nearly empty data, that is how the testing had been done for over 4 years.
With ORM generated SQL statements and a lack of PostgreSQL understanding, there were big problems in the code in May 2023. The site_aggregates table had a SQL TRIGGER UPDATE statement with no WHERE clause that was hitting on every known Instance in the database. Testing only created 5 instances, so this performance problem was not noticed. Lemmy-ui for admin/operator managing the site had no page to view this site_aggregates database table.
Scaling and performance issues were already crashing lemmy.ml in May. They opted to close the doors for new member sign-up and did massive hardware upgrades on June 13. But the performance problems and crashes did not stop, it only worsened with each new instance gong online with the site_aggregates write-activity bug going undiscovered. To try and solve crashing, more and more instances were added to the Lemmy network, exploding from 60 to 600 servers, even over 1500 at one point - almost all because of the Reddit July 1 API cutoff. So you had on June 20 lemmy.ml hammering hundreds of instance rows in the site_aggregates database table with SQL UPDATE write activity on every single new comment and post...