Lemmy Server Performance

420 readers

1 users here now

lemmy_server uses the Diesel ORM that automatically generates SQL statements. There are serious performance problems in June and July 2023 preventing Lemmy from scaling. Topics include caching, PostgreSQL extensions for troubleshooting, Client/Server Code/SQL Data/server operator apps/sever operator API (performance and storage monitoring), etc.

founded 1 year ago

MODERATORS

[email protected]

Lemmy Server - full text real-time searching of all comments&posts is a performance concern. Might have worked when lemmy database was low on content. (lemmy.ml)

submitted 1 year ago* (last edited 1 year ago) by [email protected] to c/[email protected]

5 comments fedilink hide all child comments

At minimum, the Instance installer guides and documentation needs to reflect that your own server doing real-time searches should be a conscious choice.

Lemmy.ml is erroring out already on searches, GitHub issue: https://github.com/LemmyNet/lemmy/issues/3296

And if I'm seeing things right, 0.18 changed the loging from 0.17 where searching for a community https://lemmy.ml/communities now does a full-text posting/comment search? It gives a ton of output I don't want, and it's hitting the database even harder. Instead of just searching a single database table, now it is going into comments and postings.

Reddit was legendary for not having their own in-build comment and posting search engine up to date - and also used PostgreSQL (at least they did when still open source). It is a huge amount of data and I/O processing.

top 5 comments

sorted by: hot top controversial new old

[–] phiresky 2 points 1 year ago

postgresql has integrated full text search functionality that works pretty well. shouldn't be too hard to add

[–] [email protected] 2 points 1 year ago (1 children)

Yeah defaulting the search type to "all" when searching from the Communities page is kind of silly.

[–] PriorProject 1 points 1 year ago (1 children)

The problem is, at least in v0.17.x, searching for a community url or bang-prefixed community name returns nothing when you constrain the "type" drop-down to "communities". Now, that feels pretty clearly like a bug to me... if you search community-types by url, it should return the correct result.

But if that couldn't be fixed for v0.18.0 due to time/priority constraints, defaulting to searching "all" types is a reasonable workaround even if it makes little sense absent the above context. At least until searching for communities by url is fixed.

[–] [email protected] 1 points 1 year ago

I'd rather no results, than a massive list of incorrect results of a type I'm not searching for... ie comments/posts rather than communities.

[–] [email protected] 1 points 1 year ago

https://www.meilisearch.com/

Support would be great. I use it with my Akkoma instance and it works quite well.