this post was submitted on 11 Jul 2023
36 points (97.4% liked)

Selfhosted

40260 readers
1086 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
 

I have been trying to understand how the caching of content from other Lemmy instances works. From what I have gathered, the local Lemmy instance will automatically download and store posts made to any communities that are followed by users on the local instance.

To me, this seems somewhat unsustainable in the long term - I am aware of the fact that it's only storing the text of the posts, and not any media. I'm curious if it's possible to configure the local instance to only cache the stored data for a certain amount of time (it might be better to just periodically purge the entire cache with a cronjob, or something); however, the data that I would like to store permanently is posts to any other community by users on the local instance, as well as posts made to communities on the instance (I have a suspicion that the communites data is permanently stored by default).

top 9 comments
sorted by: hot top controversial new old
[–] ClassyHatter 15 points 1 year ago (2 children)

As far as I know Lemmy stores only text locally and images and such will be linked to the external instance. Text doesn't use much disk space, so that shouldn't be a big a problem. Sometimes when you browse Lemmy, you notice posts that have broken links to images. It's because the other instance is down, but you can still see the text portion of the posts on your home instance.

[–] [email protected] 6 points 1 year ago

I think it stores thumbnails in the pictrs directory by default.

[–] PriorProject 3 points 1 year ago

The full firehose of the lemmyverse will grow your DB by about 2GB per day, plus an additional 8GB per day in thumbnails. And the rate is growing frantically. It's not necessarily trivial, depending on how much you subscribe to.

[–] [email protected] 7 points 1 year ago

AFAIK the cached images are only of thumbnail size and get deleted after a few months

[–] [email protected] 6 points 1 year ago

May be worth having a look at the ActivityPub protocol. It’s the way Lemmy instances (and other stuff!) communicate with one another. From there I think it will be clearer how a single Lenny instance could behave. https://activitypub.rocks

[–] [email protected] 2 points 1 year ago (1 children)

I have just setup an instance of Lemmy on my server. So if I federate to a community, all its posts will be stored on my server, also images and so on?

[–] PriorProject 4 points 1 year ago (1 children)

The terminology is that you federate with an instance and subscribe to a community.

And yes, if you sub a community, your instance will mirror posts and comments to your local DB and re-serve them to the public unauthenticated internet. On a single-user instance, anyone can browse the communities on your server and infer what your account subscribes to, including whatever porn, piracy, and other legally questionable shenanigans based on what gets replicated to your server.

[–] [email protected] 1 points 1 year ago (1 children)

Thank you. What about comments and posts that get updated/changed? Will my instance also reflect those changes?

[–] PriorProject 1 points 1 year ago

Yeah, they get federated as a new edit message. Not sure whether they get stored in the db as edit entries, or if the DB actually just modifies the original... but edits federate.