this post was submitted on 05 Sep 2023
16 points (100.0% liked)

Announcements

228 readers
1 users here now

Announcements about system updates or other things related to this instance.

founded 1 year ago
MODERATORS
[email protected]
16
Lemmy.today had its first unexpected downtime today (lemmy.today)
submitted 1 year ago by [email protected] to c/[email protected]
12 comments fedilink hide all child comments
 

Hi all,

The disk on the instance ran out of space today, due to the way Lemmy software caches images from all other instances. That cache had filled up about 60 GB's of disk, despite us being a small instance with very little local activity.

I had to delete the last 10 days of cached images again, and I plan to delete quite a lot of older cached images as well. The mobile apps seems to not be affected by this (they have a local image cache I believe), but on the web site, this leads to missing thumbnail images.

They are working on a fix for this in the Lemmy software so the disks dont fill up so enormously with cached thumbnails, and as soon as its out, we will install it here.

Hope you guys didnt get too annoyed or sad by the instance being unavailable for a while.

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 2 points 1 year ago (2 children)

I have been reading up on pict-rs and it seems that the environment variable PICTRS__MEDIA__RETENTION__PROXY is the one to set for cleaning up old cached images.

It seems they added this environment variable in pict-rs version 0.4.2 so I bumped our pict-rs version from 0.4.0 to 0.4.2 and applied the environment variable now.

Hopefully this will take care of the disk space growing on its own, but will keep an eye on it. :)

[–] [email protected] 2 points 1 year ago

Awesome! Hopefully it works!

[–] [email protected] 2 points 1 year ago (1 children)

I found a problem. It removes Avatar & Banner pictures from your profile!

[–] [email protected] 2 points 1 year ago

I noticed this also. Since all pictures are handled by pict rs, deleting anything from disk risks removing any picture users have uploaded also.

Interesting design choice of Lemmy to keep both cached images and user images in the same place with the same names... They all look the same on disk with generated names.

I noticed today that community pics are gone also.

Try to upload the same images again and if that doesn't work, try to create new images to replace them. I will also try tomorrow to look into this more...