this post was submitted on 12 Jun 2023
12 points (100.0% liked)

Meta (lemm.ee)

3473 readers
8 users here now

lemm.ee Meta

This is a community for discussion about this particular Lemmy instance.

News and updates about lemm.ee will be posted here, so if that's something that interests you, make sure to subscribe!

Rules:

If you're a Discord user, you can also join our Discord server: https://discord.gg/XM9nZwUn9K

Discord is only a back-up channel, [email protected] will always be the main place for lemm.ee communications.

If you need help with anything, please post in !support instead.

founded 1 year ago
MODERATORS
[email protected]
12
Image uploads, "No pornography" rule and lemm.ee Growth (lemm.ee)
submitted 1 year ago by [email protected] to c/[email protected]
23 comments fedilink hide all child comments
 

Hello, fellow lemmings!

I have three updates regarding lemm.ee:

Image uploads

As some of you have already noticed, image uploads are currently not working on lemm.ee at all. The short explanation is that I have been trying to figure out the best way to enable image uploads without incurring big additional costs in terms of storage, bandwidth, and moderation.

I have come to the conclusion that initially, the most sustainable path will be to enable image uploads only for small images (<100kb) - this should be enough to allow folks to upload avatars for themselves and their communities, but it for sure won't allow uploading many bigger images or gifs for your posts.

For bigger images, I would ask everybody to use an external image hosting service for now, as was usually done on Reddit before they added image uploads.

I am in the process of creating a patch to the image upload code that will allow all Lemmy instance admins to configure a smaller limit on the uploaded image sizes. Once that patch is finished and accepted, I will be deploying it on lemm.ee - at that point, you will be able to finally upload avatars. Sorry for the wait, hopefully it won't be much longer!

"No pornography" rule

There is a new rule landing in your sidebar - "No pornography". I was initially against adding any too-restrictive rules, but after many discussions with other instance admins, I have realized that there is a big amount of additional risk to a community when allowing pornography.

As of writing this post, there are no pornography-related communities on lemm.ee, so I'm hopeful that this rule won't be too controversial. If you were planning to create such a community on lemm.ee, I'm very sorry to ruin your plans and I hope you can find another instance that is more suitable for your community.

Growth of lemm.ee

The rate of new users joining lemm.ee has certainly been much faster than I expected. For now, the growth has been sustainable, and we can still keep growing at this same rate for a while longer without incurring additional costs.

However, if this growth continues for an extended period of time, then additional costs will become unavoidable. At that point, we will have the option of either closing sign-ups to new users, or upgrading our infrastructure to allow continued growth. The latter can only be achieved through some kind of community funding, so I will be asking the community for input if that time ever comes.

I want to be clear about one thing: if we end up closing new sign-ups, then this will not close your communities to new members. As always, Lemmy users from any other instance will be able to subscribe to your communities on lemm.ee, they will be able to participate as normal and even become moderators. The only thing that will change if we close sign-ups is that new users will not be able to choose lemm.ee as their home instance.

That's all for now, let me know in the comments if you have any feedback or thoughts about any of this!

top 23 comments
sorted by: hot top controversial new old
[–] [email protected] 4 points 1 year ago

+1 to the no porn rule. I seriously considered hosting my own instance, but balked at the liability especially in regards to porn. I'm no prude, but having adult material seems like a serious headache.

[–] [email protected] 2 points 1 year ago

First of all, thanks for setting up this instance!

I'd suggest to open some form of paid subscription right now, not waiting for the moment when we'll need to make hard choices. And don't bother with asking for single-time donations - it's not sustainable with so small number of users.

[–] [email protected] 1 points 1 year ago

@sunaurus single handedly help me beat porn addiction.

Thanks for looking out brother!

[–] [email protected] 1 points 1 year ago

To me - no pornography is even beneficial

[–] [email protected] 0 points 1 year ago (1 children)

I’m pleased about the porn, fwiw. I forgot to check prior to choosing an instance and thought I might have to move or be constantly blocking. And I didn’t want to move, because I’ve been really impressed with how you’ve implemented this.

Does anyone have a recommendation for photo hosting? Imgur, I guess? This kind of feels like a fun trip down memory lane to the early 00’s when we all had a blog and constantly lectured each other about how hotlinking was stealing :) Looks like photobucket is a paid service now, though.

[–] [email protected] 1 points 1 year ago* (last edited 1 year ago)

I advise against Imgur due to a lot of changes that they've been making. At face value it looks like they are just deleting NSFW content but also scrapping tons of old images that may still be relied on by many. If you look deeper you have to question how long they'll keep any content uploaded in the future. https://www.engadget.com/imgur-to-ban-explicit-images-and-delete-uploads-not-tied-to-an-account-122537118.html

Sadly, I don't know what the best alternative is. Postimages may be it for non NSFW stuff https://postimages.org/ and another option would be Catbox https://catbox.moe/

[–] [email protected] 0 points 1 year ago (1 children)

Glad to hear you're keeping up with all this new traffic!

Are you willing to talk about the costs you're incurring to run this? Or some details about your deployment? I'm considering spinning up an instance and curious to get a sense of what the overhead is like. I'm guessing you're handing this kind of traffic on...like a single EC2 instance+RDS?

Also, still trying to understand ActivePub -- is it mirroring all content from all other instances locally?

[–] [email protected] 1 points 1 year ago (2 children)

details about your deployment

It's not on AWS, but another cloud provider. Currently it looks like this:

  • Cloudflare for DNS (and all the Cloudflare protections)
  • Load balancer
  • 2 statless servers for the Lemmy software
  • 1 stateful server for image upload software pict-rs (which is not yet usable by users)
  • s3-type object storage for uploaded media
  • RDS-type managed database
  • Terraform cloud for infrastructure orchestration
  • Grafana cloud for monitoring
  • Some mail service for sending e-mails - not settled on a provider yet, it might be Postmark, but their support has been less than helpful so far, so it might end up being something else as well

ActivePub – is it mirroring all content from all other instances locally?

Yes! Everything is cached locally, so even if other instances go down, their content will still be visible here (and vice versa).

[–] [email protected] 0 points 1 year ago* (last edited 1 year ago) (1 children)

Thank you so much for the details!!!

Re: ActivePub (If I'm not imposing), what does your current storage volume/growth look like? My long-term concern was that, at some point, smaller instances with few users wouldn't be able to keep up with the cost of data growth or sync overhead. But it does at least look like media is requested from the instance it was created on no matter what instance you're browsing.

[–] [email protected] 1 points 1 year ago* (last edited 1 year ago) (1 children)

The database is at around 400mb currently, and most of that is in a big table that is constantly getting cleaned up. Posts and comments are just text, so I don't expect it to grow out of hand too quickly (hopefully).

I'm much more worried about image uploads. For example, the default image size limit that Lemmy ships with is 40mb. You can imagine how quickly a few abusive users could fill up an instance with 40mb images (not to mention use up a lot of bandwidth), but even without abuse, if a bigger instance has hunderds of users constantly posting images, the storage costs will add up very quickly.

[–] [email protected] 1 points 1 year ago (1 children)

Indeed. I'm just sampling some from r/pics, even the largest/highest-res photos I saw clock in at ~1MB or just over (jpg, webp). I'm going to dive into Lemmy's source and see what they're doing... I imagine the 40mb is an upload limit on the pre-compressed filesize (still enormous). It would be ideal if they shipped with sliders for "compression aggression" or let you pass flags into ImageMagick or whatever it's using. Even 1MB/image would get expensive fast unless you could prune old images or move into a cheaper cold storage product.

[–] [email protected] 2 points 1 year ago (1 children)

I've definitely seen some >10MB files in places like /r/EarthPorn and various wallpaper threads. That'd definitely get impactful fast. Cloudflare will help, especially if you can set nice long TTLs for caching. I'm glad S3 type storage is something that works with Lemmy, I was actually wondering this myself. You could set that to Infrequent or "Cold" access polices after a day or so to throw it into a cheaper bucket for long term storage being that Cloudflare will take the brunt of the hits.

Side note: It warms my DevOps heart to hear this setup. Definitely sounds scalable and easily duplicated or redeployed if need be. Major props and I do thank you for this setup!

Question: I haven't looked into it, but why does the upload server have to be stateful? Perhaps it could be a lambda or similar?

[–] [email protected] 1 points 1 year ago

I haven’t looked into it, but why does the upload server have to be stateful? Perhaps it could be a lambda or similar?

Lemmy uses pict-rs to process uploaded images, which requires a database to index all the uploaded media. Unfortunately, pict-rs currently only supports a local on-disk database for this purpose. I've seen some talk that in the future it could also support postgresql for it's database (which Lemmy uses anyway) - that could simplify things a bit for sure.

[–] [email protected] 0 points 1 year ago (1 children)

Please put this on as a sticky or as a wiki. I had this question yesterday but was too tired to type. Do you want to set up a donation as well?

[–] [email protected] 1 points 1 year ago

It seems several people have expressed their willingness to share the costs, I will see in the coming days if I can set up some kind of system for this. I think it makes sense to put all the relevant info about the infrastructure on whatever page I end up using for the donations - I'll keep you guys posted.

[–] [email protected] 0 points 1 year ago (1 children)

However, if this growth continues for an extended period of time

What do you expect the bottleneck to be as the platform/server grows? Is it user accounts taking up space, different people subscribing to different federated servers or the cpu/bandwidth for serving users? Or something else?

[–] [email protected] 1 points 1 year ago

Allow me to copy my response to a very similar question from another thread:

It’s difficult to predict, as we’ve only been live for a few days, and there are many moving parts here. We have several pieces of our infrastructure which are being affected at different rates by the growth, and only time will tell which one gets close to the limit first.

For example, I am paying for:

  • Storage for uploaded media
  • Storage for the database (all posts, user data, comments, content from other instances, etc)
  • Bandwidth
  • E-mails
  • Just plain server resources like CPU/memory for multiple servers

I’m keeping a close watch on resource usage for now, I’ll definitely keep updating everybody as time goes by.

[–] [email protected] 0 points 1 year ago* (last edited 1 year ago) (1 children)

Hey, thanks for creating the instance and keeping it ship shape!

Just a couple questions about timelines

When do you expect the patch for image posting might be done? - I'm not knowledgeable at all about any of this stuff, does the patch mean we'll be able to post images and they'll show up organically/embedded as part of the post? (As opposed to seeing a link to Imgur where you can't see the actual image)

Re growth of lemm.ee - How long would you expect the 'extended period of time' to be if growing rate stays constant? It's exciting to think we could be growing so much/fast!

Edit formatting

[–] [email protected] 1 points 1 year ago (1 children)

When do you expect the patch for image posting might be done?

I'm hoping during this week, but can't guarantee anything. I'll keep you guys posted.

does the patch mean we’ll be able to post images and they’ll show up organically/embedded as part of the post?

Yes, as long as the images are <100kb 😀

How long would you expect the ‘extended period of time’ to be if growing rate stays constant?

It's difficult to predict, as we've only been live for a few days, and there are many moving parts here. We have several pieces of our infrastructure which are being affected at different rates by the growth, and only time will tell which one gets close to the limit first.

For example, I am paying for:

  • Storage for uploaded media
  • Storage for the database (all posts, user data, comments, content from other instances, etc)
  • Bandwidth
  • E-mails
  • Just plain server resources like CPU/memory for multiple servers

I'm keeping a close watch on resource usage for now, I'll definitely keep updating everybody as time goes by.

[–] [email protected] 0 points 1 year ago (1 children)

Yes, as long as the images are <100kb 😀

Ahh I see, I thought the patch would allow for bigger (say <3-5mb) images to be uploaded. Does that mean it's not in the plans at all to be able to have larger sizes uploaded?

[–] [email protected] 0 points 1 year ago (1 children)

If it seems like a 100kb limit won't make a dent on our infrastructure, then I'm open to increasing it, but it's something that will need to be monitored carefully. Unfortunately storage is expensive, so I need to be careful about this in particular.

[–] [email protected] 1 points 1 year ago (1 children)

What're your thoughts on using something like storj for decentralized data storage?

[–] [email protected] 1 points 1 year ago* (last edited 1 year ago)

AFAIK storj has an s3-compatible API, and we're currently also on an s3-compatible service, so it shouldn't be very difficult to switch if necessary. But for now, I am quite happy with our service provider - it's one of the chepaer ones already (certainly far cheaper than AWS S3).