this post was submitted on 02 Jul 2023
235 points (98.0% liked)

Selfhosted

40437 readers
577 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago
MODERATORS
 

Looking to maybe self host my own instance, I'm still learning about the fediverse. If a different instance that I federate with hosts something illegal are there risks to me? Is anything from other instances hosted on my server like a copy of it? Or would I only end up hosting things my users post? I'm paranoid and sorry if this is a silly question.

top 50 comments
sorted by: hot top controversial new old
[–] [email protected] 124 points 1 year ago (4 children)

The Electronic Frontier Foundation wrote a pretty good blog post on the legality of the Fediverse, around the time Mastodon was getting popular. It probably applies to Lemmy too. It’s worth a read to familiarize yourself of what kind of legal things you’ll be getting yourself into. You’re on the right track; you can control you and your friends’ content, but you can’t control remote content that gets pushed to your server and that’s the part to worry the most.

https://www.eff.org/deeplinks/2022/12/user-generated-content-and-fediverse-legal-primer

One thing that stood out is to register yourself as a DMCA agent. It costs $6 or something. Having an agent on record gives instance admins certain protection.

[–] [email protected] 11 points 1 year ago (1 children)

This is awesome info. There should be a place to document all the nuance around hosting an instance plus some tips and tricks.

[–] [email protected] 17 points 1 year ago (1 children)

There should be a place to document all the nuance around hosting an instance plus some tips and tricks.

The Wiki: https://joinfediverse.wiki/What_is_Lemmy%3F

Hopefully it gets new contributors and maintainers from all the new users.

[–] [email protected] 5 points 1 year ago

Yikes, I didn't even know there was a wiki. Thank you!

[–] LunchEnjoyer 8 points 1 year ago

Awesome, thanks 🤘

[–] [email protected] 6 points 1 year ago

Thank ye,

I wonder how much of an impact being in the EU will have on that.

[–] VitaminH 6 points 1 year ago

This is great info, thank you!

[–] [email protected] 66 points 1 year ago* (last edited 1 year ago) (2 children)

Text is copied to your instance's database, but any images are hosted on the other instances and simply linked to. Worst case scenario, you get told to delete something that's illegal in the country in which you host the instance, you comply, and everything's peachy.

Edit: That being said, I'm currently hosting an instance for myself and a few friends, and it's been smooth-sailing. Just make sure to require email verification or admin approval for new sign-ups (or disable them entirely) if you don't want to be overrun with bots.

[–] VitaminH 20 points 1 year ago

Yes I'd only be allowing people I know personally to create accounts. No other registration would be allowed. The last thing I'd want to be is another one of those bot filled instances that have been popping up.

[–] [email protected] 16 points 1 year ago (1 children)

That isn't entirely true. I'm not exactly sure why, but I've definitely seen image posts made to remote communities that are hosted on my instance.

[–] [email protected] 14 points 1 year ago (1 children)

I've seen image posts I make on other instances, but the image is hosted on my own instance and just linked to the other instance.

[–] [email protected] 8 points 1 year ago (1 children)

That may explain it... point being, content for remote communities isn't entirely "remote". I'd like to understand what goes where a lot better. I've not found it explained anywhere, and I'm not a coder so can't just "read the code yourself".

[–] [email protected] 9 points 1 year ago (1 children)

I'm not familiar with lemmy, but I did pick up on the lotide code a bit recently (a similar project)

As I understand it, the text or html of the post end up in a sort of mailbox, then your server goes out to pick up the latest posts from there. It gets brought over to your instance, and then it lives there. Whatever happens, the posts your server collected are on your server, that's how they're displayed.

Then when you go to write a post, it's stored locally and if it's on a local community then it's stored there and a copy is sent to the mailbox for others, and if it's a remote community your server will reach out to the other server and drop the post there.

My lotide instance has some older posts from servers that stopped existing a long time ago because although it can't get in touch with the remote community, the posts it did receive are still there.

[–] [email protected] 2 points 1 year ago (1 children)

Yeah post content I understand. Linked or posted images though are not consistently handled, so I'm not sure what circumstances lead to my instance pulling the image from a remote community.

[–] [email protected] 2 points 1 year ago (1 children)

I don't think lemmy typically does. I'm often on networks that block a lot of the Internet, and even thumbnails on posts from other instances or their community images get blocked when I can't communicate with them.

Right now, your profile pic for example is coming from aussie.zone, and the community pic is coming from lemmy.world, but I'm on fbxl lemmy a completely different instance from either of them.

[–] [email protected] 1 points 1 year ago (1 children)

Yeah profile pics are fine. I'm specifically talking about image posts. Next time I spot one I'll see if the post came from a user on my instance. If not, I have no idea why the post image would be on my instance.

[–] [email protected] 3 points 1 year ago (1 children)

If you right click on the image, then click inspect, then you can see the url for the image. Often it's coming from somewhere else.

Incidentally, my soapbox instance is different then lemmy -- it acts as a proxy, so it soaks up all the images and then hosts them from my one server.

[–] [email protected] 2 points 1 year ago (3 children)

Nope... thats what I'm talking about.. image posts to remote communities, with the images being sourced from my instance. Like I said, next time I spot it I'll dig into it further.

What software is your instance running?

[–] [email protected] 1 points 1 year ago

I'm on Lemmy 0.18.0 right now. I host a number of other kinds of software too, I've been all-in on the fediverse for a few years.

load more comments (2 replies)
[–] [email protected] 22 points 1 year ago (2 children)

Federation is implemented by copying the content from other servers to your database and file system, so if your users subscribe to something from a different server it will be copied to your server.

But it will be only served to your users, not to the public. Only the communities hosted on your instance will be served to the public.

[–] [email protected] 7 points 1 year ago (1 children)

AFAIK only text is copied, media stays in the instance where the community is hosted.

[–] petunia 4 points 1 year ago (1 children)

It depends on the software. Some proxy all content from remote servers so you only connect to your home server (Mastodon). Others don't, instead they make clients load remote content themselves (Lemmy). If you use browser client you can see all the connections being made.

[–] [email protected] 4 points 1 year ago

Yes, depends on the software, the post is about lemmy so I was talking about lemmy

[–] [email protected] 3 points 1 year ago (2 children)

Interesting so I can't visit a Lemmy community as a magazine within kbin if I don't have an account?

[–] [email protected] 4 points 1 year ago

Yes you can, even on Mastodon, and you can subscribe to PeerTube channels in /kbin and Lemmy, etc.

[–] [email protected] 4 points 1 year ago (2 children)

You can if someone else subscribed to it in the past. If nobody ever did, then that community is unknown to kbin and you won't find any data on it whether you're logged in or not.

[–] [email protected] 2 points 1 year ago (1 children)

But you can discover it for your instance, no?

[–] [email protected] 4 points 1 year ago

Yep, you can search by name specifically on https://kbin.social/search (or your kbin instance) and subscribe to it, then it starts getting synced.

[–] [email protected] 2 points 1 year ago* (last edited 1 year ago) (1 children)

In that case, how often is the lemme community updated on the kbin instance? Does it download updates every time the user visits?

[–] [email protected] 1 points 1 year ago

My understanding is that instances have worker threads that continually pull new data from linked communities (the ones at least 1 person is subscribed to). It should be almost instant but recently it's sometimes delayed due to huge influx of traffic.

[–] [email protected] 21 points 1 year ago (1 children)

If you're in the US, The Communications Decency Act Section 230 has a couple powers.

  1. It removes liability to service providers for user generated content when active moderation is practiced, and

  2. It removes liability to service providers for any moderation actions taken to to moderate to reasonable community standards.

Prior to CDA230, the jurisprudence centered around 2 different cases. In one, an actively moderated system had illegal content and didn't remove it in time, and in another case, a non-actively moderated system had illegal content and didn't remove it in time. At that time, the actively moderated system was held to be liable for the illegal content, whereas the non-actively moderated system was held not to be liable for not removing the illegal content.

[–] [email protected] 12 points 1 year ago (1 children)

One caveat to that would be the DMCA, where liability protection as a service provider I think is contingent on there being a DMCA process available so infringing content can be removed.

I don't know enough about how that all works with the fediverse, however.

[–] CriticalMiss 7 points 1 year ago

The fediverse is still a relatively small thing, even with all the popularity it’s been getting.

So dmcas are yet to happen

[–] vynlwombat 9 points 1 year ago (2 children)

How much disk space would some need to plan for a small lemmy instance?

[–] [email protected] 11 points 1 year ago (2 children)

I'm running it in the smallest VPS of vultr with 25GB of disk.
This instance only has 3 users, with me being the only active. It says it's been up for almost a month and I've only used 3GB.

Here are the docker volumes which have the actual data of your instance, and from inside the DB the biggest table is the one called activity which the devs said it's only sometimes used to validate the data, but could be truncated if needed (there's a schedule task which only keeps up to 6 months).
Also the thing to have in mind is to properly configure the logs of whichever installation guide you follow.
After that I've seen other admins say the next biggest is the media uploaded (from bigger instances).

$ du -h --max-depth=1
640K    ./pictrs
3.2G    ./postgres
3.2G    .

lemmy=# select
  table_name,
  pg_size_pretty(pg_relation_size(quote_ident(table_name))),
  pg_relation_size(quote_ident(table_name))
from information_schema.tables
where table_schema = 'public'
order by 3 desc;
         table_name         | pg_size_pretty | pg_relation_size
----------------------------+----------------+------------------
 activity                   | 2187 MB        |       2292867072
 comment                    | 56 MB          |         58212352
 person                     | 48 MB          |         50307072
 comment_like               | 45 MB          |         47161344
 post_like                  | 22 MB          |         22781952
 comment_aggregates         | 14 MB          |         14811136
 post                       | 13 MB          |         13623296
[–] [email protected] 6 points 1 year ago (11 children)

The activity table is also used to deduplicate incoming federation data, so instead of truncating it, I'd suggest deleting rows after a certain amount of time.

For my personal instance, I set up a cron to delete entries older than 3 days, and my db is only ~500MB with a few weeks of content! I also haven't seen any duplicated posts or comments. Even with Lemmy's retries, 3 days seems to be long enough before dropping rows from that table.

[–] [email protected] 2 points 1 year ago* (last edited 1 year ago) (1 children)

Could you share the cron/script you use to do this? I'm interested in hosting my own Lemmy at some point, and having a script for that cleanup would be hugely helpful for me.

[–] [email protected] 2 points 1 year ago (2 children)

Definitely! I'm hosting in Kubernetes so I won't post the full thing, but here's the actual command that I run hourly. Make sure to replace the values for database, username, and password.

PGPASSWORD=password psql --dbname=database --username=username --command="DELETE FROM activity WHERE published < NOW() - INTERVAL '3 days';"
load more comments (2 replies)
load more comments (10 replies)
[–] [email protected] 2 points 1 year ago* (last edited 1 year ago) (2 children)

How are you keeping your pictrs directory so small?

Mine is at about 5GB after two weeks with just a single user. 😬

[–] [email protected] 1 points 1 year ago (1 children)

I also have around 3GB used for pictrs and I’m not really sure the best way to see what all content is in there.

[–] [email protected] 2 points 1 year ago (1 children)

Yeah I haven't uploaded any images on my instance myself. So none of those images are mine. Might do some reading tomorrow and see if there's any mention of this in the past on other communities. It's not an emergency but I'm curious.

[–] [email protected] 1 points 1 year ago (1 children)

That's strange. Please let me know what you find out.

[–] [email protected] 2 points 1 year ago

I had found an old post which indicates that post thumbnails are cached. So I guess there's that.

In case you didn't see it, the OP of this thread realized they didn't setup their pictrs API key.. so I guess it's possible to omit that and lemmy should still work. Not sure about the downsides.

[–] [email protected] 1 points 1 year ago (1 children)

Haha, I don't know xP.
Just checked and it has only one image.

[–] [email protected] 1 points 1 year ago (1 children)

Did you configure the pictrs API keys for Lemmy and for pictrs?

If they're not configured then I could see Lemmy not even using pictrs.

[–] [email protected] 1 points 1 year ago (1 children)

Ohh!!
That's what's happening, I haven't uploaded any pictures so I didn't noticed, aside from that I'm not sure what are the other use cases of pictrs

load more comments (1 replies)
[–] [email protected] 8 points 1 year ago

Well, here's my first post on the fediverse!

Background in IT and server administration here. I however do not know much about the intricacies of the fediverse, but am interested in learning. Here's my two cents based on a background of LAMP stacks for web hosting.

The required space would likely scale and vary greatly depending on how much content is hosted locally. Assuming minimum space similar to a basic LAMP server it'd likely have starting space requirements of less than 1GB. If local content is primarily text/links to content hosted elsewhere it would take a lot to drastically change that space requirement. Image hosting can vary greatly depending on size, quality, and number of images. Video hosting is an absolute space hog even at fairly low resolutions by today's standards.

Bandwidth requirements would scale similar to storage requirements.

Other specs would also start very low if fediverse requirements are similar to a LAMP stack. Cores are typically more important than core speed in web server hosting as each request will try to use a separate core, but doesn't need much processing power to provide that request since the server isn't actually rendering anything.

Likewise, you shouldn't need much memory on a web host. Will scale with the number of scripts running on the host but I suspect that shouldn't be many unless you're also running moderation bots, but those should ideally be run on a different server instance.

That said, I'd also be curious to hear from other people that have experience with the fediverse though and other recommended specs to potentially host an instance.

If anyone has other questions I'm happy to try to help :)

[–] [email protected] 7 points 1 year ago

Where i live there is "the hoster privilige" hosters dont have to remove user content, only of somebody reports the content to you

load more comments
view more: next ›