lemmy.ninja news

Greetings, ~~programs~~ -- er, ninjas:

Server Upgrades

We've been hard at work analyzing our analytics and tuning the site. If you've been watching our user growth, you would have noticed a dramatic increase in users last month corresponding to the Reddit exodus. Well, we certainly noticed. We started watching the server very closely and tuning it to give you the best experience. I won't go into too much boring detail, but I will say that the server has substantially more resources. We hope this will help speed up your Lemmy-browsing experience.

Software Updates

We continue to keep Lemmy as up-to-date as possible. We're now on Lemmy 0.18.4, which is the latest stable release. We work hard to upgrade to the newest versions as quickly as we can. If the inconceivable happens and we miss a major release, you can help by letting us know about it in this community.

Admin Actions

We've also been hard at work monitoring our connection to the rest of the Fediverse. Our philosophy continues to be to maintain connections to as many instances as we possibly can. This means that we focus our administrative actions on users rather than communities or instances. We've banned spam bots and bad actors from other sites. If you see someone breaking the rules here or elsewhere, please report them so that we can investigate and, if necessary, take action. We like to give everyone the benefit of the doubt as much as we can, and we strive for a permissive application of the rules. On the same token, however, we don't want to let a real problem fester and create serious issues for other administrators. So please help us keep the instance a friendly place by using that report button. It just brings the issue to our attention. No corrective action is taken automatically when you use it. And of course, thank you to everyone who has already helped us identify problems.

One of our tasks right now is to open more channels of communication between ourselves and the admins of other instances. This will help us better understand threats as they arise, and to deal with them before they impact your experience here.

Rules Updates

Speaking of rules, we have again removed some rules we felt were duplicative. Re-evaluating the rules and pruning excessive restrictions is an activity we revisit frequently. We want to make the rules easy to understand and remember. That usually means shortening them and keeping them simple, so don't be surprised if we shorten that sidebar even more in the future. We haven't added any new rules. If we do, you'll hear about it here.

Moderators Needed!

We don't have that many communities here at lemmy.ninja. But if you are subscribed to one and you don't mind acting as a moderator, we could really use your help! Reach out to me or to @[email protected] to let us know you're interested and we'll see about adding you to the team.

Our Roadmap

We plan to continue to improve Lemmy.ninja iteratively. Some things we have been considering are:

• A mobile interface for the site
• Alternative front-ends for you to use
• Live site analytics
• Collaborations with other instances

A lot of these things depend on how the site evolves and where we need to put our efforts. Nevertheless, these are some of the things that you might be seeing in the months ahead!

Wrapping Up

That's all for now, ninjas! If you have ideas for the site, feel free to comment below. Questions, too!

Update to our earlier announcement of kbin.social being temporarily blocked:

Unfortunately, the porn posts have continued to appear in our new feed for over 12 hours now. It's clear that we can't user-ban our way out of this.

Blocking kbin.social was a temporary measure that allowed us to stop the flow of spam posts, wait a while to see if kbin.social had gotten the problem under control, and then re-open the gates to check to see if more bots were posting links. Every time we checked, they were. However, they were studiously keeping to the [email protected] community, never once posting in any other community.

For now we have decided to remove the community and leave access to the rest of kbin.social open. This is a little risky, but leaves us connected to the rest of Kbin.social's content. If you were a subscriber to that community, you won't see any past or future posts in it. Nobody will be able to search for that community from lemmy.ninja, either.

We'll keep an eye on the "New" feed and take action if the bots migrate to another community on kbin.social. For now, however, consider them in chains!

For the last three hours, kbin.social has been under attack by a group of bot accounts. They've been posting hundreds of porn links to the [email protected] magazine. These posts have been flooding across to lemmy.ninja this morning. So far we've been able to remove the content and ban the bot users, but we're unable to keep up with the rate of posting.

We've decided to temporarily block kbin.social until the admins there can get the problem under control. We'll monitor the situation and restore connection to kbin.social once the attack has ceased.

Thanks for your understanding.

We've been hard at work updating and refining the site since we came online about five weeks ago. If you're a regular user of the site, you should notice significant speed and stability improvements, as well as a lot more content. Here's a quick recap of the changes we've made over the last month.

• Site Rules Refined: We've relaxed some of the rules, especially those surrounding NSFW content. We are not and probably will never be a host for dedicated NSFW communities, but our intent is not to prevent you from participating in NSFW communities on other instances. The new rules should make that clearer. We'll continue to refine and hopefully condense our site policies as time goes on.
• Defederation: As a general rule, Lemmy.ninja does not want to defederate from other Fediverse sites. Even so, we have chosen to block a couple of sites after repeated abuses of our site policies. We may also temporarily block sites who have fallen victim to spambot infestations and are imperiling our server. Rest assured that this is a last resort, and that we re-evaluate our block list very frequently in an effort to get sites off of it.
• Security and Site Enhancements: We are continually monitoring server load and taking steps to optimize and improve the server. While we have so far chosen only to deploy official releases of Lemmy software, we may deploy a release candidate if we feel the security enhancements provided by that release candidate are critical to the operation of the site. Long story short, we value stability over the bleeding edge, except when there's an imminent threat.
• New Users and Communities: As our user base has grown, so has our list of local communities. We would very much like to add more moderators to our communities. If you are interested, please reach out to the moderator(s) of the community you wish to volunteer for.

As always, if you have recommendations for the site, or questions about the rules or how we operate, or if you just want to be social and talk about how god-awful the heat is, please feel free to post in our town square, the Ninja Tea Room. Until next time, stay smoky, ninjas!

If you or a friend are looking for a set of directions that can help you find a lemmy instance to call home, this video walks you through the process step by step.

Summary

We started a Lemmy instance on June 13 during the Reddit blackout. While we were configuring the site, we accumulated a few thousand bot accounts, leading some sites to defederate with us. Read on to see how we cleaned up the mess.

Introduction

Like many of you, we came to Lemmy during the Great Reddit Blackout. @MrEUser started Lemmy.ninja on the 13th, and the rest of us on the site got to work populating some initial rules and content, learning how Lemmy worked, and finding workarounds for bugs and issues in the software. Unfortunately for us, one of the challenges to getting the site up turned out to be getting the email validation to work. So, assuming we were small and beneath notice, we opened our registration for a few days until we could figure out if the problems we were experiencing were configuration related or software bugs.

In that brief time, we were discovered by malicious actors and hundreds of new bot users were being created on the site. Of course we had no idea, since Lemmy provides no user management features. We couldn't see them, and the bots didn't participate in any of our local content.

Discovering the Bots

Within a couple of days, we discovered some third-party tools that gave us the only insights we had into our user base. Lemmy Explorer and The Federation were showing us that a huge number of users had registered. It took a while, but we eventually tracked down a post that described how to output a list of users from our Lemmy database. Sure enough, there were thousands of users there. It took some investigation, but we were eventually able to see which users were actually registered at lemmy.ninja. There were thousands, just like the third-party tools told us.

Meanwhile...

While we were figuring this out, others in Lemmy had noticed a coordinated bot attack, and some were rightly taking steps to cordon off the sites with bots as they began to interact with federated content. Unfortunately for us, this news never made it to us because our site was still young, and young Lemmy servers don't automatically download all federated content right away. (In fact, despite daily efforts to connect lemmy.ninja to as many communities as possible, I didn't even learn about the lemm.ee mitigation efforts until today.)

We know now that the bots began to interact with other Mastodon and Lemmy instances at some point, because we learned (again, today) that we had been blocked by a few of them. (Again, this required third-party tools to even discover.) At the time, we were completely unaware of the attack, that we had been blocked, or that the bots were doing anything at all.

Cleaning Up

The moment we learned that the bots were in our database, we set out to eliminate them. The first step, of course, was to enable a captcha and activate email validation so that no new bots could sign up. [Note: The captcha feature was eliminated in Lemmy 0.18.0.] Then we had to delete the bot users.

Next we made a backup. Always make a backup! After that, we asked the database to output all the users so we could manually review the data. After logging into the database docker container, we executed the following command:

select
  p.name,
  p.display_name,
  a.person_id,
  a.email,
  a.email_verified,
  a.accepted_application
from
  local_user a,
  person p
where
  a.person_id = p.id;

That showed us that yes, every user after #8 or so was indeed a bot.

Next, we composed a SQL statement to wipe all the bots.

BEGIN;
CREATE TEMP TABLE temp_ids AS
SELECT person_id FROM local_user WHERE person_id > 85347;
DELETE FROM local_user WHERE person_id IN (SELECT person_id FROM temp_ids);
DELETE FROM person WHERE id IN (SELECT person_id FROM temp_ids);
DROP TABLE temp_ids;
COMMIT;

And to finalize the change:

UPDATE site_aggregates SET users = (SELECT count(*) FROM local_user) WHERE site_id = 1;

If you read the code, you'll see that we deleted records whose person_id was > 85347. That's the approach that worked for us. But you could just as easily delete all users who haven't passed email verification, for example. If that's the approach you want to use, try this SQL statement:

BEGIN;
CREATE TEMP TABLE temp_ids AS
SELECT person_id FROM local_user WHERE email_verified = 'f';
DELETE FROM local_user WHERE person_id IN (SELECT person_id FROM temp_ids);
DELETE FROM person WHERE id IN (SELECT person_id FROM temp_ids);
DROP TABLE temp_ids;
COMMIT;

And to finalize the change:

UPDATE site_aggregates SET users = (SELECT count(*) FROM local_user) WHERE site_id = 1;

Even more aggressive mods could put these commands into a nightly cron job, wiping accounts every day if they don't finish their registration process. We chose not to do that (yet). Our user count has remained stable with email verification on.

After that, the bots were gone. Third party tools reflected the change in about 12 hours. We did some testing to make sure we hadn't destroyed the site, but found that everything worked flawlessly.

Wrapping Up

We chose to write this up for the rest of the new Lemmy administrators out there who may unwittingly be hosts of bots. Hopefully having all of the details in one place will help speed their discovery and elimination. Feel free to ask questions, but understand that we aren't experts. Hopefully other, more knowledgeable people can respond to your questions in the comments here.

Lemmy.ninja has been updated to the latest version of Lemmy. There are a whole lot of bug fixes in this release. You should notice some UI fixes as well.

As a user, you'll notice a lot fewer situations where you just see a spinning ring. New users will have to validate their email address (that's finally working!).

We recommend that you enable two-factor authentication for your account as well. This will make your account more secure and ensure that you don't lose it to a bad actor.

Q. What is Lemmy.ninja?

A. Lemmy.ninja is your portal to Lemmy, a social media network similar to Reddit. Unlike Reddit, there's no monolithic central server where everyone posts their content. Instead, content is organized under Lemmy communities on various different Lemmy sites. A community is like a subreddit -- it contains posts organized around the community's theme. At the time of this writing, there are around 13,000 communities hosted on just under 1,000 separate Lemmy sites. There are a few communities hosted on Lemmy.ninja, but the vast majority of communities you will subscribe to and participate in reside on other Lemmy sites. You can view all of the content, regardless of where it's hosted, at Lemmy.ninja.

Q. I created an account, but I can't find my validation email

A. It probably went to your spam folder. We've investigated reports of missing validation emails, and every one of them ended up in the spam folder.

Q. How do I subscribe to content?

A. Go to the list of Communities. By default, you will see the communities hosted at Lemmy.ninja. Change the list to show all communities. At Lemmy.ninja, we have tried to pre-populate as many interesting communities as we can. Click subscribe next to any and all communities you are interested in. Once you do that, content from those communities will start to appear on your feed. You can interact with that content -- leave comments, upvote and downvote, send direct messages, etc. -- all from your feed on Lemmy.ninja.

Q. How do I find a community that isn't on the list of All Communities at Lemmy.ninja?

A. Use the Lemmy Community Browser and search for a community that you want to visit. Pay attention to how many posts, users, and comments the community has. With Lemmy being so new, many communities have very low participation. Try to pick a commuinity with high participation. Once you have found a community you like, the process for subscribing to it on Lemmy is tricky. This is because the Lemmy software is still very new and has many rough edges. Here is what we've found is the best process:

1. Copy the URL for the community to your clipboard. (e.g., https://beehaw.org/c/technology)
2. Go to the list of communities at Lemmy.ninja.
3. Paste the URL you copied earlier into the search box.

If your commlunity was added successfully, you will see it appear in the search results after a few seconds! Now all you have to do is subscribe to the community for the content to start flowing to Lemmy.ninja and your feed.

4. Click on the community you just added.
5. On the right of your screen, click Subscribe.

That's it! Now when you visit your feed at Lemmy.ninja, you will see posts from your new community appearing and you can interact with them.

Q. I've subscribed to communities, but my feed doesn't update very much.

A. Part of the reason for this is that Lemmy is young. There's a lot of content out there, but not quite enough to make the Active sort type work properly. Try setting your sort type to New instead.

Q. I subscribed to a community, but there are zero comments on every post! What's going on?

A. If you subscribed to a community that has never been subscribed to at Lemmy.ninja before, then it will take time for the posts and comments to be loaded. This only happens the first time a community is added to Lemmy.ninja. After about an hour, you should notice all the content has been loaded and you won't have any loading delays in the future.

Q. Can I create my own community at Lemmy.ninja?

A. Yes! We only ask that you ensure that your community has

• An icon
• A banner
• Active moderators
• Community rules consistent with the Lemmy.ninja site rules

Q. Some of the communities I have subscribed to show "Subscribe Pending" on the Lemmy.ninja communities page. What is this?

A. This is a known bug. It happens when subscribing to communities on overloaded Lemmy servers. You are actually subscribed, and content from that server is reaching your feed.

Q. What is kbin?

A. Kbin is software that can be used to access the same content as a Lemmy server. There are subtle differences between Lemmy and Kbin, but they can access each other's posts and comments seamlessly.

Q. How do I subscribe to a kbin magazine from Lemmy.ninja?

A. @[email protected] posted this solution for subscribing to kbin content from Lemmy, which we have edited here to match the lemmy.ninja instance.

1. Note the name of the kbin magazine you want to subscribe to (example: kbin.social/m/books)
2. Log into your Lemmy instance from a browser
3. The Lemmy instance you are registered with will be the first part of the URL and the kbin community will be the second part. Example: I’m registered with lemmy.ninja so to register with the kbin Books magazine I would type https://lemmy.ninja/c/[email protected]
4. From there, just hit subscribe and it will start showing up anywhere you’re logged into your Lemmy instance

Site rules for Lemmy.ninja have been posted. Please familiarize yourself with these rules.

Lemmy.ninja Instance Rules

• Be respectful: Treat other users with respect and kindness. Personal attacks, harassment, or any form of hate speech will not be tolerated.
• Stay on topic: Keep discussions relevant to the respective community or post. Avoid going off-topic or derailing conversations.
• No spam or self-promotion: Do not use Lemmy.ninja solely for self-promotion or excessive advertising. Promote your content or products in designated areas, such as the appropriate community or specific promotion threads.
• No illegal content: Do not post or share any content that is illegal, promotes illegal activities, or violates intellectual property rights. This includes sharing copyrighted material without permission.
• Provide accurate information: Share reliable and fact-checked information. Misinformation or deliberate spreading of false information is strongly discouraged.
• No personal information: Do not post personal information about yourself or others without consent. Respect privacy and confidentiality.
• Keep it SFW (Safe for Work): Avoid posting explicit, pornographic, or NSFW (Not Safe for Work) content. Lemmy.ninja aims to be an inclusive and accessible platform for users of all ages.
• No trolling or flame wars: Engage in constructive discussions and avoid trolling, flaming, or engaging in prolonged arguments that disrupt the community. Disagreements are allowed, but they should be handled respectfully.
• Report violations: If you come across any content or behavior that violates the rules or community guidelines, report it to the moderators or site administrators promptly.
• Follow community guidelines: Each community on Lemmy.ninja may have specific guidelines or rules. Familiarize yourself with these guidelines and adhere to them when participating in those communities.
• Use appropriate language: Refrain from using offensive or abusive language. Maintain a civil tone and communicate in a manner that promotes a positive and inclusive environment.
• Be mindful of the voting system: Use upvotes and downvotes to express your opinion on the quality and relevance of content. Do not engage in vote manipulation or brigading.

After a brief period of testing, Lemmy.ninja has transitioned to the use of a block list for instances. This means that all Lemmy instances (and their communities) are accessible at Lemmy.ninja. Admins reserve the right to ban a Lemmy instance in the future, but as of right now there are no instances on the block list. This should give you unfettered access to all Lemmy content regardless of server.

That said, Lemmy.ninja still has site rules, which you can see on the sidebar after you log in. If a Lemmy instance is discovered to violate those rules, admins reserve the right to block that instance at their sole discretion.