You are correct, and therefore there shouldn't be freedom of speech for the sake of it in any social media platform worth going to. "Free speech" ignores the values of respect, ettiquite and gives plenty of room for trolls and flamebaiters to play games with moderators. Seems like a nightmare for a server operator to manage.

To have a place where people can have level-headed discussion about topics including controversial ones, certain ground rules need to be set, which inherently are limits to speech. If you don't like the rules or how they are enforced, the freedom that you have in Lemmy/the Fediverse is to move to another server in line with what you think are better rules/rulers, or for you to bother with the moderation work yourself in your own community on your own server where no one tells you what to do.