this post was submitted on 15 Sep 2023
21 points (100.0% liked)

Python

5925 readers
15 users here now

Welcome to the Python community on the programming.dev Lemmy instance!

πŸ“… Events

October 2023

November 2023

PastJuly 2023

August 2023

September 2023

🐍 Python project:
πŸ’“ Python Community:
✨ Python Ecosystem:
🌌 Fediverse
Communities
Projects
Feeds

founded 1 year ago
MODERATORS
[email protected]
[email protected]
21
Why and How Does Python Use Bloom Filters in String Processing? (codeconfessions.substack.com)
submitted 9 months ago by [email protected] to c/[email protected]
4 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 3 points 9 months ago (1 children)

The article says that CPython represents strings as UTF-8 encoded, which is not correct. The details about how it works are correct, just that's not UTF-8.

That's just a minor point though, nice article.

[–] abhi9u 3 points 9 months ago (1 children)

Hi @qwop, I am the author. Thank you for reading and the kind words. I would like to understand the error I made better so that I don't repeat in future, and if I can fix it. Could you please clarify?

[–] [email protected] 3 points 9 months ago (1 children)

UTF-8 is an encoding for unicode, that means it's a way of representing a unicode string as actual bytes on a computer.

It is variable length and works by using the first bits of each byte to indicate how many bytes are are needed to represent the current character.

Python also uses an encoding, as you describe in the article, but it's different to UTF-8. Unlike unicode, all characters in Python's representation of the unicode string use the same number of bytes, which is the maximum that any individual unicode character in the string needs.

I'd probably mess up a more detailed explanation of UTF-8 or Python's representation, so I'll let you look into how they work in more detail if you're interested.

[–] abhi9u 1 points 9 months ago

Thank you! That's helpful. I spent quite some time trying to understand the difference between UTF-8 and Python's representation and arrived at the same understanding as you wrote. However, most of the external documents simply say that strings in Python are UTF-8 which made me conclude that perhaps I am missing something and it might be safer to write it as utf-8.

I will look more in the code as you suggested.