this post was submitted on 09 Jul 2023
37 points (100.0% liked)

Python

6416 readers
5 users here now

Welcome to the Python community on the programming.dev Lemmy instance!

📅 Events

PastNovember 2023

October 2023

July 2023

August 2023

September 2023

🐍 Python project:
💓 Python Community:
✨ Python Ecosystem:
🌌 Fediverse
Communities
Projects
Feeds

founded 1 year ago
MODERATORS
[email protected]
[email protected]
[email protected]
37
How to improve Python packaging, or why fourteen tools are at least twelve too many (chriswarrick.com)
submitted 1 year ago* (last edited 1 year ago) by [email protected] to c/[email protected]
18 comments fedilink hide all child comments
 

I stumbled upon this while researching package management options for python, and found it a really interesting read.

I like python as a language but this mess is something that needs to be addressed for me to consider python for future projects. I can't imagine how confusing it must be for new users.

you are viewing a single comment's thread
view the rest of the comments
[–] coffeewithalex 11 points 1 year ago (1 children)

It's worth noting that there are basically just 3 systems worth considering, maybe even just 2.

pip is usually part of the python distribution, so any lightweight project can be finished in 1-5 minutes with pip. It's also quite widespread and the vast majority of publishers (if not all) target pip compatibility.

Poetry is a great project management framework and it deals with dependency management beautifully. If you're doing any data engineering or backend development, for any project that has more than 1 dependency and 200 lines of code, then Poetry is probably the best tool to use. Poetry makes the whole mess with helper tools like pip-tools seem outdated.

Conda is for the crazy world of data science libraries where developers don't bother with compatibility too much. Conda does it for them. And the users of those libraries can benefit from using conda.

I think the big competition is between poetry and pip. Maybe one day poetry will come as part of some Python distributions.

[–] [email protected] 4 points 1 year ago* (last edited 1 year ago) (1 children)

Worth noting the thing that conda does well is handle system-level dependencies, there are a lot more binaries available there (beyond just Python packages) that make it easier to install packages without having to compile C++ code or something.

Besides that I hate it haha, it makes everything else so complicated

I’ve also not had great experiences with poetry, it’s supposed to use the lock file to lock in dependencies but whenever I did anything the lock file was always being updated which kind of defeats the point. I’m sure it was user error but the fact that the lock file doesn’t stay static by default is already weird to me

[–] [email protected] 2 points 1 year ago (1 children)

It would be interesting to hear what you mean about the lock file being updated. Many Poetry commands should and do touch the lock, like poetry add or poetry update, but of course poetry installshould leave it untouched.

[–] [email protected] 2 points 1 year ago (1 children)

But if I want to add a single new dependency, then I probably don’t want all the rest updated at the same time

[–] coffeewithalex 2 points 1 year ago (1 children)

That can't be achieved due to dependency compatibility. What if you installed y==1.4, and froze it for a while, and then you install x==3.2, and it depends on y==1.5 or later?

pyproject.toml defines dependency restrictions, so it will be in accordance with that, but the lock file will change every time you add/remove dependencies. Naturally.

[–] [email protected] 1 points 1 year ago (1 children)

I don’t find that behaviour natural unless there is a hard conflict or I request it. So I guess it’s just a philosophical difference that led me to having a bad tint with it.

[–] [email protected] 3 points 1 year ago

If you use poetry add it should only update what is necessary, and you can use poetry lock --no-update to lock without updating everything.