this post was submitted on 25 Oct 2023
5 points (100.0% liked)

Python

6229 readers
9 users here now

Welcome to the Python community on the programming.dev Lemmy instance!

πŸ“… Events

October 2023

November 2023

PastJuly 2023

August 2023

September 2023

🐍 Python project:
πŸ’“ Python Community:
✨ Python Ecosystem:
🌌 Fediverse
Communities
Projects
Feeds

founded 1 year ago
MODERATORS
[email protected]
[email protected]
5
Unable to include data files in python package with setuptools. (self.python)
submitted 11 months ago* (last edited 10 months ago) by breadcrumb to c/[email protected]
3 comments fedilink hide all child comments
 

I want to build a python package using setuptools. The folder structure of the project is the following (some non-essential parts have been deleted):

energy-monitor
β”œβ”€β”€ config
β”‚ β”œβ”€β”€ config.yml
β”‚ └── secrets.yml
β”œβ”€β”€ data
β”‚ └── cpu_tdp.json
β”œβ”€β”€ energy_monitor
β”‚ β”œβ”€β”€ core
β”‚ β”‚ β”œβ”€β”€ gui.py
β”‚ β”‚ β”œβ”€β”€ __init__.py
β”‚ β”œβ”€β”€ data
β”‚ β”‚ └── tableExport.json
β”‚ β”œβ”€β”€ __init__.py
β”‚ β”œβ”€β”€ main.py
β”‚ └── utils
β”‚     β”œβ”€β”€ api_calls.py
β”‚     └── __init__.py
β”œβ”€β”€ energy-monitor.py
β”œβ”€β”€ LICENSE
β”œβ”€β”€ MANIFEST.in
β”œβ”€β”€ pyproject.toml
β”œβ”€β”€ README.md
└── requirements.txt

The content of the pyproject.toml file is the following (some non-essential parts have been deleted):

[build-system]
requires = ["setuptools>=68.0"]
build-backend = "setuptools.build_meta"

[project]
name = "energy_monitor"
version = "0.0.1"
description = "Energy monitor"
readme = "README.md"
requires-python = ">=3.11"
license = {text = "GPLv3"}
classifiers = [
  "Programming Language :: Python :: 3",
  "Operating System :: OS Independent",
]
dynamic = ["dependencies"]

[tool.setuptools.dynamic]
dependencies = {file = ["requirements.txt"]}

[tool.setuptools]
packages = [
  "energy_monitor", 
  "energy_monitor.core", 
  "energy_monitor.utils"
]
include-package-data = true

[project.scripts]
energy-monitor = "energy_monitor.main:main"

Finally, the content of the MANIFEST.in file is the following:

include README.md
include LICENSE
graft config

I generate the package with python -m build and install the .tar.gz archive with pipx. According to setuptools documentation, I expect to find my config folder, together with README and LICENSE in the interpreter directory (site-packages) after the installation. However, this doesn't happen and I cannot run the app becase it complains that it doesn't find the config. What am I missing?

top 3 comments
sorted by: hot top controversial new old
[–] [email protected] 2 points 11 months ago

I don't have a proper answer, but if you are up to try ditching the setuptools backend, flit has great docs and I've never had a problem with it.

[–] [email protected] 2 points 10 months ago* (last edited 10 months ago)

Doesn't MANIFEST.in specify which other-than-the-known-package files and folders to include?

I bet you need to add them to the MANIFEST.in.

I use poetry, so I don't need the MANIFEST.in, but I used it use a setup.py a few years ago, which is why I somewhat know of it.

https://packaging.python.org/en/latest/guides/using-manifest-in/

[–] [email protected] 1 points 10 months ago* (last edited 10 months ago)

I don't have any experience with pipx and personally prefer to just skip the .toml and place the whole pyprojectsetup in setup.py.

With that method, I would write inside setup()

packages=find_packages()  # Include every python packages
package_data={  # Specify additional data files
    'yourpackagename': [
        'config/*'
        etc...
    ]
}

This would however require you to have a package folder which all your package files/folders are inside, meaning the top level repo folder should not have any files or other folders that you want to distribute. Your MANIFEST.in looks fine.