Packaging & Publishing to PyPI

A package is what someone installs when they run pip install yourpkg. Inside, it's a directory of Python files plus metadata, compressed into a wheel and uploaded to the Python Package Index (PyPI). This used to be complicated: setup.py, MANIFEST.in, eggs, and three competing build systems. Modern Python uses a single file that describes the package: pyproject.toml. This lesson uses the modern way from start to finish.

By the end you'll have a project layout, a working pyproject.toml, a wheel on TestPyPI, and enough practice to do it again on the real PyPI tomorrow.

1. The Big Picture

A published Python package has three parts:

1. Source code: your modules and subpackages.
2. Metadata: name, version, description, dependencies, entry points, classifiers.
3. A built artifact: usually a .whl (wheel) and a .tar.gz (sdist) sitting in dist/.

You write the first two. A build backend turns them into the third. pip downloads the wheel and unpacks it into the user's site-packages. That's the whole process.

The old way used setup.py, a Python script that was the metadata. The modern way uses pyproject.toml, a static TOML file. New projects should never start with setup.py. If you see it in a tutorial dated after 2023, the tutorial is out of date.

2. The Minimum Project Layout

quickstats/
    pyproject.toml
    README.md
    LICENSE
    src/
        quickstats/
            __init__.py
            core.py
            cli.py
    tests/
        test_core.py

There are two common layouts: flat (quickstats/ at the top level next to pyproject.toml) and src (src/quickstats/). Use the src layout. Here's why:

• With a flat layout, import quickstats works from the project root because the current directory is on sys.path, even if the package isn't installed. Your tests can pass on your machine while the installed package is broken for everyone who pip installs it.
• The src/ layout forces you to install the package (pip install -e .) before importing it. Tests then exercise the same import path your users do.

A quick check: if pytest passes on your tests/ directory without the package being installed, your tests are importing the code from the project folder, not the installed package.

3. A Minimal pyproject.toml

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "quickstats"
version = "0.1.0"
description = "Tiny statistics helpers for everyday data work."
readme = "README.md"
requires-python = ">=3.10"
license = { file = "LICENSE" }
authors = [
    { name = "Hedy", email = "hedy@example.com" },
]
dependencies = [
    "numpy>=1.24",
]

[project.optional-dependencies]
dev = ["pytest>=7", "ruff", "mypy"]

[project.scripts]
quickstats = "quickstats.cli:main"

[project.urls]
Homepage = "https://github.com/surya/quickstats"
Issues = "https://github.com/surya/quickstats/issues"

Walking through it:

• [build-system]: declares which tool builds your wheel. hatchling is the modern choice: fast, well maintained, with sensible defaults. Alternatives below.
• [project]: PEP 621 metadata. name and version are the only required fields, but everything else is strongly recommended.
• dependencies: runtime requirements. For libraries, use loose version limits (>=1.24), because exact pins can make your package impossible to install together with other packages.
• [project.optional-dependencies]: extras. Users opt in with pip install quickstats[dev].
• [project.scripts]: entry points. Installing quickstats puts a quickstats command on the user's PATH that calls quickstats.cli:main.

4. Build Backends: Pick One

If you have no opinion, use hatchling. The rest of this lesson assumes it.

5. The src/quickstats/__init__.py

# src/quickstats/__init__.py
"""quickstats: tiny statistics helpers."""

from .core import mean, median, stdev

__version__ = "0.1.0"
__all__ = ["mean", "median", "stdev"]

The __init__.py defines the public API that you choose to offer. Re-export the names you want at the top level so users write from quickstats import mean instead of from quickstats.core import mean. Anything not re-exported is treated as internal.

A matching cli.py:

# src/quickstats/cli.py
import sys
from .core import mean, median, stdev

def main():
    if len(sys.argv) < 2:
        print("Usage: quickstats <numbers...>")
        sys.exit(1)
    values = [float(x) for x in sys.argv[1:]]
    print(f"mean   = {mean(values):.3f}")
    print(f"median = {median(values):.3f}")
    print(f"stdev  = {stdev(values):.3f}")

[project.scripts] in pyproject.toml points at quickstats.cli:main, so installing the package puts a quickstats command on the user's PATH.

6. Versioning

SemVer (MAJOR.MINOR.PATCH) is the default convention:

• MAJOR: breaking changes. Anything that could break a caller's existing code.
• MINOR: new features, backward-compatible.
• PATCH: bug fixes only.

Pre-releases: 0.2.0a1 (alpha), 0.2.0b1 (beta), 0.2.0rc1 (release candidate). pip skips pre-releases unless you pass --pre.

Calendar versioning (2026.05.0) is used by some projects (pip itself, Ubuntu). Fine for tools where users care about freshness more than API stability, but less suited to libraries.

The version lives in pyproject.toml. Don't also hardcode it in __init__.py unless you keep them in sync. See single source of truth below.

7. Building

pip install build
python -m build

That produces:

dist/
    quickstats-0.1.0-py3-none-any.whl
    quickstats-0.1.0.tar.gz

The wheel (.whl) is the install artifact. The sdist (.tar.gz) is the source archive, used as a fallback when no wheel matches the user's platform.

8. Editable Installs: The Local Development Loop

pip install -e .
pip install -e ".[dev]"             # with dev extras

-e installs the package as a link, so your src/quickstats/ directory is the installed package. When you edit a file, the change applies the next time you run python, with no rebuild. This way you don't have to run pip install again after every change.

Quick check after installing:

python -c "import quickstats; print(quickstats.__version__)"
quickstats 1 2 3 4 5                # the CLI entry point

9. Publishing to TestPyPI First

TestPyPI is the staging environment. Every release goes there first.

pip install twine
twine upload --repository testpypi dist/*

You'll need an account on https://test.pypi.org/ and an API token (Account settings → API tokens). The username is the literal text __token__, and the password is the token string (it starts with pypi-). PyPI no longer accepts your account password for uploads.

Store credentials in ~/.pypirc:

[testpypi]
  username = __token__
  password = pypi-AgEIcHlwaS5vcmcCJ...

Verify the package installs cleanly from TestPyPI in a fresh venv:

python -m venv /tmp/check
source /tmp/check/bin/activate
pip install --index-url https://test.pypi.org/simple/ \
            --extra-index-url https://pypi.org/simple/ \
            quickstats
python -c "import quickstats; print(quickstats.__version__)"

The --extra-index-url is needed because TestPyPI doesn't have copies of your runtime dependencies (numpy, etc.), so pip gets those from the real PyPI.

10. Publishing to Real PyPI

Once TestPyPI looks clean:

twine upload dist/*

That's it. Within a minute, pip install quickstats works for anyone.

You cannot overwrite a release. PyPI lets you yank a version (this hides it from new installs, but anyone who pinned that exact version can still install it), and you can delete a release. But you can never upload the same version number again with different content. If you broke something, bump the version and ship 0.1.1.

11. Trove Classifiers and Discoverability

Classifiers are how users find your package via the PyPI search filters:

classifiers = [
    "Development Status :: 4 - Beta",
    "Intended Audience :: Developers",
    "License :: OSI Approved :: MIT License",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Topic :: Scientific/Engineering :: Mathematics",
]

The full list is at https://pypi.org/classifiers/. Pick three to five that really describe your package. Don't add extra ones just to appear in more searches.

12. Including Data Files

If your package ships templates, JSON schemas, or any non-Python files, the build backend needs to know to include them. For hatchling:

[tool.hatch.build.targets.wheel]
packages = ["src/quickstats"]

[tool.hatch.build.targets.wheel.force-include]
"src/quickstats/data/schema.json" = "quickstats/data/schema.json"

Most of the time hatchling picks up everything under src/<pkg>/ automatically. Use explicit force-include only when files are somewhere else.

Access the file at runtime with importlib.resources, not by building a path from __file__, which can break when the package is imported from a zip file.

13. Auto-Versioning From Git Tags

If you hard-code the version in pyproject.toml and again in __init__.py, the two will drift apart. The fix is a single source of truth, usually a git tag.

[build-system]
requires = ["hatchling", "hatch-vcs"]
build-backend = "hatchling.build"

[project]
name = "quickstats"
dynamic = ["version"]               # version comes from somewhere else

[tool.hatch.version]
source = "vcs"

[tool.hatch.build.hooks.vcs]
version-file = "src/quickstats/_version.py"

Now git tag v0.2.0 is the version. The build reads the tag, writes _version.py, and the wheel is stamped automatically. setuptools-scm does the same for setuptools projects.

14. Type Hints and the py.typed Marker

If your package has type hints and you want the people who use it to get them, add an empty py.typed marker file at the top of the package:

src/quickstats/
    __init__.py
    py.typed                        # an empty file. Its presence is the signal.
    core.py

Then in pyproject.toml:

[tool.hatch.build.targets.wheel]
packages = ["src/quickstats"]
include = ["src/quickstats/py.typed"]

Without py.typed, tools like mypy treat your package as untyped: mypy reports an error on the import and treats everything from your package as Any. With the marker, users get your types.

15. The CI Release Pattern

The usual workflow for mature projects: you create a tag in git, and CI builds and publishes the package. You push a tag and the release happens without any more work from you.

# .github/workflows/release.yml
name: release
on:
  push:
    tags: ["v*"]

jobs:
  publish:
    runs-on: ubuntu-latest
    permissions:
      id-token: write               # for trusted-publisher OIDC
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12" }
      - run: pip install build
      - run: python -m build
      - uses: pypa/gh-action-pypi-publish@release/v1

Trusted publishing (OIDC) is the modern way. There is no API token in your CI secrets, because GitHub Actions authenticates directly with PyPI. Configure it once in your PyPI project settings.

Common Mistakes

1. setup.py in new code

If you're starting a project in 2026, write pyproject.toml and never touch setup.py. The only setup.py you might write today is the three-line file that some old tools still expect, and even that is almost never needed now.

2. Flat layout instead of src/

quickstats/
    quickstats/             # flat layout: importable from the project root without installing
        __init__.py

Tests pass for you but the package fails for users. Move the package into src/quickstats/, and your tests then use the installed package, the same as everyone else.

3. No README on PyPI

If you forget readme = "README.md" in pyproject.toml, your project page on PyPI is empty. The project looks unmaintained, and users may not trust it. Always set readme.

4. Drifting versions

# pyproject.toml
version = "0.2.0"

# __init__.py
__version__ = "0.1.9"           # forgotten on the last bump

With two places to update, they will drift apart sooner or later. Use hatch-vcs/setuptools-scm to derive the version from a git tag, or have __init__.py read it from package metadata at runtime:

from importlib.metadata import version
__version__ = version("quickstats")

5. Publishing untested code

Always upload to TestPyPI first, install from there in a clean venv, and run a smoke test. If a real release breaks pip install for users, you can't replace it. You can only yank it.

6. License keyword without a LICENSE file

license = { text = "MIT" } and no LICENSE file in the repo is an incomplete license: it confuses users and causes problems for lawyers. Include the actual file. Use one of the SPDX-identified open-source licenses. MIT, Apache-2.0 and BSD-3-Clause are common, permissive choices.

7. Pinning every dep to exact versions

dependencies = ["numpy==1.24.3", "pandas==2.0.1"]

For a library, this makes your package impossible to install together with any other package that pins different versions. pip cannot find a set of versions that satisfies both. Pin loosely (numpy>=1.24,<3) for libraries. Pin tightly (lockfiles) only for applications, where being able to reproduce the exact environment matters more than flexibility.

🎯 Your Turn: Write pyproject.toml for quickstats

Write a complete pyproject.toml for a fictional quickstats package. Requirements:

• Build backend: hatchling.
• Requires Python 3.10+.
• Runtime dependency: numpy>=1.24.
• Provides a CLI command quickstats that calls quickstats.cli:main.
• A dev extra containing pytest, ruff, and mypy.
• Description, author info, MIT license, readme, and at least three sensible classifiers.

Skeleton:

[build-system]
# TODO 1: hatchling backend

[project]
# TODO 2: name, version, description, readme, license, authors
# TODO 3: requires-python, dependencies, classifiers

[project.optional-dependencies]
# TODO 4: dev extra

[project.scripts]
# TODO 5: the CLI entry point

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "quickstats"
version = "0.1.0"
description = "Tiny statistics helpers for everyday data work."
readme = "README.md"
requires-python = ">=3.10"
license = { file = "LICENSE" }
authors = [
    { name = "Hedy", email = "hedy@example.com" },
]
keywords = ["statistics", "numpy", "cli"]
classifiers = [
    "Development Status :: 4 - Beta",
    "Intended Audience :: Developers",
    "License :: OSI Approved :: MIT License",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Topic :: Scientific/Engineering :: Mathematics",
]
dependencies = [
    "numpy>=1.24",
]

[project.optional-dependencies]
dev = [
    "pytest>=7",
    "ruff",
    "mypy",
]

[project.scripts]
quickstats = "quickstats.cli:main"

[project.urls]
Homepage = "https://github.com/surya/quickstats"
Issues = "https://github.com/surya/quickstats/issues"

[tool.hatch.build.targets.wheel]
packages = ["src/quickstats"]

python -m build
pip install -e ".[dev]"
quickstats 1 2 3 4 5
pytest

What You Learned

• A package is source + metadata + build artifact. You write the first two, and a build backend produces the third.
• Modern packaging is pyproject.toml (PEP 621). setup.py is legacy.
• src/ layout prevents the "works for me, fails on install" bug. Use it from the start.
• hatchling is the recommended build backend. Setuptools, flit, poetry-core are alternatives.
• [project.scripts] exposes CLI commands. Syntax: cmd = "module:callable".
• [project.optional-dependencies] for extras: pip install pkg[dev].
• Pin loosely for libraries, tightly for applications.
• Build with python -m build, install editable with pip install -e ., and publish with twine.
• Always TestPyPI first. Once a version is on real PyPI, you cannot replace it. You can only yank it.
• Use API tokens, not passwords. Use trusted publishing in CI.
• Single source of truth for version via hatch-vcs or importlib.metadata.version.
• Drop a py.typed marker so downstream type checkers see your annotations.

Next: Virtual Environments. Give each project its own separate set of installed packages.