Packaging & Publishing to PyPI

A package is what someone installs when they run pip install yourpkg. Under the hood it's a directory of Python files plus metadata, eventually compressed into a wheel and uploaded to the Python Package Index. The mechanics used to be ugly — setup.py, MANIFEST.in, eggs, three competing build systems. Modern Python has consolidated around a single declarative file: pyproject.toml. This lesson uses the modern path end-to-end.

By the end you'll have a project layout, a working pyproject.toml, a wheel on TestPyPI, and the muscle memory to do it again on the real PyPI tomorrow.

1. The Mental Model

A published Python package is three things stacked:

1. Source code — your modules and subpackages.
2. Metadata — name, version, description, dependencies, entry points, classifiers.
3. A built artifact — usually a .whl (wheel) and a .tar.gz (sdist) sitting in dist/.

You write the first two. A build backend turns them into the third. pip downloads the wheel and unpacks it into the user's site-packages. That's the whole pipeline.

The legacy way used setup.py — a Python script that was the metadata. The modern way uses pyproject.toml — a static TOML file. New projects should never start with setup.py. If you see it in a tutorial dated after 2023, the tutorial is out of date.

2. The Minimum Project Layout

python

quickstats/
    pyproject.toml
    README.md
    LICENSE
    src/
        quickstats/
            __init__.py
            core.py
            cli.py
    tests/
        test_core.py

Two layouts compete: flat (quickstats/ at the top level next to pyproject.toml) and src (src/quickstats/). Use src. Here's why:

With a flat layout, import quickstats works from the project root because the current directory is on sys.path — even if the package isn't installed. Your tests pass locally and fail for everyone who actually pip installs it.
The src/ layout forces you to install the package (pip install -e .) before importing it. Tests then exercise the same import path your users do.

The five-second rule: if your tests/ directory passes pytest without you having installed the package, you have a src-layout bug waiting to happen.

3. A Minimal `pyproject.toml`

toml

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "quickstats"
version = "0.1.0"
description = "Tiny statistics helpers for everyday data work."
readme = "README.md"
requires-python = ">=3.10"
license = { file = "LICENSE" }
authors = [
    { name = "Hedy", email = "hedy@example.com" },
]
dependencies = [
    "numpy>=1.24",
]

[project.optional-dependencies]
dev = ["pytest>=7", "ruff", "mypy"]

[project.scripts]
quickstats = "quickstats.cli:main"

[project.urls]
Homepage = "https://github.com/surya/quickstats"
Issues = "https://github.com/surya/quickstats/issues"

Walking through it:

[build-system] — declares which tool builds your wheel. hatchling is the modern pick: fast, well-maintained, sensible defaults. Alternatives below.
[project] — PEP 621 metadata. name and version are the only required fields; everything else is strongly recommended.
dependencies — runtime requirements. Pin loosely (>=1.24) for libraries; tight pins make your package un-installable alongside others.
[project.optional-dependencies] — extras. Users opt in with pip install quickstats[dev].
[project.scripts] — entry points. Installing quickstats puts a quickstats command on the user's PATH that calls quickstats.cli:main.

4. Build Backends — Pick One

Backend	Notes
hatchling	Modern default. Fast, declarative, no plugins required for the common cases. Recommended.
setuptools	Battle-tested, ubiquitous. Heavier, slower, but works for everything legacy.
flit	Minimalist. Pure-Python packages only.
poetry-core	Tightly coupled to the `poetry` CLI. Pick this only if you're committing to poetry.

If you have no opinion, use hatchling. The rest of this lesson assumes it.

5. The `src/quickstats/init.py`

python

# src/quickstats/__init__.py
"""quickstats — tiny statistics helpers."""

from .core import mean, median, stdev

__version__ = "0.1.0"
__all__ = ["mean", "median", "stdev"]

The __init__.py is the curated public API. Re-export the names you want at the top level so users write from quickstats import mean instead of from quickstats.core import mean. Anything not re-exported is implicitly internal.

A matching cli.py:

python

# src/quickstats/cli.py
import sys
from .core import mean, median, stdev

def main():
    if len(sys.argv) < 2:
        print("Usage: quickstats <numbers...>")
        sys.exit(1)
    values = [float(x) for x in sys.argv[1:]]
    print(f"mean   = {mean(values):.3f}")
    print(f"median = {median(values):.3f}")
    print(f"stdev  = {stdev(values):.3f}")

[project.scripts] in pyproject.toml points at quickstats.cli:main — installing the package puts a quickstats command on the user's PATH.

6. Versioning

SemVer (MAJOR.MINOR.PATCH) is the default convention:

MAJOR — breaking changes. Anything that could break a caller's existing code.
MINOR — new features, backward-compatible.
PATCH — bug fixes only.

Pre-releases: 0.2.0a1 (alpha), 0.2.0b1 (beta), 0.2.0rc1 (release candidate). pip skips pre-releases unless you pass --pre.

Calendar versioning (2026.05.0) is used by some projects (pip itself, Ubuntu). Fine for tools where users care about freshness more than API stability; less suited to libraries.

The version lives in pyproject.toml. Don't also hardcode it in __init__.py unless you keep them in sync — see single source of truth below.

7. Building

bash

pip install build
python -m build

That produces:

python

dist/
    quickstats-0.1.0-py3-none-any.whl
    quickstats-0.1.0.tar.gz

The wheel (.whl) is the install artifact. The sdist (.tar.gz) is the source archive — used as a fallback when no wheel matches the user's platform.

8. Editable Installs — The Local Dev Loop

bash

pip install -e .
pip install -e ".[dev]"             # with dev extras

-e installs the package by linking — your src/quickstats/ directory is the installed package. Edit a file, the change applies on the next python invocation. No rebuild required. This is how you develop a package without spending your life running pip install after every change.

Sanity-check after install:

bash

python -c "import quickstats; print(quickstats.__version__)"
quickstats 1 2 3 4 5                # the CLI entry point

9. Publishing to TestPyPI First

TestPyPI is the staging environment. Every release goes there first.

bash

pip install twine
twine upload --repository testpypi dist/*

You'll need an account on https://test.pypi.org/ and an API token (Account settings → API tokens). Username is literally __token__; the password is the token string (starts with pypi-). Never use password auth — it's deprecated and 2FA-blocked.

Store credentials in ~/.pypirc:

ini

[testpypi]
  username = __token__
  password = pypi-AgEIcHlwaS5vcmcCJ...

Verify the package installs cleanly from TestPyPI in a fresh venv:

bash

python -m venv /tmp/check
source /tmp/check/bin/activate
pip install --index-url https://test.pypi.org/simple/ \
            --extra-index-url https://pypi.org/simple/ \
            quickstats
python -c "import quickstats; print(quickstats.__version__)"

The --extra-index-url is because TestPyPI doesn't mirror your runtime dependencies (numpy, etc.) — pip falls back to real PyPI for those.

10. Publishing to Real PyPI

Once TestPyPI looks clean:

bash

twine upload dist/*

That's it. Within a minute pip install quickstats works for anyone on the planet.

You cannot delete or overwrite a release. PyPI lets you yank a version (hides it from new installs while leaving it installable by anyone who pinned to it), but you can't re-upload the same version with different content. If you broke something, bump the version and ship 0.1.1.

11. Trove Classifiers and Discoverability

Classifiers are how users find your package via the PyPI search filters:

toml

classifiers = [
    "Development Status :: 4 - Beta",
    "Intended Audience :: Developers",
    "License :: OSI Approved :: MIT License",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Topic :: Scientific/Engineering :: Mathematics",
]

The full list is at https://pypi.org/classifiers/. Pick three to five that actually describe your package — don't classifier-stuff.

12. Including Data Files

If your package ships templates, JSON schemas, or any non-Python files, the build backend needs to know to include them. For hatchling:

toml

[tool.hatch.build.targets.wheel]
packages = ["src/quickstats"]

[tool.hatch.build.targets.wheel.force-include]
"src/quickstats/data/schema.json" = "quickstats/data/schema.json"

Most of the time hatchling picks up everything under src/<pkg>/ automatically. Reach for explicit force-include only when files live elsewhere.

Access the file at runtime via importlib.resources — never with __file__ path manipulation, which breaks in zipped wheels.

13. Auto-Versioning From Git Tags

Hard-coding the version in pyproject.toml and again in __init__.py means they drift. The fix is a single source of truth — usually a git tag.

toml

[build-system]
requires = ["hatchling", "hatch-vcs"]
build-backend = "hatchling.build"

[project]
name = "quickstats"
dynamic = ["version"]               # version comes from somewhere else

[tool.hatch.version]
source = "vcs"

[tool.hatch.build.hooks.vcs]
version-file = "src/quickstats/_version.py"

Now git tag v0.2.0 is the version. The build reads the tag, writes _version.py, and the wheel is stamped automatically. setuptools-scm does the same for setuptools projects.

14. Type Hints and the `py.typed` Marker

If your package has type hints and you want downstream users to actually see them, drop an empty py.typed marker file at the top of the package:

python

src/quickstats/
    __init__.py
    py.typed                        # empty file — its presence is the signal
    core.py

Then in pyproject.toml:

toml

[tool.hatch.build.targets.wheel]
packages = ["src/quickstats"]
include = ["src/quickstats/py.typed"]

Without py.typed, tools like mypy assume your package is untyped and silently fall back to Any. That marker is the difference between users getting your types and not.

15. The CI Release Pattern

The grown-up workflow: tag in git → CI builds and publishes. Push a tag, walk away, the release lands.

yaml

# .github/workflows/release.yml
name: release
on:
  push:
    tags: ["v*"]

jobs:
  publish:
    runs-on: ubuntu-latest
    permissions:
      id-token: write               # for trusted-publisher OIDC
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12" }
      - run: pip install build
      - run: python -m build
      - uses: pypa/gh-action-pypi-publish@release/v1

Trusted publishing (OIDC) is the modern way — no API token in CI secrets, GitHub Actions authenticates directly with PyPI. Configure it once in your PyPI project settings.

Common Mistakes

1. setup.py in new code

If you're starting a project in 2026, write pyproject.toml and never touch setup.py. The only setup.py you should write today is the three-line shim some legacy tools still expect — and even that's almost never needed any more.

2. Flat layout instead of src/

python

quickstats/
    quickstats/             # flat — importable from project root without install
        __init__.py

Tests pass for you, fail for users. Move the package into src/quickstats/ and your tests now exercise the installed package, same as everyone else.

3. No README on PyPI

Forget readme = "README.md" in pyproject.toml and your project page is a blank slate. Looks unmaintained. Users assume the worst. Always set readme.

4. Drifting versions

python

# pyproject.toml
version = "0.2.0"

# __init__.py
__version__ = "0.1.9"           # forgotten on the last bump

Two sources of truth, guaranteed to diverge. Use hatch-vcs/setuptools-scm to derive the version from a git tag, or have __init__.py read it from package metadata at runtime:

python

from importlib.metadata import version
__version__ = version("quickstats")

5. Publishing untested code

Always TestPyPI first, install it in a clean venv, run a smoke test. A real release that breaks pip install on the first user is a release you can't take back — only yank.

6. License keyword without a LICENSE file

license = { text = "MIT" } and no LICENSE file in the repo is half a license — confusing to users, awkward for lawyers. Include the actual file. Use one of the SPDX-identified open-source licenses — MIT, Apache-2.0, BSD-3-Clause are the friendly defaults.

7. Pinning every dep to exact versions

toml

dependencies = ["numpy==1.24.3", "pandas==2.0.1"]

For a library, this makes your package un-installable alongside anything else that pins differently — dependency-resolution hell. Pin loosely (numpy>=1.24,<3) for libraries. Pin tightly (lockfiles) only for applications, where reproducibility outweighs flexibility.

🎯 Your Turn — Write `pyproject.toml` for `quickstats`

Write a complete pyproject.toml for a fictional quickstats package. Requirements:

Build backend: hatchling.
Requires Python 3.10+.
Runtime dependency: numpy>=1.24.
Provides a CLI command quickstats that calls quickstats.cli:main.
A dev extra containing pytest, ruff, and mypy.
Description, author info, MIT license, readme, and at least three sensible classifiers.

Skeleton:

toml

[build-system]
# TODO 1: hatchling backend

[project]
# TODO 2: name, version, description, readme, license, authors
# TODO 3: requires-python, dependencies, classifiers

[project.optional-dependencies]
# TODO 4: dev extra

[project.scripts]
# TODO 5: the CLI entry point

Hint 1 — Build system

The build-system table needs both requires (the list of packages needed to build) and build-backend (the dotted module path). For hatchling: requires = ["hatchling"] and build-backend = "hatchling.build".

Hint 2 — Entry point syntax

quickstats = "quickstats.cli:main" — left of the equals is the command name; right is module:callable. The colon is mandatory.

Show full solution

toml

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "quickstats"
version = "0.1.0"
description = "Tiny statistics helpers for everyday data work."
readme = "README.md"
requires-python = ">=3.10"
license = { file = "LICENSE" }
authors = [
    { name = "Hedy", email = "hedy@example.com" },
]
keywords = ["statistics", "numpy", "cli"]
classifiers = [
    "Development Status :: 4 - Beta",
    "Intended Audience :: Developers",
    "License :: OSI Approved :: MIT License",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Topic :: Scientific/Engineering :: Mathematics",
]
dependencies = [
    "numpy>=1.24",
]

[project.optional-dependencies]
dev = [
    "pytest>=7",
    "ruff",
    "mypy",
]

[project.scripts]
quickstats = "quickstats.cli:main"

[project.urls]
Homepage = "https://github.com/surya/quickstats"
Issues = "https://github.com/surya/quickstats/issues"

[tool.hatch.build.targets.wheel]
packages = ["src/quickstats"]

Build and install it:

bash

python -m build
pip install -e ".[dev]"
quickstats 1 2 3 4 5
pytest

This file is the entire configuration for a publishable package. No setup.py, no setup.cfg, no MANIFEST.in — one TOML file, declarative, version-controlled, and clear at a glance about what the package is and what it needs.

What You Learned

A package is source + metadata + build artifact. You write the first two; a build backend produces the third.
Modern packaging is pyproject.toml (PEP 621). setup.py is legacy.
src/ layout prevents the "works for me, fails on install" bug — adopt it from day one.
hatchling is the recommended build backend; setuptools, flit, poetry-core are alternatives.
[project.scripts] exposes CLI commands. Syntax: cmd = "module:callable".
[project.optional-dependencies] for extras — pip install pkg[dev].
Pin loosely for libraries, tightly for applications.
Build with python -m build; install editable with pip install -e .; publish with twine.
Always TestPyPI first. Once a version is on real PyPI, you cannot replace it — only yank.
Use API tokens, not passwords. Use trusted publishing in CI.
Single source of truth for version via hatch-vcs or importlib.metadata.version.
Drop a py.typed marker so downstream type checkers see your annotations.

Next: Virtual Environments — the per-project isolation that keeps your installs sane.

1. The Mental Model

2. The Minimum Project Layout

3. A Minimal pyproject.toml

4. Build Backends — Pick One

5. The src/quickstats/__init__.py

6. Versioning

7. Building

8. Editable Installs — The Local Dev Loop

9. Publishing to TestPyPI First

10. Publishing to Real PyPI

11. Trove Classifiers and Discoverability

12. Including Data Files

13. Auto-Versioning From Git Tags

14. Type Hints and the py.typed Marker

15. The CI Release Pattern

Common Mistakes

🎯 Your Turn — Write pyproject.toml for quickstats

What You Learned

3. A Minimal `pyproject.toml`

5. The `src/quickstats/init.py`

14. Type Hints and the `py.typed` Marker

🎯 Your Turn — Write `pyproject.toml` for `quickstats`