GitHub Actions for Python

Examples require Docker / a GitHub repo / a deployment target. Not browser-runnable. Commands shown with the expected output in comments.

If your tests only run when you remember to run them, your tests don't run. Continuous integration is the discipline of letting a robot run them for you — on every push, on every pull request, on every tag — and refusing to merge anything that fails. GitHub Actions is the path of least resistance if your code already lives on GitHub: no separate server, no separate billing, no separate auth, just a YAML file in .github/workflows/.

This lesson covers the shape of a workflow, the patterns you'll reuse forever (caching, matrices, secrets, service containers), and how to extend a CI pipeline into a CD pipeline that builds and pushes Docker images on green.

1. The Mental Model

Three nouns:

• Workflow — a YAML file in .github/workflows/. Triggered by events.
• Job — a unit of work that runs on a fresh VM (a "runner"). Jobs in a workflow run in parallel by default.
• Step — a single command or a reusable action. Steps in a job run sequentially.

A workflow is triggered by an event — push, pull_request, release, schedule (cron), workflow_dispatch (manual button). The runner is a clean Ubuntu/Windows/macOS VM, spun up for the run, destroyed after. Nothing persists between runs except what you explicitly cache or upload.

The file lives at .github/workflows/ci.yml (the name is arbitrary; any .yml in that directory becomes a workflow).

2. The Minimum CI for a Python Project

# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.13"

      - run: pip install -r requirements.txt

      - run: pytest

Push this file. Within seconds GitHub spawns a runner, clones your repo, installs Python 3.13, installs deps, and runs pytest. The result shows up as a green/red check on the commit and on any associated PR.

Walking through the steps:

• actions/checkout@v4 — clones the repo into the runner's filesystem. Required as step 1 in almost every workflow.
• actions/setup-python@v5 — installs the requested Python version. Faster than installing from apt.
• run — shell command. Defaults to bash on Linux/macOS, PowerShell on Windows.

Pin action versions to a major tag (@v4, @v5). Don't pin to @main — the action's maintainer can push a change that breaks you without warning.

3. Caching — The Single Biggest Speed Win

pip install on every run is wasteful when your dependencies haven't changed. actions/setup-python has built-in pip caching:

- uses: actions/setup-python@v5
  with:
    python-version: "3.13"
    cache: "pip"
    cache-dependency-path: requirements.txt

The cache is keyed on the hash of requirements.txt — change a dep, the cache invalidates and rebuilds; otherwise it's restored in seconds. Typical CI run drops from 4 minutes to 45 seconds.

For more control, use actions/cache directly:

- uses: actions/cache@v4
  with:
    path: ~/.cache/pip
    key: ${{ runner.os }}-pip-${{ hashFiles('requirements*.txt') }}
    restore-keys: |
      ${{ runner.os }}-pip-

The restore-keys fallback means a partial-miss still gets something — useful when you've changed one dep but most are unchanged.

4. Matrix Builds — Test Across Versions

If you're publishing a library, you need to know it works on every Python version you claim to support. A matrix runs the same job N times with different parameters:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.11", "3.12", "3.13"]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
          cache: pip
      - run: pip install -r requirements.txt
      - run: pytest

Three parallel runs, one per Python version. fail-fast: false means one failing version doesn't cancel the others — you want to know if it's broken on every version or just one.

Multi-dimensional matrices (OS × Python) are equally easy:

strategy:
  matrix:
    os: [ubuntu-latest, macos-latest, windows-latest]
    python-version: ["3.12", "3.13"]
runs-on: ${{ matrix.os }}

That's six parallel runs. Use sparingly — runner minutes add up, especially on macOS (10× the cost of Linux on private repos).

5. Lint and Type Check

Tests prove your code works. Lint proves it's readable and your team agrees on style. Type checks catch bugs before tests do. Add both as separate jobs so they run in parallel:

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.13", cache: pip }
      - run: pip install ruff mypy
      - run: ruff check .
      - run: ruff format --check .
      - run: mypy src

  test:
    runs-on: ubuntu-latest
    # ... as before

Two jobs, two parallel runners. Total wall time stays under a minute. A failing lint or type check turns the commit red — the PR can't merge until both pass (if you've configured branch protection; see Section 13).

6. Coverage and Reports

pytest --cov measures which lines your tests actually exercise. Upload the report to codecov.io (or coveralls) for a per-PR coverage diff:

- run: pip install pytest pytest-cov
- run: pytest --cov=src --cov-report=xml --cov-report=term

- uses: codecov/codecov-action@v4
  with:
    files: ./coverage.xml
    fail_ci_if_error: true
  env:
    CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

PRs now get a comment showing coverage changes line-by-line. Useful for catching "the test passes but the new branch isn't actually tested."

7. Service Containers — Tests That Need a Database

For tests that need a real Postgres (which integration tests should — mocked DBs lie), spin one up as a sidecar:

jobs:
  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_USER: test
          POSTGRES_PASSWORD: test
          POSTGRES_DB: test
        ports: ["5432:5432"]
        options: >-
          --health-cmd="pg_isready -U test"
          --health-interval=5s
          --health-timeout=3s
          --health-retries=5

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.13", cache: pip }
      - run: pip install -r requirements.txt
      - run: pytest
        env:
          DATABASE_URL: postgres://test:test@localhost:5432/test

The options: block uses Docker's healthcheck so the job doesn't start tests until Postgres is ready to accept connections. Without it, the first query in your test suite races the database boot and fails 30% of the time.

The same pattern works for Redis, Elasticsearch, RabbitMQ — anything with a Docker image.

8. Secrets

Anything sensitive — API tokens, deploy keys, signing certs — lives in repo settings → Secrets and variables → Actions. Reference them via ${{ secrets.NAME }}:

- run: ./deploy.sh
  env:
    AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
    AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

GitHub automatically masks secret values in logs — a secret in echo "$AWS_SECRET_ACCESS_KEY" shows up as ***. Don't try to outsmart that: piping a secret through base64 then echo-ing it bypasses the mask.

${{ secrets.GITHUB_TOKEN }} is the built-in one — a short-lived token scoped to the current repo, available in every workflow. Use it for anything that talks to GitHub itself (creating releases, commenting on PRs, pushing to ghcr.io).

For cloud deploys, prefer OpenID Connect (OIDC) over long-lived keys — covered in secrets.

9. Conditional Steps and Jobs

The same workflow often handles "test on every push" and "deploy on tag." Conditionals gate steps and jobs:

- name: Deploy to staging
  if: github.event_name == 'push' && github.ref == 'refs/heads/main'
  run: ./scripts/deploy-staging.sh

- name: Publish release
  if: startsWith(github.ref, 'refs/tags/v')
  run: twine upload dist/*

Common predicates:

Whole-job gating uses the same syntax with if: at the job level. Combined with needs:, you can wire "build only after test passes" pipelines.

10. Building and Pushing a Docker Image

CI shines for Docker — every push gets an image with the commit SHA baked in, available for staging/prod deploys:

build:
  needs: test
  runs-on: ubuntu-latest
  if: github.event_name == 'push' && github.ref == 'refs/heads/main'
  permissions:
    contents: read
    packages: write                              # write to ghcr.io
  steps:
    - uses: actions/checkout@v4

    - uses: docker/setup-buildx-action@v3

    - uses: docker/login-action@v3
      with:
        registry: ghcr.io
        username: ${{ github.actor }}
        password: ${{ secrets.GITHUB_TOKEN }}

    - uses: docker/metadata-action@v5
      id: meta
      with:
        images: ghcr.io/${{ github.repository }}
        tags: |
          type=sha,prefix=git-
          type=ref,event=branch
          type=semver,pattern={{version}}

    - uses: docker/build-push-action@v5
      with:
        context: .
        push: true
        tags: ${{ steps.meta.outputs.tags }}
        labels: ${{ steps.meta.outputs.labels }}
        cache-from: type=gha
        cache-to: type=gha,mode=max

What this gives you:

• docker/metadata-action produces a sensible set of tags automatically — git-abc1234, main, plus semver tags on a v1.2.3 push.
• cache-from: type=gha / cache-to: type=gha — GitHub Actions cache backend for Docker layers. First build is slow; subsequent builds reuse cached layers like local Docker would.
• needs: test — the build only runs after the test job is green. No green tests, no image.

11. Reusable Workflows

Once you have CI on five projects, you'll notice you're copy-pasting. Extract the common bits as a reusable workflow:

# .github/workflows/python-test.yml
on:
  workflow_call:
    inputs:
      python-version:
        type: string
        default: "3.13"

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: ${{ inputs.python-version }}, cache: pip }
      - run: pip install -r requirements.txt
      - run: pytest

Call it from any other workflow in any repo (subject to access):

jobs:
  ci:
    uses: surya/shared-workflows/.github/workflows/python-test.yml@v1
    with:
      python-version: "3.12"

This is how you maintain CI for 20 microservices without losing your mind.

12. Concurrency Control

Push five commits in two minutes, and by default you get five parallel CI runs — most of which are obsolete by the time the last one starts. Concurrency groups auto-cancel superseded runs:

concurrency:
  group: ci-${{ github.ref }}
  cancel-in-progress: true

Per ref (branch / tag), only one run can be active. New push? Cancel the in-flight one. Halves your CI bill on busy branches.

For production deploys, you almost always want the opposite — don't cancel mid-deploy:

concurrency:
  group: deploy-prod
  cancel-in-progress: false                       # queue, don't cancel

13. Branch Protection — Make CI Mandatory

A CI pipeline that nobody waits for is decoration. Configure branch protection on main (repo Settings → Branches):

• Require status checks to pass before merging — name the jobs (test, lint) explicitly.
• Require branches to be up to date before merging — forces rebases, prevents stale CI passes.
• Require linear history if you want a clean log.
• Restrict who can push directly to main — PR-only.

With these on, the only way code lands in main is via a green PR. CI is no longer optional.

14. Local CI — Don't Push to Debug

The first time a workflow fails on the runner with a "syntax error in YAML" message after you've already burned ten minutes, you'll wish you'd validated locally. act runs GitHub Actions workflows on your laptop using Docker:

brew install act                                  # or use the official installer on Linux/Windows
act -j test                                       # run the "test" job locally

Not 100% faithful — some actions don't work outside the real runner — but catches 80% of typos before they cost you a CI cycle.

Common Mistakes

1. No caching. Ten-minute CI runs that should take one. Add cache: pip to setup-python — five seconds of YAML, minutes of speedup.

2. Secrets echoed to logs. run: echo "Key is $MY_SECRET". GitHub does mask known secret values, but expanding a secret through manipulation (base64, JSON-stringify) can bypass the mask. Treat any log output as public.

3. Running on every push to every branch. A push: trigger with no branches: filter runs the full CI on git push my-experiment. Restrict the heavy jobs to branches: [main] and PR triggers; let feature branches just rely on the PR run.

4. Pinning actions to @main. The action's maintainer pushes a change, your CI breaks Monday morning. Pin to @v4 (or a specific SHA for total safety).

5. Ignoring lint errors. "We'll fix the warnings later." You won't. Either fix or rule the warnings out — never let lint be advisory.

6. Forgetting fail-fast: false in matrix builds. One Python version fails, the others are cancelled, you don't know if the bug is version-specific or universal. Almost always wrong.

7. Building Docker images without cache. Every push rebuilds from scratch. cache-from: type=gha / cache-to: type=gha,mode=max solves it in two lines.

8. Long-lived AWS keys in repo secrets. Compromised once, abused forever. Use OIDC trusted publishing — short-lived tokens minted per run.

🎯 Your Turn — A Complete CI Pipeline

Write .github/workflows/ci.yml for a Python project with these requirements:

1. Trigger on push to main and on every pull request.
2. Lint job: ruff check + ruff format check + mypy on src/.
3. Test job: matrix over Python 3.11, 3.12, 3.13. Use pip caching. Run pytest --cov=src --cov-report=xml. Upload coverage to codecov on the 3.13 run only.
4. Build job: only on pushes to main and only after lint and test pass. Build a Docker image and push to ghcr.io/${{ github.repository }} tagged with the commit SHA and latest.
5. Concurrency group keyed on github.ref, cancel in progress.

Skeleton:

name: CI
on:
  # TODO 1: triggers

concurrency:
  # TODO 5: concurrency block

jobs:
  lint:
    # TODO 2

  test:
    # TODO 3

  build:
    needs: [lint, test]
    # TODO 4

# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main]
  pull_request:

concurrency:
  group: ci-${{ github.ref }}
  cancel-in-progress: true

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.13"
          cache: pip
          cache-dependency-path: requirements.txt

      - name: Install lint tools
        run: pip install ruff mypy -r requirements.txt

      - run: ruff check .
      - run: ruff format --check .
      - run: mypy src

  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.11", "3.12", "3.13"]
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
          cache: pip
          cache-dependency-path: requirements.txt

      - run: pip install -r requirements.txt pytest pytest-cov

      - run: pytest --cov=src --cov-report=xml --cov-report=term

      - name: Upload coverage
        if: matrix.python-version == '3.13'
        uses: codecov/codecov-action@v4
        with:
          files: ./coverage.xml
          fail_ci_if_error: false                # don't break CI on codecov flakes
        env:
          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

  build:
    needs: [lint, test]
    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write
    steps:
      - uses: actions/checkout@v4

      - uses: docker/setup-buildx-action@v3

      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: |
            ghcr.io/${{ github.repository }}:${{ github.sha }}
            ghcr.io/${{ github.repository }}:latest
          cache-from: type=gha
          cache-to: type=gha,mode=max
          labels: |
            org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
            org.opencontainers.image.revision=${{ github.sha }}

What You Learned

• A workflow is YAML in .github/workflows/; jobs run in parallel on fresh runners; steps are sequential commands or actions.
• Pin actions to a major tag (@v4), never @main.
• Caching (cache: pip on setup-python) is the single biggest speed win — turns 4-minute runs into 45 seconds.
• Matrix builds test across Python versions and OSes; always set fail-fast: false.
• Service containers (Postgres, Redis) let integration tests run against real dependencies — use Docker healthchecks to avoid race conditions.
• Secrets live in repo settings and are auto-masked in logs. GITHUB_TOKEN is the built-in one; for cloud deploys prefer OIDC.
• Conditional jobs with if: separate test-on-PR from deploy-on-tag.
• Build and push Docker images with docker/build-push-action plus cache-from/to: type=gha for layer caching.
• Reusable workflows (uses: ./.github/workflows/test.yml) eliminate copy-paste across projects.
• Concurrency groups auto-cancel superseded runs; halves your CI bill on busy branches.
• Branch protection makes CI mandatory — green required to merge.

Next: Environment Management — how the same Docker image deploys to dev, staging, and production with different config.