GitHub Actions for Python

The examples need Docker, a GitHub repository or a deployment target. They don't run in the browser. The commands are shown with the expected output in comments.

If your tests only run when you remember to run them, they will often not run at all. Continuous integration (CI) means a server runs them automatically on every push, every pull request and every tag, and nothing that fails is merged. If your code is already on GitHub, GitHub Actions is the easiest option: no separate server, no separate billing, no separate login, just a YAML file in .github/workflows/.

This lesson covers the shape of a workflow, the patterns you will use again and again (caching, matrices, secrets, service containers), and how to extend a CI pipeline into a CD pipeline that builds and pushes Docker images when the tests pass.

1. The Mental Model

Three terms:

• Workflow: a YAML file in .github/workflows/. Events start it.
• Job: a unit of work that runs on a fresh VM (a "runner"). Jobs in a workflow run in parallel by default.
• Step: a single command or a reusable action. Steps in a job run sequentially.

A workflow is started by an event: push, pull_request, release, schedule (cron) or workflow_dispatch (a manual button). The runner is a clean Ubuntu/Windows/macOS VM. It is created for the run and deleted afterwards. Nothing persists between runs except what you explicitly cache or upload.

The file lives at .github/workflows/ci.yml (the name is arbitrary, and any .yml in that directory becomes a workflow).

2. The Minimum CI for a Python Project

# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.13"

      - run: pip install -r requirements.txt

      - run: pytest

Push this file. Within seconds GitHub starts a runner, clones your repo, installs Python 3.13, installs deps, and runs pytest. The result shows up as a green/red check on the commit and on any associated PR.

Walking through the steps:

• actions/checkout@v4: clones the repo into the runner's filesystem. Required as step 1 in almost every workflow.
• actions/setup-python@v5: installs the requested Python version. Faster than installing from apt.
• run: a shell command. Defaults to bash on Linux/macOS, PowerShell on Windows.

Pin action versions to a major tag (@v4, @v5). Don't pin to @main: the action's maintainer can push a change that breaks your workflow without warning.

3. Caching: The Biggest Speed Improvement

pip install on every run is wasteful when your dependencies haven't changed. actions/setup-python has built-in pip caching:

- uses: actions/setup-python@v5
  with:
    python-version: "3.13"
    cache: "pip"
    cache-dependency-path: requirements.txt

The cache key is the hash of requirements.txt. If you change a dependency, the cache is rebuilt. Otherwise, it is restored in seconds. This can make a CI run several times faster.

For more control, use actions/cache directly:

- uses: actions/cache@v4
  with:
    path: ~/.cache/pip
    key: ${{ runner.os }}-pip-${{ hashFiles('requirements*.txt') }}
    restore-keys: |
      ${{ runner.os }}-pip-

With the restore-keys fallback, you still get an older cache when the exact key is not found. This helps when you've changed one dependency but most are unchanged.

4. Matrix Builds: Test Across Versions

If you're publishing a library, you need to know it works on every Python version you claim to support. A matrix runs the same job N times with different parameters:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.11", "3.12", "3.13"]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
          cache: pip
      - run: pip install -r requirements.txt
      - run: pytest

Three parallel runs, one per Python version. fail-fast: false means one failing version doesn't cancel the others. You want to know whether the code is broken on every version or just one.

Multi-dimensional matrices (OS × Python) are just as easy:

strategy:
  matrix:
    os: [ubuntu-latest, macos-latest, windows-latest]
    python-version: ["3.12", "3.13"]
runs-on: ${{ matrix.os }}

That's six parallel runs. Use them carefully, because runner minutes add up, especially on macOS (10× the cost of Linux on private repos).

5. Lint and Type Check

Tests prove your code works. Lint checks that it is readable and follows your team's style. Type checks catch bugs before tests do. Add both as separate jobs so they run in parallel:

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.13", cache: pip }
      - run: pip install ruff mypy
      - run: ruff check .
      - run: ruff format --check .
      - run: mypy src

  test:
    runs-on: ubuntu-latest
    # ... as before

Two jobs, two parallel runners. The total time is usually under a minute. A failing lint or type check marks the commit as failed, and the PR can't be merged until both pass (if you've configured branch protection). See Section 13.

6. Coverage and Reports

pytest --cov measures which lines your tests really run. Upload the report to codecov.io (or coveralls) for a per-PR coverage diff:

- run: pip install pytest pytest-cov
- run: pytest --cov=src --cov-report=xml --cov-report=term

- uses: codecov/codecov-action@v4
  with:
    files: ./coverage.xml
    fail_ci_if_error: true
  env:
    CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

PRs now get a comment showing coverage changes line-by-line. This helps you find cases where "the test passes, but the new branch is not tested."

7. Service Containers: Tests That Need a Database

Integration tests should use a real Postgres, because a mocked database can behave differently from the real one. Start one as a sidecar:

jobs:
  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_USER: test
          POSTGRES_PASSWORD: test
          POSTGRES_DB: test
        ports: ["5432:5432"]
        options: >-
          --health-cmd="pg_isready -U test"
          --health-interval=5s
          --health-timeout=3s
          --health-retries=5

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.13", cache: pip }
      - run: pip install -r requirements.txt
      - run: pytest
        env:
          DATABASE_URL: postgres://test:test@localhost:5432/test

The options: block uses Docker's healthcheck so the job doesn't start tests until Postgres is ready to accept connections. Without it, the first query in your test suite may run before the database has started, and the tests fail at random.

The same pattern works for Redis, Elasticsearch, RabbitMQ and anything else with a Docker image.

8. Secrets

Anything sensitive (API tokens, deploy keys, signing certificates) goes in repo settings → Secrets and variables → Actions. Reference them via ${{ secrets.NAME }}:

- run: ./deploy.sh
  env:
    AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
    AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

GitHub automatically masks secret values in logs: a secret in echo "$AWS_SECRET_ACCESS_KEY" shows up as ***. But the mask is easy to bypass by accident: if you pipe a secret through base64 and then echo it, the encoded value is not masked.

${{ secrets.GITHUB_TOKEN }} is the built-in one: a short-lived token limited to the current repo, available in every workflow. Use it for anything that talks to GitHub itself (creating releases, commenting on PRs, pushing to ghcr.io).

For cloud deploys, prefer OpenID Connect (OIDC) over long-lived keys. This is covered in secrets.

9. Conditional Steps and Jobs

The same workflow often handles "test on every push" and "deploy on tag." Conditions decide which steps and jobs run:

- name: Deploy to staging
  if: github.event_name == 'push' && github.ref == 'refs/heads/main'
  run: ./scripts/deploy-staging.sh

- name: Publish release
  if: startsWith(github.ref, 'refs/tags/v')
  run: twine upload dist/*

Common predicates:

Whole-job gating uses the same syntax with if: at the job level. Together with needs:, you can build pipelines like "build only after the tests pass".

10. Building and Pushing a Docker Image

CI works very well for Docker: every push gets an image tagged with the commit SHA, available for staging/prod deploys:

build:
  needs: test
  runs-on: ubuntu-latest
  if: github.event_name == 'push' && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v'))
  permissions:
    contents: read
    packages: write                              # lets the job push images to ghcr.io
  steps:
    - uses: actions/checkout@v4

    - uses: docker/setup-buildx-action@v3

    - uses: docker/login-action@v3
      with:
        registry: ghcr.io
        username: ${{ github.actor }}
        password: ${{ secrets.GITHUB_TOKEN }}

    - uses: docker/metadata-action@v5
      id: meta
      with:
        images: ghcr.io/${{ github.repository }}
        tags: |
          type=sha,prefix=git-
          type=ref,event=branch
          type=semver,pattern={{version}}

    - uses: docker/build-push-action@v5
      with:
        context: .
        push: true
        tags: ${{ steps.meta.outputs.tags }}
        labels: ${{ steps.meta.outputs.labels }}
        cache-from: type=gha
        cache-to: type=gha,mode=max

What this gives you:

• docker/metadata-action produces a sensible set of tags automatically: git-abc1234 and main on a push to main, and 1.2.3 when you push a v1.2.3 tag. The tag build only happens if the workflow also runs on tags: add tags: ["v*"] under on: push: next to branches: [main].
• cache-from: type=gha / cache-to: type=gha: uses the GitHub Actions cache to store Docker layers. First build is slow, but subsequent builds reuse cached layers like local Docker would.
• needs: test: the build only runs after the test job passes. If the tests fail, no image is built.

11. Reusable Workflows

Once you have CI on five projects, you'll notice you are copying the same YAML. Extract the common bits as a reusable workflow:

# .github/workflows/python-test.yml
on:
  workflow_call:
    inputs:
      python-version:
        type: string
        default: "3.13"

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ inputs.python-version }}
          cache: pip
      - run: pip install -r requirements.txt
      - run: pytest

Call it from any other workflow in any repo (subject to access):

jobs:
  ci:
    uses: surya/shared-workflows/.github/workflows/python-test.yml@v1
    with:
      python-version: "3.12"

This is how you keep CI for 20 microservices manageable.

12. Concurrency Control

Push five commits in two minutes, and by default you get five parallel CI runs, and most of them are already out of date by the time the last one starts. Concurrency groups automatically cancel runs that a newer push has replaced:

concurrency:
  group: ci-${{ github.ref }}
  cancel-in-progress: true

Per ref (branch / tag), only one run can be active. A new push cancels the run that is still in progress. On busy branches, this can greatly reduce your CI bill.

For production deploys, you almost always want the opposite. Don't cancel a deploy halfway:

concurrency:
  group: deploy-prod
  cancel-in-progress: false                       # wait in a queue, do not cancel

13. Branch Protection: Make CI Mandatory

A CI pipeline is useless if people can merge without waiting for it. Configure branch protection on main (repo Settings → Branches):

• Require status checks to pass before merging: name the jobs (test, lint) explicitly.
• Require branches to be up to date before merging: this forces a rebase or merge, so a CI pass on old code doesn't count.
• Require linear history if you want a clean log.
• Restrict who can push directly to main: changes go through PRs only.

With these settings, code can only reach main through a PR that passes CI. CI is no longer optional.

14. Local CI: Don't Push to Debug

The first time a workflow fails on the runner with a "syntax error in YAML" message after you have already waited ten minutes, you'll wish you had checked it locally. act runs GitHub Actions workflows on your laptop using Docker:

brew install act                                  # or use the official installer on Linux/Windows
act -j test                                       # run the "test" job locally

It is not a perfect copy (some actions don't work outside the real runner), but it finds most typos before they cost you a CI run.

Common Mistakes

1. No caching. CI runs take ten minutes when they could take one. Add cache: pip to setup-python. It is one line of YAML and can save minutes on every run.

2. Secrets echoed to logs. run: echo "Key is $MY_SECRET". GitHub does mask known secret values, but expanding a secret through manipulation (base64, JSON-stringify) can bypass the mask. Treat any log output as public.

3. Running on every push to every branch. A push: trigger with no branches: filter runs the full CI on git push my-experiment. Restrict the heavy jobs to branches: [main] and PR triggers. Let feature branches just rely on the PR run.

4. Pinning actions to @main. If the action's maintainer pushes a breaking change, your CI breaks without warning. Pin to @v4 (or a specific SHA for total safety).

5. Ignoring lint errors. "We'll fix the warnings later" usually means they are never fixed. Either fix the warnings or turn those rules off. Never let lint failures be optional.

6. Forgetting fail-fast: false in matrix builds. If one Python version fails, the others are cancelled, and you don't know whether the bug affects one version or all of them. The default is almost always wrong for testing.

7. Building Docker images without cache. Every push rebuilds the image from scratch. cache-from: type=gha / cache-to: type=gha,mode=max solves it in two lines.

8. Long-lived AWS keys in repo secrets. If a long-lived key leaks, it can be misused until someone notices and revokes it. Use OIDC instead: it gives each run a short-lived token.

🎯 Your Turn: A Complete CI Pipeline

Write .github/workflows/ci.yml for a Python project with these requirements:

1. Trigger on push to main and on every pull request.
2. Lint job: ruff check + ruff format check + mypy on src/.
3. Test job: matrix over Python 3.11, 3.12, 3.13. Use pip caching. Run pytest --cov=src --cov-report=xml. Upload coverage to codecov on the 3.13 run only.
4. Build job: only on pushes to main and only after lint and test pass. Build a Docker image and push to ghcr.io/${{ github.repository }} tagged with the commit SHA and latest.
5. Concurrency group keyed on github.ref, cancel in progress.

Skeleton:

name: CI
on:
  # TODO 1: triggers

concurrency:
  # TODO 5: concurrency block

jobs:
  lint:
    # TODO 2

  test:
    # TODO 3

  build:
    needs: [lint, test]
    # TODO 4

# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main]
  pull_request:

concurrency:
  group: ci-${{ github.ref }}
  cancel-in-progress: true

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.13"
          cache: pip
          cache-dependency-path: requirements.txt

      - name: Install lint tools
        run: pip install ruff mypy -r requirements.txt

      - run: ruff check .
      - run: ruff format --check .
      - run: mypy src

  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.11", "3.12", "3.13"]
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
          cache: pip
          cache-dependency-path: requirements.txt

      - run: pip install -r requirements.txt pytest pytest-cov

      - run: pytest --cov=src --cov-report=xml --cov-report=term

      - name: Upload coverage
        if: matrix.python-version == '3.13'
        uses: codecov/codecov-action@v4
        with:
          files: ./coverage.xml
          fail_ci_if_error: false                # do not fail CI when Codecov has a random error
        env:
          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

  build:
    needs: [lint, test]
    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write
    steps:
      - uses: actions/checkout@v4

      - uses: docker/setup-buildx-action@v3

      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: |
            ghcr.io/${{ github.repository }}:${{ github.sha }}
            ghcr.io/${{ github.repository }}:latest
          cache-from: type=gha
          cache-to: type=gha,mode=max
          labels: |
            org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
            org.opencontainers.image.revision=${{ github.sha }}

What You Learned

• A workflow is YAML in .github/workflows/, jobs run in parallel on fresh runners, and steps are sequential commands or actions.
• Pin actions to a major tag (@v4), never @main.
• Caching (cache: pip on setup-python) is the biggest speed improvement. It can make runs several times faster.
• Matrix builds test across Python versions and OSes. Always set fail-fast: false.
• Service containers (Postgres, Redis) let integration tests run against real dependencies. Use Docker healthchecks so tests don't start before the service is ready.
• Secrets live in repo settings and are auto-masked in logs. GITHUB_TOKEN is the built-in one, but for cloud deploys prefer OIDC.
• Conditional jobs with if: separate test-on-PR from deploy-on-tag.
• Build and push Docker images with docker/build-push-action plus cache-from/to: type=gha for layer caching.
• Reusable workflows (uses: ./.github/workflows/test.yml) remove copied YAML across projects.
• Concurrency groups automatically cancel older runs, which reduces your CI bill on busy branches.
• Branch protection makes CI mandatory: a PR must pass before it can be merged.

Next: Environment Management: how the same Docker image deploys to dev, staging and production with different config.