Environment Management for Python Projects

The examples need Docker, a GitHub repository or a deployment target. They don't run in the browser. The commands are shown with the expected output in comments.

With only one environment, every deploy is a gamble (people call this "deploy and pray"). The same artefact that passed your unit tests yesterday now serves real users. The first time it meets realistic data, traffic or auth providers is in production, and that is when it breaks for customers. A professional setup has at least three environments (dev, staging, prod). The same code runs in all three, and only the config is different.

This lesson is about the overall structure of a multi-environment Python deployment: what changes between environments, what stays the same, how config flows in, how branches and tags map to deploys, and the monitoring setup that tells you which environment a bug came from. It builds directly on envconfig, docker, and github-actions.

1. The Three-Environment Baseline

The minimum setup that works well:

Staging is the environment most teams under-invest in. It copies prod's infrastructure (same database engine, same versions, same auth provider, same secrets manager, same CDN), but with separate data and separate credentials. Its job is to find the problems production would hit, before production hits them. If staging never has bugs, it isn't working as staging. It is just a second production that nobody calls by that name.

The most important property of staging: it must be safe to break. Engineers should feel free to deploy half-finished work, drop tables, restart services. When a team starts treating staging like prod (asking for change windows, requiring approvals), staging stops doing its job.

2. What Varies Between Environments

The list is shorter than you might think:

• Database URL: always a separate DB per environment.
• External API keys: Stripe test keys in dev/staging, live keys in prod. Same for Anthropic, Twilio, AWS.
• OAuth client IDs / redirect URIs: registered per environment.
• Log level: DEBUG locally, INFO in staging, WARNING or INFO in prod (preference varies).
• Feature flags: new features on in staging, off in prod until rollout.
• Rate limits: usually relaxed in staging for load testing, tight in prod.
• Email / SMS providers: sandbox in dev/staging (never spam real users from staging), real in prod.
• CORS allowed origins: http://localhost:3000 in dev, the real domain in prod.
• Sentry DSN / log-aggregator endpoint: separate projects per env, so prod errors don't get lost among dev errors.
• Domain / base URL: affects email links, webhooks, OAuth redirects.

Everything else (your code, your migrations, your container images, your CI pipeline) is identical. Same Docker image, same git SHA, same dependencies.

3. The 12-Factor Principle (Recap)

From envconfig and worth restating: anything that varies between environments goes in environment variables. Code is identical across environments. This is the main reason to combine containers with env-var config: the same image runs anywhere, and what is in os.environ at startup configures it.

The minimum Python pattern:

import os

class Config:
    DATABASE_URL: str = os.environ["DATABASE_URL"]
    DEBUG: bool = os.environ.get("DEBUG", "false").lower() == "true"
    LOG_LEVEL: str = os.environ.get("LOG_LEVEL", "INFO")
    APP_ENV: str = os.environ.get("APP_ENV", "development")

import os  # noqa: F401
os.environ.setdefault("DATABASE_URL", "postgresql://user:password@localhost:5432/example")

For more than about 5 fields, use pydantic-settings. It gives you types, validation and automatic .env support, so you don't need to write int()/bool() conversions yourself:

from pydantic_settings import BaseSettings, SettingsConfigDict

class Settings(BaseSettings):
    model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8")

    app_env: str = "development"
    database_url: str
    log_level: str = "INFO"
    debug: bool = False
    sentry_dsn: str | None = None
    feature_new_search: bool = False

settings = Settings()

Production preference: pydantic-settings. The validation proves its value the first time someone sets PORT=eight-thousand: pydantic refuses to start the app and shows a clear message.

4. The Env-Var Hierarchy

Where does a config value come from? Multiple sources, with a strict precedence:

host / cloud secrets (highest)
       ↓
shell environment (CI runner, your terminal)
       ↓
.env file (local development only, never in production)
       ↓
.env.example (committed template, defaults)
       ↓
code defaults (lowest)

In practice:

• .env.example: committed. Lists every variable with placeholder values. DATABASE_URL=postgres://user:pass@host/db. Documents what config exists.
• .env: gitignored. Local-dev values. Created by copying .env.example and filling in real values.
• Shell env / CI secrets: override .env. Useful for one-off runs (DEBUG=true python app.py) and CI matrix variations.
• Host secrets: production. Injected by the platform (AWS Secrets Manager, Kubernetes secrets, ECS task definitions). Highest precedence: overrides everything else.

The rule: no environment-specific files in git. No .env.production, no config.prod.yaml. The closest is docker-compose.prod.yml, which contains references to env vars (${DATABASE_URL}) but never values.

5. Multi-Environment docker compose

Compose supports file merging. You keep a base file with the universal config and per-environment overrides:

# docker-compose.yml  (the base file, committed)
services:
  app:
    image: ghcr.io/surya/myapp:${IMAGE_TAG:-latest}
    environment:
      APP_ENV: ${APP_ENV}
      DATABASE_URL: ${DATABASE_URL}
      LOG_LEVEL: ${LOG_LEVEL:-INFO}
    ports: ["8000:8000"]

# docker-compose.dev.yml  (committed)
services:
  app:
    build: .                                     # build from the local Dockerfile
    environment:
      APP_ENV: development
      LOG_LEVEL: DEBUG
    volumes:
      - ./src:/app/src                           # code changes show up live

  db:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: dev
    ports: ["5432:5432"]

# docker-compose.prod.yml  (committed, but ONLY variable names, no values)
services:
  app:
    image: ghcr.io/surya/myapp:${IMAGE_TAG}
    environment:
      APP_ENV: production
      LOG_LEVEL: WARNING
    deploy:
      replicas: 3
      resources:
        limits: { cpus: "1", memory: "512M" }

Invoke with the right combination. (One catch: docker compose up can't start replicas: 3 while the base file binds host port 8000:8000, because all three would need that port. In production, drop the host port and put a reverse proxy in front, or deploy with docker stack deploy, whose routing mesh shares the port.)

docker compose -f docker-compose.yml -f docker-compose.dev.yml up
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

Later files override earlier ones. The real values (DB passwords, API keys) come from the shell environment or a secrets manager, never from any of these files.

6. Branching Strategy → Deployment

Map git refs to environments, so each deploy follows a fixed rule:

The rules:

• Never deploy from a branch other than main. If it's not on main, it hasn't been reviewed and CI hasn't fully tested it.
• Production tags are immutable. Push a tag, deploy that exact SHA. Never re-tag.
• Promote, don't rebuild. The Docker image that deployed to staging is the exact same image that goes to prod: same digest, only a new tag. Don't start a new build for prod.

GitHub Actions for the deploy half:

deploy-staging:
  if: github.event_name == 'push' && github.ref == 'refs/heads/main'
  needs: [test, build]
  environment: staging                           # GitHub "environment" with its own secrets
  runs-on: ubuntu-latest
  steps:
    - run: ./scripts/deploy.sh staging ${{ github.sha }}

deploy-production:
  if: startsWith(github.ref, 'refs/tags/v')
  needs: [test, build]
  environment:
    name: production
    url: https://app.example.com
  runs-on: ubuntu-latest
  steps:
    - run: ./scripts/deploy.sh production ${{ github.ref_name }}

The environment: block is how GitHub controls deployments (deployment gating). In repo settings → Environments → "production", you can require manual approval before this job runs. Someone checks the diff and presses the green button, and the deploy continues.

7. Database Migrations Across Environments

Schema changes need a deployment story. The pattern:

1. In CI, on every push: run migrations against a throwaway DB to verify they apply cleanly.
2. In staging deploy: apply migrations automatically before the new app image takes traffic.
3. In production deploy: apply migrations as a separate, gated step. This is often run by hand or by a dedicated migration job, not auto-applied with the deploy.

Why the asymmetry? Schema changes can lock tables, break running queries, or fail mid-way. In staging, that is a small problem. In prod, it means downtime. A separate step lets you:

• Review the SQL the migration will run (alembic upgrade head --sql).
• Schedule it for a low-traffic window.
• Roll back the app to the previous version first if the migration fails.

For zero-downtime: never write a migration that's incompatible with the current production code. The pattern has several steps: add a nullable column, deploy code that writes to it, backfill, deploy code that reads from it, drop the old column in a future deploy. It is slow and careful, but there are no outages.

8. Feature Flags

A feature flag is config that toggles a code path without a redeploy. The minimum:

class Settings(BaseSettings):
    feature_new_search: bool = False
    feature_realtime_updates: bool = False

# in code
if settings.feature_new_search:
    return new_search(query)
else:
    return legacy_search(query)

For more than a few flags, use a real flag service: LaunchDarkly, GrowthBook, Flagsmith, Unleash. These let you turn flags on for specific users, roll out to a percentage of users and run A/B tests, all without redeploying.

The rule: flags are temporary. A flag older than 6 months is usually a TODO that everyone has forgotten. Check your flags every quarter: for each flag whose rollout is complete, remove the flag and keep the "on" code path permanently.

9. Logging Per Environment

The same code logs differently based on env:

Pattern with structlog:

import structlog, logging, sys, os

def configure_logging():
    level = os.environ.get("LOG_LEVEL", "INFO")
    logging.basicConfig(stream=sys.stdout, level=level, format="%(message)s")

    processors = [
        structlog.stdlib.add_log_level,
        structlog.processors.TimeStamper(fmt="iso"),
    ]
    if os.environ.get("APP_ENV") == "development":
        processors.append(structlog.dev.ConsoleRenderer())          # coloured, easy for people to read
    else:
        processors.append(structlog.processors.JSONRenderer())      # one JSON object per line

    structlog.configure(processors=processors)

import os  # noqa: F401
os.environ.setdefault("APP_ENV", "example-app-env")

One JSON object per line is the format that log aggregators handle best. Datadog, Loki, Splunk and CloudWatch all parse it directly.

10. Error Tracking: Sentry, Per Environment

Production errors are different from dev errors. Sentry (or Rollbar, Honeybadger) gives you a per-environment view with stack traces, breadcrumbs, and user context:

import sentry_sdk

sentry_sdk.init(
    dsn=settings.sentry_dsn,
    environment=settings.app_env,                # tags every error with "staging" or "production"
    release=settings.git_sha,                    # links each error to a specific deploy
    traces_sample_rate=0.1 if settings.app_env == "production" else 1.0,
)

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

settings = _AutoMock('settings')

# Earlier blocks on this page that this one builds on, re-run quietly first.
try:
    from pydantic_settings import BaseSettings, SettingsConfigDict
except Exception:
    pass
_pm_earlier = [
    'from pydantic_settings import BaseSettings, SettingsConfigDict\n\nclass Settings(BaseSettings):\n    model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8")\n\n    app_env: str = "development"\n    database_url: str\n    log_level: str = "INFO"\n    debug: bool = False\n    sentry_dsn: str | None = None\n    feature_new_search: bool = False\n\nsettings = Settings()',
]
def _pm_run_earlier(_srcs, _g):
    import sys as _s, io as _io, warnings as _w
    for _src in _srcs:
        _o, _e = _s.stdout, _s.stderr
        _s.stdout = _s.stderr = _io.StringIO()
        try:
            with _w.catch_warnings():
                _w.simplefilter('ignore')
                exec(compile(_src, '<earlier block>', 'exec'), _g)
        except (Exception, SystemExit):
            pass
        finally:
            _s.stdout, _s.stderr = _o, _e
    if 'matplotlib.pyplot' in _s.modules:
        _s.modules['matplotlib.pyplot'].close('all')   # their plots belong to their own blocks
_pm_run_earlier(_pm_earlier, globals())
del _pm_earlier, _pm_run_earlier

The environment tag is the key part. In Sentry's UI you filter environment:production and dev errors disappear. Errors tagged with the git SHA link back to a specific deploy: if an exception started after deploy v1.2.3, you know what to roll back.

Don't set sentry_dsn for dev unless you really want dev errors in the same Sentry project. Usually you don't.

11. Health and Readiness Probes

Two endpoints, both required for a production deployment:

# /health: is the app running?
@app.get("/health")
def health():
    return {"status": "ok"}

# /ready: is the app ready to handle requests?
@app.get("/ready")
def ready():
    try:
        db.execute("SELECT 1")
        cache.ping()
    except Exception as e:
        return {"status": "not_ready", "reason": str(e)}, 503
    return {"status": "ready"}

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

app = _AutoMock('app')
db = _AutoMock('db')
cache = _AutoMock('cache')

• /health: minimal. Returns 200 if the process is alive. Used by Docker/Kubernetes to decide "restart this container?"
• /ready: checks downstream dependencies (DB, cache, external API). Used by load balancers to decide "route traffic here?"

The distinction matters: a container can be alive (process running) but not ready (DB connection still being established at startup). Both checks let the orchestrator do the right thing.

12. Deployment Strategies: Beyond "Push and Restart"

The simplest deploy: stop the old container, start the new one. This causes a brief outage. That is acceptable for hobby projects, but not for paying customers.

Rolling deploy: replace containers one at a time. There is no outage, but new and old versions run at the same time during the rollout. Your code must handle this (mainly: it must be backwards-compatible with the previous version's database schema and message formats).

Blue/green: two identical production environments. "Blue" is live, and you deploy the new version to "green," run quick checks on it, then switch the load balancer to green. To roll back, you switch it back. It doubles your infrastructure cost while the deploy is running.

Canary: deploy to a small % of traffic (1%, then 5%, then 25%), monitor error rates and latency, expand if healthy. This is the safest pattern, but it needs infrastructure that can split traffic (Istio, Linkerd, ALB weighted target groups).

Choose based on what is at risk. For a hobby app, rolling is enough. For paying customers, use blue/green or canary.

13. Observability per Environment

Staging gets observability so you can find issues there before prod. If staging has no monitoring, you can't find problems there early. Production gets it for incident response.

Only production alerts should page people. Send staging alerts to a low-priority channel (or just a dashboard). If you page engineers because staging is down on a Sunday, your team will burn out.

Common Mistakes

1. Sharing a database between staging and prod. Sooner or later, a developer runs a migration against the wrong environment and destroys real customer data. Always use separate databases: different host, different name, different credentials.

2. No staging at all ("deploy and pray"). Production becomes your test environment, and customers become your testers. Fix: set up a second deploy target today. Even a small, cheap Render service can be your staging.

3. Editing prod config by hand. Someone logs into the server, exports an env var and "fixes it." Six months later nobody remembers the fix, the server is replaced, and the fix is lost. Config changes go through the same review/audit path as code: PR → merge → redeploy.

4. Hardcoded URLs for "the API." https://api.example.com in code means staging calls the production API. Either move it to an env var or compute it from APP_ENV.

5. Feature-flag drift. A flag created for a 2-week rollout is still in the code 18 months later. Check flags every quarter, and delete each flag once its rollout is done. Old flags often cause bugs later.

6. Different Docker images per environment. "Staging build" and "production build" with different code paths. The main reason to use containers is that the same artefact runs everywhere. Build once, promote across envs.

7. Auto-applying migrations in prod. A bad migration now causes an outage during the deploy. Keep them separate: the app deploy applies migrations in staging automatically, but in prod they're a separate, gated step.

8. Same Sentry project for all envs. Dev exceptions drown out prod incidents. Either use separate projects or use the environment: tag to filter, and configure Sentry's alert rules to only fire on environment:production.

🎯 Your Turn: A Config Class Hierarchy Per Environment

Design a Config class hierarchy in plain Python (no Pydantic dependency) that:

1. Has a BaseConfig with universal defaults (PORT = 8000, LOG_LEVEL = "INFO").
2. Has three subclasses: DevConfig, StagingConfig, ProductionConfig.
3. Each subclass overrides only what differs (e.g. DevConfig.DEBUG = True, ProductionConfig.LOG_LEVEL = "WARNING").
4. A Config.from_env() classmethod reads APP_ENV and returns an instance of the correct subclass.
5. Required secrets (e.g. DATABASE_URL) are read from os.environ and the app refuses to start in staging/prod if they're missing. In dev, a local default is acceptable.

Skeleton:

import os
from dataclasses import dataclass

@dataclass
class BaseConfig:
    # TODO 1: universal defaults
    ...

    @classmethod
    def from_env(cls):
        # TODO 4: dispatch on APP_ENV
        ...

class DevConfig(BaseConfig): ...
class StagingConfig(BaseConfig): ...
class ProductionConfig(BaseConfig): ...

config = BaseConfig.from_env()

import os
from dataclasses import dataclass, field


@dataclass
class BaseConfig:
    # Universal defaults
    app_env: str = "development"
    port: int = 8000
    log_level: str = "INFO"
    debug: bool = False
    database_url: str = ""

    @classmethod
    def from_env(cls):
        """Pick the right subclass based on APP_ENV."""
        env = os.environ.get("APP_ENV", "development").lower()
        subclass = _REGISTRY.get(env)
        if subclass is None:
            raise SystemExit(
                f"unknown APP_ENV={env!r}; expected one of {list(_REGISTRY)}"
            )
        return subclass()

    def _require(self, name: str) -> str:
        """Read a required environment variable, or stop at once if it is missing."""
        value = os.environ.get(name)
        if not value:
            raise SystemExit(
                f"[{self.app_env}] missing required env var: {name}"
            )
        return value


@dataclass
class DevConfig(BaseConfig):
    app_env: str = "development"
    debug: bool = True
    log_level: str = "DEBUG"
    database_url: str = field(
        default_factory=lambda: os.environ.get(
            "DATABASE_URL", "postgres://localhost/app_dev"
        )
    )


@dataclass
class StagingConfig(BaseConfig):
    app_env: str = "staging"
    debug: bool = False
    log_level: str = "INFO"
    database_url: str = ""

    def __post_init__(self):
        self.database_url = self._require("DATABASE_URL")


@dataclass
class ProductionConfig(BaseConfig):
    app_env: str = "production"
    debug: bool = False
    log_level: str = "WARNING"
    database_url: str = ""

    def __post_init__(self):
        self.database_url = self._require("DATABASE_URL")
        if "localhost" in self.database_url:
            raise SystemExit(
                "production refused to start: DATABASE_URL points at localhost"
            )


_REGISTRY = {
    "development": DevConfig,
    "staging": StagingConfig,
    "production": ProductionConfig,
}


# --- Demo ---
if __name__ == "__main__":
    # Simulate different environments
    os.environ["APP_ENV"] = "staging"
    os.environ["DATABASE_URL"] = "postgres://staging-db.internal/app"

    config = BaseConfig.from_env()
    print(type(config).__name__)             # StagingConfig
    print(config.app_env)                    # staging
    print(config.log_level)                  # INFO
    print(config.debug)                      # False
    print(config.database_url)               # postgres://staging-db.internal/app

What You Learned

• Three environments minimum: dev (laptop), staging (mirrors prod), production. Staging must be safe to break.
• Anything that varies between environments goes in env vars. Code is identical across envs. Same Docker image, same git SHA.
• Env-var hierarchy: code defaults < .env.example < .env < shell env < host secrets. Highest wins.
• docker compose -f base.yml -f prod.yml merges layered compose files: a base file plus overrides per env.
• Branching strategy maps to environments: main → staging (auto), tag v* → production (gated).
• Promote images, don't rebuild. The same image digest goes from staging to prod.
• Migrations: auto-apply in staging, gated in production. Multi-step backwards-compatible schemas for zero-downtime.
• Feature flags for safe rollouts, and audit quarterly to delete dead flags.
• /health vs /ready: liveness vs readiness. Both are required for production deploys.
• Sentry tagged with environment keeps dev/staging/prod errors separated. Only prod alerts page people.
• Logging per env: human-readable in dev, JSON to an aggregator in staging/prod.
• Deploy strategies: rolling for most apps, blue/green or canary for high-stakes.

Next: Secrets Management: where to store your credentials, and how to rotate them when they leak (and sooner or later, one will).