Secrets Management for Python Apps

The examples assume AWS, a CI provider or a local .env file. They don't run in the browser. The commands are shown with the expected output in comments.

The worst mistake is putting secrets in git. Automated scanners read pushes to public repos within minutes. Researchers, and attackers, run bots that watch GitHub's public event stream to collect leaked AWS keys, Stripe tokens and OpenAI API keys. By the time you notice your mistake, someone may already have used the key.

This lesson is about handling production secrets safely: what counts as one, where they belong (and don't), how to load them at runtime, how to rotate them, and what to do when one leaks. It builds on envconfig (the local-dev pattern) and goes well with security-checklist.

1. What Counts as a Secret

If leaking it would cause harm (financial loss, data exposure, account takeover), it's a secret. The categories you'll handle:

• Database passwords: give read/write to your entire dataset
• API keys (Anthropic, Stripe, AWS, GitHub): let attackers run up bills or read your data
• OAuth client secrets: let attackers impersonate your app to providers
• JWT signing secrets: let attackers forge any user's session (link auth-jwt)
• Webhook signing secrets: let attackers send fake webhook events
• Encryption keys: decrypt your stored data
• TLS private keys: let attackers impersonate your domain
• CI/CD tokens: let attackers push code, deploy and read other secrets

Things that aren't secrets, but are often mistaken for them:

• Public API keys (Stripe pk_*, Google Maps client-side keys): safe in client code
• Username / app name / public URLs: not secret
• Schema/structure of your data: not secret on its own

2. The Wrong Ways (and What They Look Like)

# In code: DON'T
STRIPE_KEY = "sk_live_51abc..."

# In a committed config file: DON'T
# config.yaml
database:
  password: "actual_password_here"

# In a Dockerfile: DON'T (it stays in the image layers, visible to anyone with the image)
ENV STRIPE_KEY=sk_live_...

# In a public CI config: DON'T
env:
  STRIPE_KEY: "sk_live_..."

The pattern: secrets and code travel separately. If you can find the secret value with grep in your repo, the secret is compromised.

3. Local Development: .env Files

For your laptop, a gitignored .env file is the standard:

# .env (gitignored)
DATABASE_URL=postgresql://localhost/myapp_dev
STRIPE_KEY=sk_test_localdevvalue
JWT_SECRET=local-dev-only-not-prod

Load it via python-dotenv:

from dotenv import load_dotenv
import os

load_dotenv()  # reads .env into os.environ
db_url = os.environ["DATABASE_URL"]

import os  # noqa: F401
os.environ.setdefault("DATABASE_URL", "postgresql://user:password@localhost:5432/example")

The companion file you DO commit:

# .env.example (committed)
DATABASE_URL=postgresql://localhost/myapp_dev
STRIPE_KEY=sk_test_replace_me
JWT_SECRET=replace_with_a_real_secret

This documents what env vars your app needs without exposing values. New developers run cp .env.example .env and fill in their own.

4. The Essential .gitignore

# Local secrets
.env
.env.local
.env.*.local

# Credentials and keys
*.pem
*.key
*.p12
credentials.json
service-account*.json

# Config files that hold real values
config.local.yaml
config.local.toml
secrets/

Add this to every Python project on day one. Audit existing repos with git ls-files | grep -E '\.(env|pem|key)$' to catch accidents.

5. Pre-Commit Hook: Catch It Before You Push

detect-secrets (Yelp's tool) scans staged changes for high-entropy strings that look like secrets:

pip install detect-secrets pre-commit
detect-secrets scan > .secrets.baseline

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/Yelp/detect-secrets
    rev: v1.5.0
    hooks:
      - id: detect-secrets
        args: ['--baseline', '.secrets.baseline']

pre-commit install

Now every git commit runs the scanner first. False positives go in the baseline, and real secrets stop the commit.

Other tools that do the same job: gitleaks, trufflehog. GitHub also runs its own scanner on public pushes, but by then the secret has already left your machine.

6. Already Committed a Secret? The Recovery Steps

If a secret reaches git history, the secret is compromised. No exceptions. The recovery order matters:

1. Rotate the secret immediately. Generate a new one with the provider. Update everywhere that uses it. Then revoke the old key at the provider so it stops working.
2. Then scrub history with git filter-repo or BFG:
bash pip install git-filter-repo git filter-repo --replace-text <(echo 'OLD_SECRET==>REMOVED')
3. Force-push (coordinate with the team first):
bash git push --force-with-lease origin main
4. Audit provider logs for unauthorised use of the old secret.
5. Anyone who cloned the repo still has the secret in their local history. Tell them to clone it again.

The most common mistake is trying to scrub history before rotating. Once the secret has been on the public internet (even briefly), someone may already have a copy. Rotate first. Cleaning the history is only tidying up.

7. Production Secrets: The Managed Options

In production you don't ship a .env file. You inject secrets from a managed store:

The right answer is usually "whatever your cloud provider offers". Its IAM integration removes a whole type of startup problem: you don't need a secret to fetch your secrets.

Reading from AWS Secrets Manager

import boto3
import json
from functools import lru_cache

@lru_cache(maxsize=None)
def get_secret(name: str) -> dict:
    client = boto3.client("secretsmanager", region_name="us-east-1")
    response = client.get_secret_value(SecretId=name)
    return json.loads(response["SecretString"])

db = get_secret("prod/myapp/database")
db_url = f"postgresql://{db['user']}:{db['password']}@{db['host']}/{db['name']}"

The @lru_cache is important: you don't want to call Secrets Manager on every request. Set a TTL via a custom cache if you want rotation to take effect without a restart (see Section 10).

8. Container Secrets

Three approaches, ranked from best to worst:

Best: inject at runtime from a secrets manager

# AWS ECS task definition (snippet)
secrets:
  - name: DATABASE_URL
    valueFrom: arn:aws:secretsmanager:us-east-1:123456789:secret:prod/myapp/db-AbCdEf

ECS injects the secret as an env var when the container starts. The secret is never stored in the container image.

Acceptable: Kubernetes Secrets with a real backend

kubectl create secret alone stores secrets as base64. Base64 is an encoding, not encryption, so it gives no protection. Pair with:

• AWS EKS + External Secrets Operator → pulls from Secrets Manager


• Google GKE + Workload Identity → pulls from Secret Manager


• Sealed Secrets / SOPS for GitOps workflows

Avoid: secrets built into the image

Anything in ENV STRIPE_KEY=... stays in the image layers forever and is visible to anyone who can docker pull. Even private registries can leak through stolen CI tokens.

9. CI/CD Secrets

GitHub Actions: Repo settings → Secrets and variables → Actions (or Environment-scoped for prod-only secrets):

# .github/workflows/deploy.yml
jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: production
    steps:
      - uses: actions/checkout@v4
      - name: Deploy
        env:
          STRIPE_KEY: ${{ secrets.STRIPE_KEY }}
          AWS_DEPLOY_ROLE: ${{ secrets.AWS_DEPLOY_ROLE_ARN }}
        run: python deploy.py

The modern recommended approach for cloud deploys: OIDC, not long-lived keys. GitHub Actions can request short-lived AWS credentials via OIDC, so you never store an AWS_SECRET_ACCESS_KEY in CI:

- uses: aws-actions/configure-aws-credentials@v4
  with:
    role-to-assume: arn:aws:iam::123456789:role/github-actions-deploy
    aws-region: us-east-1

GitHub authenticates to AWS as itself, and AWS issues a 1-hour credential. No long-lived secret to leak.

10. Rotation: Easy to Skip, Costly to Forget

The principle: every long-lived secret should rotate on a schedule. Recommended cadences:

• High value (root keys, signing keys, prod DB roots): 30–90 days
• Application secrets (API keys, OAuth client secrets): 90–180 days
• On personnel change: rotate everything that person had access to, same day

The zero-downtime rotation pattern:

1. Generate new secret alongside the old one (both valid)
2. Update apps to read the new secret (rolling deploy)
3. Wait one full deploy cycle to confirm
4. Revoke the old secret

Most managed stores support this directly, for example Secrets Manager rotation Lambdas and Vault dynamic credentials. Use them.

For app-level cache invalidation when a secret rotates without a restart:

import time, threading

class CachedSecret:
    def __init__(self, name, ttl_seconds=300):
        self._name = name
        self._ttl = ttl_seconds
        self._value = None
        self._fetched_at = 0
        self._lock = threading.Lock()

    def get(self) -> dict:
        # Stop many threads fetching at once. Only one fetch runs at a time.
        if time.time() - self._fetched_at > self._ttl:
            with self._lock:
                if time.time() - self._fetched_at > self._ttl:  # check again inside the lock
                    self._value = _fetch_from_aws(self._name)
                    self._fetched_at = time.time()
        return self._value

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

def _fetch_from_aws(*_a, **_kw):
    print('-> _fetch_from_aws() called')
    return _AutoMock('_fetch_from_aws()')

11. Logging Safely

Many secrets leak through logs:

# DON'T
logger.info(f"Config loaded: {config}")           # dumps the whole dict, secrets and all
logger.error(f"Auth failed for token {token}")    # logs the token

# DO
logger.info("Config loaded", extra={"keys": list(config.keys())})
logger.error("Auth failed", extra={"token_prefix": token[:4] + "***"})

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

logger = _AutoMock('logger')
config = _AutoMock('config')
token = ["alpha", "beta", "gamma"]

For structured logging, configure redaction at the formatter level. structlog has built-in redactors, and python-json-logger accepts a filter callback. The principle: assume every log line is sent to Datadog, Splunk or a similar service, and read by anyone with dashboard access.

Mask helper:

def mask_secret(s: str, show: int = 4) -> str:
    if not s or len(s) <= show:
        return "***"
    return f"{s[:show]}***"

logger.info("Stripe key loaded", extra={"key": mask_secret(stripe_key)})
# → Stripe key loaded {"key": "sk_l***"}

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

logger = _AutoMock('logger')
stripe_key = _AutoMock('stripe_key')

12. Encryption at Rest (for the Data Itself)

A secrets manager protects the keys. For sensitive data (PII, payment details, health records) you also need to encrypt the data with those keys.

cryptography library, Fernet for symmetric encryption:

from cryptography.fernet import Fernet

# The key comes from a secrets manager, NEVER from code
key = get_secret("prod/myapp/data-key")["fernet_key"].encode()
f = Fernet(key)

token = f.encrypt(b"user's social security number")
plain = f.decrypt(token)

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

def get_secret(*_a, **_kw):
    print('-> get_secret() called')
    return _AutoMock('get_secret()')

Store ciphertext in your DB, keep the key in the secrets manager, and rotate the key with re-encryption (or use AWS KMS envelope encryption for big datasets).

13. Audit and Least Privilege

• Every secret should have a named owner: the person responsible for rotating it and for acting when it leaks.
• IAM policies: grant access to the specific secret your app needs, not secretsmanager:*.
• Audit logs: enable CloudTrail / Vault audit log, and review who's reading what.
• On offboarding: rotate every secret the departing person had access to. Do it the same day, not "later".
• Periodic dependency audit: pip install pip-audit && pip-audit. Vulnerable libraries can leak secrets through bugs you don't control.

Common Mistakes

• Scrubbing git history before rotating the secret. Order matters: rotate first, scrub second. Scrubbing history is only tidying up. Rotation is the real fix.
• .env not gitignored. Audit any project on day one with git check-ignore .env. If it doesn't return .env, you have a problem.
• Logging the whole config dict for "debugging". Use a mask_secret() helper and redact at the formatter level.
• Reusing the same secret across environments. Prod and staging should have different DB passwords, JWT signing keys, OAuth secrets. Otherwise a staging leak also exposes prod.
• Storing the secrets-manager auth token in the same place as your secrets. IAM roles and workload identity solve this chicken-and-egg problem: the infrastructure authenticates, not a secret.
• Assuming "private repo" = secret-safe. Hacked developer laptops, contractor access and GitHub data breaches can all expose private repos. The real defence is to never put the secret in the repo.
• Long-lived CI tokens with no expiry. Use OIDC where supported, and rotate the rest quarterly.
• pip install from typos. Typosquatting attacks publish malicious packages with near-correct names (requestes vs requests). They steal env vars on import. Mitigate with pip-audit, pinning, and lockfiles.

🎯 Your Turn: Environment-Aware Secrets Loader

Build a Secrets class that:

• Reads from AWS Secrets Manager when APP_ENV=production (or staging)


• Reads from .env locally otherwise


• Caches results with a configurable TTL (so rotated secrets are picked up without a restart, but you don't call the API too often)


• Exposes get(key, default=None) for a single field


• Masks values in __repr__ so logging the object doesn't leak

import os
import time
from typing import Any

class Secrets:
    """Environment-aware secrets loader."""

    def __init__(self, secret_name: str, ttl_seconds: int = 300):
        # TODO 1: store the secret name and ttl, and set up the cache fields
        ...

    def get(self, key: str, default: Any = None) -> Any:
        # TODO 2: refresh the cache if expired, return the requested key
        ...

    def _load(self) -> dict:
        # TODO 3: switch on os.environ["APP_ENV"]
        #   - "production" or "staging": fetch from boto3 secretsmanager
        #   - anything else: read from os.environ (loaded from .env by dotenv)
        ...

    def __repr__(self) -> str:
        # TODO 4: return "<Secrets name=... loaded=... keys=[...]>"
        # MUST NOT include values
        ...


# Demo (with a fake boto3. In real use, boto3 calls AWS):
os.environ["APP_ENV"] = "development"
os.environ["DATABASE_URL"] = "postgresql://localhost/dev"
os.environ["STRIPE_KEY"] = "sk_test_abc"

s = Secrets("prod/myapp/main")
print(s.get("DATABASE_URL"))   # → postgresql://localhost/dev
print(repr(s))                 # → <Secrets name=prod/myapp/main loaded=2026-... keys=[...every environment variable, 'DATABASE_URL' and 'STRIPE_KEY' among them]>

import os  # noqa: F401
os.environ.setdefault("APP_ENV", "example-app-env")

import os
import json
import time
from datetime import datetime
from typing import Any


class Secrets:
    """Environment-aware secrets loader with TTL cache and safe repr."""

    def __init__(self, secret_name: str, ttl_seconds: int = 300):
        self._name = secret_name
        self._ttl = ttl_seconds
        self._value: dict | None = None
        self._fetched_at: float = 0.0

    def get(self, key: str, default: Any = None) -> Any:
        if self._value is None or (time.time() - self._fetched_at) > self._ttl:
            self._value = self._load()
            self._fetched_at = time.time()
        return self._value.get(key, default)

    def _load(self) -> dict:
        env = os.environ.get("APP_ENV", "development")
        if env in ("production", "staging"):
            # Production / staging: fetch from AWS Secrets Manager.
            import boto3  # imported only here, so dev machines don't need boto3
            client = boto3.client("secretsmanager")
            response = client.get_secret_value(SecretId=self._name)
            return json.loads(response["SecretString"])
        # Dev or test: read the process environment (the app loads .env with dotenv).
        return dict(os.environ)

    def __repr__(self) -> str:
        loaded = (
            datetime.fromtimestamp(self._fetched_at).isoformat(timespec="seconds")
            if self._fetched_at else "never"
        )
        keys = list(self._value.keys()) if self._value else []
        # Critical: never include values in repr, because many people read the logs.
        return f"<Secrets name={self._name} loaded={loaded} keys={keys}>"

What You Learned

• The eight categories of real secrets, and what looks like a secret but isn't.
• Local dev uses .env + python-dotenv, and production uses a managed secrets store with IAM-based auth.
• Three layers of defence against committed secrets: .gitignore, pre-commit hooks (detect-secrets), and recovery steps if one gets through.
• Recovery order: rotate first, scrub history second. The secret is already compromised the moment it hits git.
• OIDC eliminates the "long-lived credential in CI" problem for cloud deploys.
• Rotation is a regular task, not a one-off, and managed stores support zero-downtime patterns natively.
• Logs leak secrets without anyone noticing. Mask them with a mask_secret() helper and configure formatter-level redaction.
• Encryption at rest needs a key, and that key belongs in the secrets manager, not next to the data.

Next: this is the last lesson in the DevOps & Deploy track. From here, head back to the Web Frameworks track to wire the auth + secrets patterns into a real Flask or FastAPI app, or move on to Auth & Security for the application-layer side of the same problem.