Virtual Environments

Every Python project should have its own virtual environment. When you pip install packages globally, you start collecting version conflicts between projects, and they can take a long time to sort out.

Here is a concrete example. Project A needs requests==2.31. Project B is an older codebase that still depends on requests==2.20. Your system Python can only have one of these versions installed. A virtual environment (venv) is a self-contained Python installation that lives inside a project folder, with its own python, its own pip, and its own site-packages. Each project gets one, and they are completely separate.

Every command here is typed in a terminal. If that's new, The Terminal for Python Developers comes first. This lesson covers venv, the modern uv replacement, pipx for CLI tools, lockfiles, and the conventions that keep the packages on your computer organised.

1. venv: The Standard Library Tool

Every Python 3 install includes venv. You don't need any third-party tool.

python -m venv .venv

That creates a .venv/ directory in the current folder containing a Python binary, a fresh pip, and an empty site-packages. The convention is to name the directory .venv (leading dot, hidden in ls) so it's easy to gitignore and easy to spot.

2. Activate, Use, Deactivate

The activation script is different for each shell.

macOS / Linux:

source .venv/bin/activate

Windows PowerShell:

.venv\Scripts\Activate.ps1

Windows cmd:

.venv\Scripts\activate.bat

Your prompt changes to show the active venv, usually (.venv) $. Now python and pip point at the venv's copies, not the system ones:

which python                        # /path/to/project/.venv/bin/python   (macOS/Linux)
where python                        # ...\project\.venv\Scripts\python.exe (Windows)
python -V                           # the Python version inside the venv
pip list                            # only what you've installed here

Install packages. They go into .venv/lib/.../site-packages and nowhere else:

pip install requests pytest

Deactivate when you're done:

deactivate

The venv directory still exists, because activation is just a PATH change. Re-activate whenever you come back to the project.

3. The Most Important Rule

.venv/ must always be in .gitignore.

# .gitignore
.venv/
venv/
env/

A venv contains absolute paths written into its scripts, platform-specific binaries, and often many downloaded packages. None of it works on another machine. Committing it makes your repo much bigger, breaks for every other developer (different paths, different OS), and gives no benefit, because the venv can be recreated from your dependency file in seconds.

4. Recording and Restoring Dependencies

You don't commit a venv. You recreate it from a list of packages. There are two ways:

The classic way: requirements.txt

pip freeze > requirements.txt

# requirements.txt
certifi==2024.2.2
charset-normalizer==3.3.2
idna==3.7
requests==2.31.0
urllib3==2.2.1

pip freeze lists every installed package, including the dependencies of your dependencies, with exact versions. To recreate the environment elsewhere:

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

The modern way: pyproject.toml

If your project has a pyproject.toml (see the packaging lesson), declare dependencies there and do an editable install:

pip install -e ".[dev]"

This is the better way for any project that will become a package. requirements.txt is still useful, usually as a lockfile generated from pyproject.toml for reproducible installs.

5. Lockfiles: Reproducible Installs

requirements.txt from pip freeze is a snapshot, not a lockfile. It captures versions but not hashes, and doesn't separate direct dependencies from transitive ones.

For real reproducibility, use a lockfile tool:

A lockfile guarantees that pip install -r requirements.txt (or uv sync) on your colleague's machine, in CI, and in production produces the exact set of packages and versions you tested with. Libraries don't need this, because loose version ranges are better for them. Applications do need it.

6. uv: The Modern, Fast Replacement

uv is a tool written in Rust that does everything pip and venv do, and its authors measure it as 10 to 100 times faster. Many new projects now use it by default.

# install once (Homebrew, pipx, or the official installer script)
brew install uv                     # macOS
pipx install uv                     # cross-platform

# inside a project:
uv venv                             # create .venv (faster than python -m venv)
uv pip install requests             # like pip install, but fast
uv pip install -r requirements.txt
uv sync                             # install + lock from pyproject.toml
uv add requests                     # add a dependency to pyproject AND install it
uv run pytest                       # run a command in the project's venv, no activation needed

uv run is its most useful feature: you don't have to activate the venv at all. It finds the project, uses (or creates) the right venv, and runs your command. For everyday work, this one command replaces the activate, run, deactivate steps.

uv also manages Python versions:

uv python install 3.12              # download a Python interpreter
uv venv --python 3.12               # create a venv using that version

Recommendation: default to uv for new projects. Stick with stock venv + pip if you're contributing to a project that hasn't adopted it.

7. pipx: Global CLI Tools

Some packages are tools, not libraries: black, ruff, httpie, poetry, pre-commit. You want them available everywhere, but you don't want them in your system Python, and installing them in every project venv is wasteful.

pipx solves this. It installs each tool into its own isolated venv and links the entry point onto your PATH.

pipx install black
pipx install ruff
pipx install httpie
pipx install uv

pipx list                           # see what's installed
pipx upgrade-all

You get black on your PATH everywhere, without changing any project's packages or the system Python. This is the right tool for end-user applications, and venv is the right tool for project dependencies. Don't confuse them.

8. conda / mamba: When You Need Non-Python Dependencies

conda (and the faster reimplementation mamba) is a different kind of environment manager. It handles non-Python dependencies too: compiled libraries, CUDA toolkits, R, system tools. Mostly relevant in scientific computing and ML.

conda create -n myenv python=3.12 numpy scipy pytorch
conda activate myenv

For pure-Python projects, conda is more than you need. venv or uv is lighter, faster, and works better with the rest of the Python tools. Use conda when you really need its non-Python package handling (NumPy with Intel MKL, CUDA-linked PyTorch, GDAL with its system libraries).

9. poetry: The All-In-One Tool

poetry bundles env management, dependency resolution, lockfiles, and publishing into a single CLI:

poetry new myproject
poetry add requests
poetry install
poetry run pytest
poetry publish

It has firm rules about how a project should work, many people like it, and it is slower than uv. If you're starting a new project today, uv does the same jobs and runs faster. Poetry is still a reasonable choice if you prefer its workflow, but it is no longer the obvious modern choice.

10. Python Version Management

It is normal to have several Python versions on one machine: Python 3.10 for one client's older code, 3.12 for a new project, and the system Python, which you should never change. Use a version manager:

• pyenv (macOS / Linux): installs and switches Python versions per-project via a .python-version file.
• pyenv-win: the Windows version.
• uv python install: built into uv. Simplest if you already use uv.

pyenv install 3.12.3
pyenv local 3.12.3                  # writes .python-version in the current dir
python -V                           # Python 3.12.3

What you should not use as your project's Python: the system Python that ships with macOS or comes preinstalled on Linux. Those belong to the operating system, and if you upgrade or break them, parts of the OS can break too. Always install your own.

11. The Directory Pattern

A convention that works for any number of projects:

~/code/
    project-a/
        .venv/                      # project-a's venv, gitignored
        pyproject.toml
        src/
    project-b/
        .venv/                      # project-b's venv, gitignored
        pyproject.toml
        src/

Every project gets its own .venv inside the project directory. Never share a venv across projects. Don't store venvs in a central location like ~/venvs/. With the venv inside the project, when you delete the project, the venv goes with it, and when you move the project, you don't have to update tool configs pointing at a separate venv path.

IDE tip: set the Python interpreter to .venv/bin/python (or .venv\Scripts\python.exe on Windows) explicitly. VS Code and PyCharm auto-detect this if the .venv is at the project root.

Common Mistakes

1. No venv at all

You pip install straight into the system Python. Six months later, a different project upgrades a shared dependency, and the original project breaks for no obvious reason. This is the reason venvs exist. Use one from the start: python -m venv .venv takes fifteen seconds.

2. Committing .venv/

You get a huge diff, with absolute paths written into the shebang lines. It won't even work on a colleague's machine, because their Python is at a different path. Add .venv/ to .gitignore immediately.

3. Mixing global and project installs

pip install pandas                  # was the venv active? unclear

Before any pip install, check which python (or where python on Windows). If it doesn't say .venv, you're not in the venv. The venv name in your prompt also tells you: (.venv) $ means you're safe, and no prefix means you're about to install globally.

4. Forgetting to activate before installing

This has the same cause as the previous mistake. It is easy to do after you change directory or open a new terminal. Make which python a habit.

5. requirements.txt without pinned versions

requests
pandas

This works today. If pandas 3.0 comes out two weeks later with breaking changes, your CI fails. Pin to known-good versions:

requests==2.31.0
pandas==2.2.1

For libraries, pin loosely (requests>=2.30,<3). For applications, pin tightly or use a lockfile.

6. Multiple Pythons on PATH, wrong one creates the venv

python -m venv .venv uses whichever python runs first on your PATH. If that's the wrong version, every install after that targets the wrong Python. Be explicit:

python3.12 -m venv .venv
# or with uv:
uv venv --python 3.12

7. venv name collisions in IDEs

PyCharm sometimes auto-creates venv/ while you've made .venv/. Now you have two venvs: the IDE uses one and your terminal uses the other. Pick one name, configure the IDE to use that exact path, delete the duplicate.

🎯 Your Turn: A Setup Script

Write a setup script that creates a fresh development environment with one command. Write it for both shells, so that it works on every platform.

Requirements. Your script should:

1. Create a .venv/ in the current directory using python -m venv .venv.
2. Activate it for the script's session.
3. Upgrade pip to the latest version.
4. Install everything from requirements.txt.
5. Print a final "done, activate with …" hint.

Write setup.sh (bash, for macOS/Linux) and setup.ps1 (PowerShell, for Windows).

Skeleton for setup.sh:

#!/usr/bin/env bash
set -euo pipefail

# TODO 1: create .venv
# TODO 2: activate it
# TODO 3: upgrade pip
# TODO 4: install requirements.txt if it exists
# TODO 5: print a friendly hint

Skeleton for setup.ps1:

$ErrorActionPreference = "Stop"

# TODO 1-5 as above, with PowerShell syntax

#!/usr/bin/env bash
set -euo pipefail

PYTHON="${PYTHON:-python3}"

echo "==> Creating .venv with $PYTHON"
$PYTHON -m venv .venv

# shellcheck disable=SC1091
source .venv/bin/activate

echo "==> Upgrading pip"
python -m pip install --upgrade pip

if [ -f requirements.txt ]; then
    echo "==> Installing requirements.txt"
    pip install -r requirements.txt
elif [ -f pyproject.toml ]; then
    echo "==> Installing project (editable, with dev extras)"
    pip install -e ".[dev]" || pip install -e .
else
    echo "==> No requirements.txt or pyproject.toml found, so nothing to install."
fi

echo ""
echo "Done. Activate the venv with:"
echo "    source .venv/bin/activate"

chmod +x setup.sh
./setup.sh

$ErrorActionPreference = "Stop"

$Python = if ($env:PYTHON) { $env:PYTHON } else { "python" }

Write-Host "==> Creating .venv with $Python"
& $Python -m venv .venv

Write-Host "==> Activating .venv"
. .\.venv\Scripts\Activate.ps1

Write-Host "==> Upgrading pip"
python -m pip install --upgrade pip

if (Test-Path requirements.txt) {
    Write-Host "==> Installing requirements.txt"
    pip install -r requirements.txt
} elseif (Test-Path pyproject.toml) {
    Write-Host "==> Installing project (editable, with dev extras)"
    pip install -e ".[dev]"
    if ($LASTEXITCODE -ne 0) { pip install -e . }   # try/catch can't see a native exit code
} else {
    Write-Host "==> No requirements.txt or pyproject.toml found, so nothing to install."
}

Write-Host ""
Write-Host "Done. Activate the venv with:"
Write-Host "    .\.venv\Scripts\Activate.ps1"

Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned
.\setup.ps1

What You Learned

• A virtual environment is a per-project Python install with its own site-packages. Always use one per project.
• Create with python -m venv .venv. Activate with source .venv/bin/activate (or Activate.ps1 on Windows). Deactivate with deactivate.
• Always gitignore .venv/. Recreate from requirements.txt or pyproject.toml, never commit the directory.
• pip freeze > requirements.txt captures a snapshot. Lockfiles (pip-tools, uv.lock, poetry.lock) give true reproducibility.
• uv is the modern, fast, Rust-based replacement for pip + venv. uv run removes the activation step entirely.
• pipx installs CLI tools (black, ruff, httpie) in their own isolated venvs, so they are available everywhere without affecting other environments.
• conda is for non-Python dependencies (CUDA, MKL, GDAL). Skip it for pure-Python projects.
• Use a Python version manager (pyenv, uv python) rather than the system Python.
• Before every pip install, check which python. If it doesn't show .venv, you're about to install globally.

Next in the How-to path: Git, then command-line tools and configuration. Or go back to the packaging lesson to publish your first project to PyPI.