Python Environments: venv, pyenv, and Poetry Guide

Managing Python environments is one of the first pain points new developers hit and one of the last things experienced developers get right. Global installs, version conflicts, and broken system Python installations are all symptoms of skipping proper environment management. This guide covers the three tools that solve the problem at different levels.

Why Environment Management Matters

Without isolation, every project shares the same Python interpreter and packages. Install django==4.2 for one project and django==5.0 for another, and one will break. Worse, some Linux distributions depend on the system Python, and modifying it can break your OS.

The solution has three layers:

1. Version management (pyenv): Install and switch between Python 3.10, 3.11, 3.12, etc.
2. Virtual environments (venv): Isolate packages per project.
3. Dependency management (Poetry): Track, resolve, and lock exact package versions.

venv: Built-in Virtual Environments

venv ships with Python 3.3+ and creates lightweight, isolated environments.

Creating and Using a venv

# Create a virtual environment
# python3 -m venv .venv

# Activate it (macOS/Linux)
# source .venv/bin/activate

# Activate it (Windows)
# .venv\Scripts\activate

# Your prompt changes to show the active environment:
# (.venv) $ python --version
# Python 3.12.4

# Install packages into the isolated environment
# pip install requests flask

# See what's installed
# pip list

# Deactivate when done
# deactivate

How venv Works Internally

A venv is just a directory with a copy of (or symlink to) the Python binary and its own site-packages:

# .venv/
# +-- bin/             (or Scripts/ on Windows)
# |   +-- python       (symlink to system Python)
# |   +-- pip
# |   +-- activate
# +-- lib/
# |   +-- python3.12/
# |       +-- site-packages/   <- your packages go here
# +-- pyvenv.cfg       <- points to the base Python

When activated, .venv/bin is prepended to your PATH, so python and pip resolve to the virtual environment copies.

Freezing and Reproducing Dependencies

# Export current packages
# pip freeze > requirements.txt

# Reproduce on another machine
# python3 -m venv .venv
# source .venv/bin/activate
# pip install -r requirements.txt

The problem with pip freeze is that it captures everything, including transitive dependencies. If you installed requests, you also get certifi, charset-normalizer, idna, and urllib3 in your requirements file. This makes upgrades harder. Poetry solves this problem.

venv Best Practices

• Name your environment .venv and add it to .gitignore.
• Create one venv per project, never share across projects.
• Use python3 -m venv instead of virtualenv (the third-party tool). The built-in venv is sufficient for most cases.
• On systems with multiple Python versions, specify the version explicitly: python3.12 -m venv .venv.

pyenv: Python Version Management

pyenv lets you install multiple Python versions and switch between them per-project or globally.

Installing pyenv

# macOS with Homebrew
# brew install pyenv

# Linux (automatic installer)
# curl https://pyenv.run | bash

# Add to your shell profile (~/.bashrc, ~/.zshrc):
# export PYENV_ROOT="$HOME/.pyenv"
# export PATH="$PYENV_ROOT/bin:$PATH"
# eval "$(pyenv init -)"

Managing Python Versions

# List available versions
# pyenv install --list | grep "3.12"

# Install a specific version
# pyenv install 3.12.4

# Install another version
# pyenv install 3.11.9

# See installed versions
# pyenv versions
#   system
#   3.11.9
# * 3.12.4 (set by ~/.pyenv/version)

Setting Python Versions

pyenv uses a hierarchy to determine which Python to use:

# Global default (for your user account)
# pyenv global 3.12.4

# Local override (per directory, creates .python-version file)
# cd my-project/
# pyenv local 3.11.9

# Shell override (current terminal session only)
# pyenv shell 3.12.4

The .python-version file is just a text file containing the version number. Commit it to your repository so every developer uses the same Python version.

# .python-version
# 3.12.4

pyenv with venv

Combine pyenv and venv for version + package isolation:

# Set the Python version for your project
# cd my-project/
# pyenv local 3.12.4

# Verify
# python --version  -> Python 3.12.4

# Create a venv with that version
# python -m venv .venv
# source .venv/bin/activate

# Now you have Python 3.12.4 with isolated packages

Build Dependencies

pyenv compiles Python from source, so you need build dependencies:

# macOS
# brew install openssl readline sqlite3 xz zlib

# Ubuntu/Debian
# sudo apt install -y make build-essential libssl-dev zlib1g-dev \
#   libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm \
#   libncurses5-dev libncursesw5-dev xz-utils tk-dev \
#   libffi-dev liblzma-dev

Without these, pyenv install may succeed but produce a Python without key modules like ssl or sqlite3.

Poetry: Modern Dependency Management

Poetry handles dependency resolution, lock files, virtual environments, and packaging in one tool. It replaces pip, pip-tools, setup.py, and twine.

Installing Poetry

# Official installer (recommended)
# curl -sSL https://install.python-poetry.org | python3 -

# Verify installation
# poetry --version

Creating a New Project

# Create a new project
# poetry new my-project
# cd my-project/

# Or initialize in an existing directory
# cd existing-project/
# poetry init

Poetry creates a pyproject.toml file:

# pyproject.toml
# [tool.poetry]
# name = "my-project"
# version = "0.1.0"
# description = "A sample project"
# authors = ["Your Name <you@example.com>"]
# readme = "README.md"
#
# [tool.poetry.dependencies]
# python = "^3.12"
#
# [tool.poetry.group.dev.dependencies]
# pytest = "^8.0"
#
# [build-system]
# requires = ["poetry-core"]
# build-backend = "poetry.core.masonry.api"

Adding Dependencies

# Add a runtime dependency
# poetry add requests
# poetry add "flask>=3.0,<4.0"

# Add a development dependency
# poetry add --group dev pytest mypy ruff

# Remove a dependency
# poetry remove requests

Each poetry add updates pyproject.toml and regenerates poetry.lock.

The Lock File

poetry.lock pins every package (including transitive dependencies) to exact versions with hashes. This guarantees reproducible installs:

# Install from the lock file (exact versions)
# poetry install

# Update dependencies within version constraints
# poetry update

# Update a single package
# poetry update requests

Always commit poetry.lock to version control. poetry install uses the lock file, so every team member and CI server gets identical packages.

Poetry Virtual Environments

Poetry creates and manages virtual environments automatically:

# Run a command inside the environment
# poetry run python my_script.py
# poetry run pytest

# Activate the environment shell
# poetry shell

# See where the environment lives
# poetry env info

# Use a specific Python version (works with pyenv)
# poetry env use python3.12

Dependency Groups

Poetry organizes dependencies into groups:

# pyproject.toml
# [tool.poetry.dependencies]
# python = "^3.12"
# fastapi = "^0.110"
# sqlalchemy = "^2.0"
#
# [tool.poetry.group.dev.dependencies]
# pytest = "^8.0"
# mypy = "^1.10"
# ruff = "^0.4"
#
# [tool.poetry.group.docs.dependencies]
# sphinx = "^7.0"
#
# [tool.poetry.group.docs]
# optional = true

# Install everything including dev
# poetry install

# Skip optional groups
# poetry install --without docs

# Install only specific groups
# poetry install --only main,dev

Poetry Scripts

Define shortcuts for common commands:

# pyproject.toml
# [tool.poetry.scripts]
# serve = "my_project.main:run_server"
# migrate = "my_project.db:run_migrations"

Then run with poetry run serve.

The Complete Workflow: pyenv + Poetry

Here is how the three tools work together on a real project:

# 1. Install the right Python version
# pyenv install 3.12.4
# cd my-project/
# pyenv local 3.12.4

# 2. Tell Poetry to use it
# poetry env use python3.12

# 3. Install dependencies
# poetry install

# 4. Run your code
# poetry run python -m my_project

# 5. Add a new dependency
# poetry add httpx

# 6. Run tests
# poetry run pytest

# 7. Update all dependencies
# poetry update

Your project directory looks like this:

# my-project/
# +-- .python-version     <- pyenv: "3.12.4"
# +-- pyproject.toml       <- Poetry: dependencies and config
# +-- poetry.lock          <- Poetry: exact pinned versions
# +-- .venv/               <- Poetry-managed virtual environment
# +-- src/
# |   +-- my_project/
# |       +-- __init__.py
# |       +-- main.py
# +-- tests/
#     +-- test_main.py

CI Configuration Example

# .github/workflows/test.yml (conceptual)
# steps:
#   - uses: actions/checkout@v4
#   - uses: actions/setup-python@v5
#     with:
#       python-version: "3.12"
#   - run: pip install poetry
#   - run: poetry install
#   - run: poetry run pytest --cov

Alternatives Worth Knowing

uv is a newer tool written in Rust that aims to replace pip, pip-tools, and virtualenv with much faster performance. It is compatible with requirements.txt and pyproject.toml.

conda is popular in data science. It manages both Python versions and non-Python dependencies (like C libraries). Use it if you work heavily with numpy, scipy, or other scientific packages that need compiled binaries.

pip-tools is a lighter alternative to Poetry. It uses pip-compile to generate pinned requirements from a high-level requirements.in file. Good for projects that do not need Poetry’s full feature set.

Wrapping Up

Python environment management comes down to three layers. Use pyenv to install and switch between Python versions without touching your system Python. Use venv (or let Poetry handle it) to isolate packages per project. Use Poetry to declare dependencies cleanly, lock exact versions, and manage the full lifecycle from install to publish. The pyenv local command plus poetry install gives you a reproducible setup that works the same on every machine. Start with venv if you are learning, adopt Poetry when you collaborate with others, and add pyenv when you need multiple Python versions.