Python Environments
Summary. An isolated Python environment keeps each project’s packages (and often its Python version) separate, so upgrading one project never breaks another and your work stays reproducible. In the lab we use three tools depending on the job: venv for quick, pure-Python projects, conda when a project needs specific Python/CUDA or non-Python dependencies, and Docker when we need a fully reproducible or deployable environment. Never install project packages into the system Python.
Contents
venv
venv ships with Python (3.3+) and creates a lightweight environment in a folder inside your project. Docs: https://docs.python.org/3/library/venv.html
# 1. Create an environment in a .venv folder
python3 -m venv .venv
# 2. Activate it
source .venv/bin/activate # macOS / Linux
.venv\Scripts\Activate.ps1 # Windows (PowerShell)
# 3. Install packages
pip install numpy pandas
pip install -r requirements.txt # from a file
# 4. Save the exact packages so others can reproduce
pip freeze > requirements.txt
# 5. Leave the environment
deactivate
Add .venv/ to your .gitignore — commit requirements.txt, not the environment folder.
conda
Use conda (install Miniconda — small and enough for us: https://www.anaconda.com/download/success) when you need a particular Python version, GPU/CUDA builds, or scientific packages that are painful to pip install. Docs: https://docs.conda.io/projects/conda/en/stable/user-guide/index.html
# Create a named environment with a specific Python version
conda create -n myproj python=3.11
conda activate myproj
# Install packages (prefer the conda-forge channel)
conda install -c conda-forge numpy pandas scikit-learn
# Reproduce from / export to a file
conda env create -f environment.yml # create from a shared file
conda env export --from-history > environment.yml # export your env
conda deactivate
Keep environments per project and share the environment.yml, not the whole env.
Docker
Use Docker when you need the environment itself (OS libraries, system tools, exact versions) to be reproducible or deployable — e.g. servers, GPU jobs, or handing a project to someone else. Get Docker: https://docs.docker.com/get-started/
Create a Dockerfile in the project root:
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["python", "main.py"]
Build and run it:
docker build -t myproject .
docker run --rm -it myproject
# mount your code for live editing:
docker run --rm -it -v "$PWD":/app myproject bash
For GPU work, install the NVIDIA Container Toolkit and add --gpus all to docker run.
Which one should I use?
| Situation | Use |
|---|---|
| Quick, pure-Python project | venv |
| Need a specific Python/CUDA build or heavy scientific stack | conda |
| Must reproduce the whole OS-level environment, deploy, or run on a server/GPU box | Docker |
Whatever you choose: one environment per project, and commit the environment file (requirements.txt / environment.yml / Dockerfile) so anyone can rebuild it.