Raspberry Pi

WSL2 Python Development Environment

This guide describes the ultimate, cross-platform development environment. Below, we are going to cover a full Python development environment installation on WSL2. This means you can develop your Python code for both Linux and Windows . Needless to say, if you reproduce the steps below, a good environment can be setup on basically anything Linux - including your Raspberry Pi.

The base of this tutorial environment is Debian. These guidelines will however work on an image of Ubuntu and others with little adjustments (some installations for the WSL Debian image are already present in the Ubuntu for example). We also assume, that you have installed WSL2 and are comfortable using it.

Introduction

So you want the following:

  • develop code safely in virtual environments (pyenv), with usage unified across platforms
  • see how your code runs in any available version of Python
  • test your code on multiple distributions and OS flavors
  • use productivity tools for package development
  • get testing support out of the box
  • create and view existing MkDocs documentation out of the box
  • work with Jupyter and Jupyter goodies on any environment
  • recover from having an unstable development environment by simply re-installing it with a one-click operation

Look no further. The WSL environment described below covers these requirements. Plus some.

Please note, that we have described other development environment approaches elsewhere (for example here), and the recipes still work and serve well. This recipe covers another approach, which we currently use and find very elegant.

Note: After you finish the installation, you may consider extending the repository with a Git server - we have you covered here: WSL - Installing Gitea.

Setup pre-requisites

  • First, obtain an empty Debian installation from Debian - Microsoft Store. Currently the distro in Windows is small, without python or anything. Exactly what we need.
  • Run the initial installation (i.e. launch it for the first time). You will be asked to setup a default username and password.
  • Setup the default login user (WSL logs in as root) for the username you have setup in the previous step.
    • To setup a default login user, create an empty /etc/wsl.conf file within your distro, containing your default login user. The file will contain the section shown below:
[user]
default=your-username
  • Once your distro is restarted, log in with Terminal, and update and upgrade
sudo apt update && sudo apt full-upgrade
  • Let us now install some basic software
sudo apt install bash-completion # current debian distros are really bare
sudo apt install wget curl
sudo apt install git
# installing Midnight Commander is absolutely optional. We do
sudo apt install mc
  • we like to add Visual Studio Code support as well. Just run the following command, which installs the Code server for x64 or similar.
code .

Let's now setup a recent Python 3 installation. It doesn't need to be the latest and greatest, as we will mostly work with virtual environments in pyenv. We will however need it for having a base when installing common Python libraries (such as cookiecutter). If it weren't for this requirement, it would be enough to setup only pyenv, which would then take care of all other required Python versions.

sudo apt install python3 python3-pip
# for pipx to work (see below), we need also venv
sudo apt install python3-venv

Installing pyenv

To manage virtual environments, we have chosen pyenv As we run our WSL Linux environment wherever we go, we don't need to worry about differences on Windows or other systems and we can choose this tool over others. If you are used to a different tooling, it's great - just skip the pyenv part.

  • Install pre-requisites
sudo apt install -y make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev

There are multiple options to install pyenv. We choose to use the provided installer, which will install pyenv along with a few useful plugins:

  • pyenv: The actual pyenv application
  • pyenv-virtualenv: Plugin for pyenv and virtual environments
  • pyenv-update: Plugin for updating pyenv
  • pyenv-doctor: Plugin to verify that pyenv and build dependencies are installed
  • pyenv-which-ext: Plugin to automatically lookup system commands
curl -L https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer | bash

Once the installation is done, you will see a small guide asking you to setup your shell files. For us, all we need to do is to edit the .bashrc file and add the below lines at the end. The .bashrc file is in your home directory.

  • run your favorite text editor,
cd ~
sudo nano .bashrc
  • and insert the following at the end of the .bashrc file:
export PYENV_ROOT="$HOME/.pyenv"
command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"
  • restart WSL and test everything is fine by checking the version of pyenv
pyenv -v

Before going further, we recommend to use pyenv to install your preferred Python versions, and to setup a globa Python version. The below commands are an example, modify as you see fit.

# ---------------------------
# EXAMPLE pyenv setup EXAMPLE
# ---------------------------

# get the last few Python versions
pyenv install 3.11.4
pyenv install 3.10.12
pyenv install 3.9.5
pyenv install 3.8.10
pyenv install 3.7.10
pyenv install 3.6.13

# make sure to setup a global version
pyenv global 3.11.4

Note: in our real life, we have little need for historical versions. So we end up with installing the latest Python available (that makes 2 versions on the environment) and setting it to global.

Installing pipx

We are going to install pipx for managing other python applications in nicely sandboxed environments. The tools basically creates an environment per tool, nicely exposing the features, and at the same time being easy to use (you don't need to activate specific environments etc.). We love the convenience.

NOTE: make sure to setup the pyenv global version, as described in previous section!

  • Install pipx
# Install pipx  
pip install pipx

# Make sure pipx is added to the path  
pipx ensurepath

Note: in case you were wondering, what is the way to add extra packages to your favorite pipx application, the search is over: you can use pipx inject distroname packagename to do so.

  • Install global apps via pipx. Consider the below as an example, it's completely up to you. It's also time to add your favorite extensions
# installing our global tools via pipx
pipx install cookiecutter
pipx install mkdocs
pipx install poetry

# install one or more linters, based on your preference
pipx install git+https://github.com/psf/black
pipx install pylint
pipx install flake8
pipx install ruff

# installing jupyter (jupyter7 is coming, so check this before proceeding)
pipx install jupyter --include-deps

# extend mkdocs with our favourite extensions
pipx inject mkdocs pymdown-extensions mkdocs-material markdown-include cython nbconvert
  • [optional] If you have installed poetry as described above, you might want to install the poetry recommended bash completion:
poetry completions bash >> ~/.bash_completion

Wrapping Up

If you have followed our guideline up to now, you have a brand new distro on WSL. It took time to tweak it well and to configure everything. In this section, you will cleanup and prepare for re-use.

  • Do a final update
sudo apt update && sudo apt upgrade
  • Go to your Windows terminal, and backup your distro. Be aware - the export closes a running distro, so be sure to have all installations done.
wsl.exe --export <distro name> <file name to export to>

Keep your backup somewhere safe. There is bound to be a time when you break your development environment beyond repair. Then, instead of spending hours to fix it, you can simply re-import and update. Ready for action again!

Notes

Running pyenv on WSL and pyenv-win

You will get some nasty errors, when you setup our environment, and then use pyenv on Windows as well. This is due to a path mismatch (WSL appends the WIndows path)

windows - Installing pyenv-win on top of working wsl2 Ubuntu instance breaks Ubuntu pyenv - Stack Overflow

Poetry and Virtual Environments

  • Poetry will detect and respect an existing virtual environment that has been externally activated. This is a feature we use (by working with pyenv) instead of the built-in virtual environment settings. This means that we have also disabled venv handling in poetry:
# set the poetry virtualenv settings to false

poetry config virtualenvs.create false
poetry config virtualenvs.in-project false

Using pyenv

Here is a short cheat sheet for pyenv

  • List available python versions for installation (a long list)
pyenv install --list
  • Install a specific Python version
# example - installing Python 3.11.4 and 3.10.12

pyenv install 3.11.4
pyenv install 3.10.12
  • Set global python version
pyenv global 3.11.4
  • Create a virtual environment and activate it
pyenv virtualenvironment mygreatenv
pyenv activate mygreatenv
  • List virtual environments
pyenv virtualenvironments

Using pipx

Below we provide a small reminder of basic pipx functionality.

  • List all pipx installed applications
pipx list
  • Install a pipx Python app
# installation examples with applications we use
pipx install mkdocs

# install an application where python executables are included in dependencies
pipx install jupyter --include-deps
  • Install dependencies into an existing application environment
pipx inject mkdocs mkdocs-material
  • Upgrading
# upgrade specific application
pipx upgrade mkdocs

# or, you can upgrade everythong
pipx upgrade-all
  • Uninstalling
pipx uninstall mkdocs

Our WSL articles: