After upgrading from Ubuntu 22.10 to 23.04, pipx broke on my laptop and fixing it was not exactly straight-forward, so I thought I'd write here about this, in case there are others running into this issue. While I ran into this issue on Ubuntu, this isn't Ubuntu-specific, and it can happen on other Linux distributions as well.

pipx is a tool that makes it easy to install (using PyPI as the package index) isolated Python applications. pip is a Python package installer for both libraries and apps, while pipx is made specifically for applications, creating an isolated environment for each app and its dependencies.

The next Debian/Ubuntu releases will likely no longer allow pip install outside a virtual environment or separate Python installs because of conflicts between pip and the OS package manager. It can still be forced, but that's strongly not recommended.

Software shipped with a Linux distribution can be (quite easily, I might add) broken by installing packages using pip. All the user has to do is install a package (or some of its dependencies) that's newer and backwards-incompatible with a version that was installed from the Linux distribution official repositories. And this isn't even about breaking some application—because Python is so widely used nowadays, users can easily break critical distro packages.

The breakage can occur for both system-wide pip installations (sudo pip install), as well as user installations (pip install --user), but especially the latter, since trying to recover from this might result in removing (using pip, not the distro package manager) packages installed using the Linux distribution's package manager.

For this reason, the next Debian (Debian 12 Bookworm) and Ubuntu (Ubuntu 23.04 Lunar Lobster) releases will likely adopt PEP668 (PEP = Python Enhancement Proposal), marking the Python base environments as "externally managed", and no longer allowing regular pip install usage for both user and system installations. This can still be forced though, and obviously, there are alternatives—see below.

The change is already live in Debian Testing and Ubuntu 23.04 Lunar Lobster (which will have a beta release on March 30, with the final release expected on April 20). There's also a proposal to include this in Fedora 38 (which had a beta release today), but this has not landed in Fedora 38 for now.

I said "likely adopt" because, even though this change is already present in Debian Testing and Ubuntu 23.04 Lunar Lobster, Stefano Rivera, one of the Debian/Ubuntu Python maintainers, mentioned that "if necessary, we can roll back EXTERNALLY-MANAGED in our python3.11 for bookworm’s release, but I’d like to make this happen…". 

[[Edit]] This change has made it into the Ubuntu 23.04 Lunar Lobster release! When trying to install a Python package using pip, users will now see this message: "error: externally-managed-environment" / "This environment is externally managed".

So what are the alternatives to using pip install directly and solve this externally managed environment pip error? Take a look below: