System-wide installation of Python (also called a global installation) is the most popular way to install Python. In fact, the operating system you use probably came with a pre-installed, global installation of Python since many OS’s make use of Python for various routines. And if you download Python from Python.org or a commercial vendor like Anaconda (or even our own ActivePython Community Edition), it will be installed system-wide by default.
If you only work with your original installation of Python, the rest of this article may not apply to you. But most of us have had to upgrade to a newer version of Python at some point. Of course, since we’re busy developers, we install first and ask questions later, after things go wrong:
- Why do I need to fix the ‘/usr/bin/python’ symlink before I can even do the install?
- Why am I still working with the old version of Python after installing a newer version?
- Why is the package I was working with before I upgraded now “not found?”
In other cases, it’s not Python itself that’s the problem, but rather the third party packages you use to build your applications. Most developers install these packages with pip, which downloads them from the Python Package Index (PyPI) and installs them locally into your /site-packages directory. Unfortunately, Python doesn’t distinguish between different versions of the same package in the /site-packages directory, which leads to problems when you have two projects that require different versions of the same package.
These issues mean that even after three decades, it’s still all too easy to screw up your Python installation despite – and in some cases because of – multiple solutions being created to help.
Python Environment – by xkcd.com
Of course, global environment corruption is not unique to Python (it’s just as popular with many other open source languages), but it’s always painful when the packages you were just working with can suddenly “no longer be found” sending you hunting for directories and manually updating symlinks and PATH environment variables.
Virtual Python Environments – A Better Solution?
One way to avoid global Python environment corruption is to avoid global installations in the first place. That’s where virtual environments come in. They’re essentially just directory trees that contain:
- A version of Python
- All the third party packages your project requires
- Your project code
But the key benefit of virtual environments is that they isolate each of your project(s) from any other installation of Python on your computer, allowing multiple versions of Python to coexist without stepping on each other. So now you can work on upgrading that Python 2 project to Python 3, or test that the Python 3.6 application you built works on Python 3.8 – all on the same system.
While virtual environments solve many of the problems associated with global installations of Python, they introduce a number of other issues:
- Different versions of Python and different operating systems require different tools to create and work with virtual environments, from pyenv to pywin to virtualenv to venv, and more
- There is no central console (although virtualenvwrapper definitely helps), so tracking and managing multiple virtual environments gets harder the more environments you create
- Virtual environments are terminal-specific, activated and deactivated on a per-terminal basis
While PEP 582 has been proposed to update the Python core to address many of these issues, it remains just a proposal for now.
ActiveState Virtual Python Environments
At ActiveState, we’ve been handcrafting our ActivePython Community Edition (CE) distribution, which contains a version of Python as well as hundreds of popular third-party packages, for almost two decades. But our time to market has been getting longer and longer, so we created the ActiveState Platform to help us automate the creation of ActivePython CE, as well as our other language distributions.
Unfortunately, ActivePython CE continues to be part of the problem, since it provides installers that create yet another global install by default.
As a result, we’ve been working on building virtual environment capabilities into our State Tool, which is the command-line interface for the ActiveState Platform. The State Tool offers:
- A native way to install ActivePython into virtual environments with a single command
- A better way to manage multiple projects than virtualenvwrapper by providing the ability to list, switch between, update, restore, fork, and even share them with your team members using a single appropriate command
- Cross-platform support, including Windows, Mac, and Linux
- Cross-language support, including Perl and Python (more languages coming)
Those Pythonistas used to working with virtual environments will find much to like in the State Tool’s implementation. Those that haven’t worked with virtual environments previously may be a little lost at first, but we’re working on a quick start tutorial that should help ease you into the habit.
You can try it out today by installing the State Tool and then using it to automatically install a copy of ActivePython 3.6 into a virtual environment.
For Windows users, run the following in Powershell to install the State Tool:
IEX(New-Object Net.WebClient).downloadString('https://platform.activestate.com/dl/cli/install.ps1')
Now type the following at a cmd prompt in order to automatically create a virtual environment, and then download and install ActivePython 3.6 into it:
state activate ActiveState/ActivePython-3.6
For Mac and Linux users, run the following command to install the State Tool:
sh <(curl -q https://platform.activestate.com/dl/cli/install.sh)
Now run the following to automatically create a virtual environment, and then download and install ActivePython 3.6 into it:
state activate ActiveState/ActivePython-3.6
To learn more about working with the State Tool, refer to the documentation. And don’t forget to let us know about your experience in our Community Forums.
Listen to the episode related to Python environments on The Local Maximum Podcast. Team Leader Shaun L of ActiveState joins show host Max for a short but insightful discussion.
Related Blogs:
