Dependency Management with Pip, Python’s Package Manager

Dependency Management with Pip

Pip is Python’s package manager, providing essential core features for installing and maintaining Python packages and their dependencies. Dependency Management with Pip was never intended to be as pip will not flag dependency conflicts. As a result, it will happily install multiple versions of a dependency into your project, which will likely result in errors.

But pip has also been integrated into higher-level tools, such as Pipenv which utilizes Pip’s command-line interface (CLI) to help perform enhanced dependency management in virtual environments. There are several different approaches to dealing with dependencies in Python, one of which is using Pip with Pipenv.

Another solution for better managing dependencies with Pip is to use it in conjunction with a requirements.txt file, which allows for the pinning of packages and their dependencies to specific versions (details below). 

Dependency Management with Pip

Pip is installed by default with any version of Python greater than v3.4. To check which version of Pip you are running, enter the following command in a terminal or command window:

$ pip –version

You can also check to see if Pip is up-to-date:

$ pip install –upgrade pip

The pip command installs packages and their dependencies from PyPI (Python Package Index) by default:

$ pip install <packagename>

Pip relies on package authors to stipulate the dependencies for their code in order to successfully download and install the package plus all required dependencies from the Python Package Index (PyPI). 

But if packages are installed one at a time, it may lead to dependency conflicts. A common scenario may involve an initially installed “Package 1” that requires v1.0 of “Dependency A,” while a subsequently installed “Package 2” requires v1.2 of “Dependency A,” which is incompatible with Package 1. 

Instead, consider installing a pre-built runtime environment, such as Anaconda or ActivePython, or else installing all required packages for your project from a requirements.txt.

Wheels and Dependencies

A wheel is an archived Python project consisting of one or more packages and their dependencies. Pip is able to install packages configured with their dependencies, because most authors package their code as wheels by default before submitting to PyPI.

It is not necessary to provide the full wheel name when installing a package from PyPI. The package name is all that is required:

$ pip install <packagename>

For example, the Twine package and its dependencies are provided in a Wheel named ‘twine-3.1.1-py3-none-any.whl‘ located in https://pypi.org/project/twine.

Twine package and its dependencies

Requirements.txt and Dependencies

Pip does not provide true dependency resolution, but this can be solved by using it in conjunction with a requirements.txt file. Requirements.txt files can be used to make pip resolve dependency conflicts between different packages.

The requirements.txt example below resolves a dependency conflict between two packages, where <package1> and <package2> require different versions of the same dependency.

If <package1> requires <package3==1.0> and <package2> requires <package3>=2.0, and if <package1> is resolved first, then Pip will use <package3>==1.0. In this scenario, a <package3> version that conflicts with <package2>‘s dependency requirement will be installed. The solution is to place <package3>=2.0 in the requirements.txt file together with the other requirements.

Requirements.txt

<package1>

<package2>

<package3>=2.0

To install a requirements.txt file, cd into the directory where the file is be located, and enter:

$ pip install -r requirements.txt

-r, –requirement option installs multiple packages listed in a requirements.txt file.

Requirements.txt files can be used multiple times in a default single environment or in multiple virtual environments, as a method of version control to override dependencies with patches (different package versions).

Managing Dependencies when Updating Python Packages

Updating one or more packages in a project can result in dependency conflicts since newer package versions will often require newer versions of dependencies, as well. 

To learn more about updating Python packages, refer to our Quick Read: How to Update all Python Packages.

Related Quick Reads:

How to Best Manage Python Dependencies

Managing Dependencies & Runtime Security

Mike