How to Package Python dependencies with PIP setuptools

If you’re packaging a Python package for publication, how do you ensure that all of the required dependencies are included? Just like pip is the standard package manager for Python, setup.py is the heart and center of Python projects installed with pip. Simply put, setup.py is a build script template distributed with Python’s setuptools package.

Setuptools is the Python Packaging Authority (PyPA) package development process library and utility for building Python projects based on packages and their dependencies listed in a setup.py script. A Python file that relies only on the standard library can be redistributed and reused without the need to use setuptools. But for projects that consist of multiple files, need additional libraries, or need a specific version of Python, setuptools will be required.

This article provides a brief tutorial on how to work with setuptools.

Update Python Tools

Setuptools is typically installed with Python downloaded from python.org, so there’s no need to separately install setuptools. Instead, your first step should be to use pip to update your Python installation to the latest version of setuptools on popular operating systems like Windows, Linux or macOS:

$ python -m pip install –upgrade pip setuptools

If you get a permissions error, instead of using sudo, consider creating a virtual environment with virtualenv, venv, pipenv or pyenv. If you’re working with more than one project, rather than installing everything in /site-packages, you’ll want to work with virtual environments to ensure each project has just the dependencies it requires.

Setup.py Example

The following script is an example of how to create a setup.py script for your Python 3 (not Python 2) project. The script syntax spells out the information you need to provide, including metadata such as:

• Project name – enter a name for your project in quotes
• Python packages and Python modules to include in the distribution (dist) – the find_packages(‘,’) default argument will incorporate all packages that include an __init__.py file and are located in the local directory (dir) where setup.py is installed
• Project version number – enter a version number for the project in quotes
• List a license for the project – enter the name of the license you are licensing your project for use under in quotes
• Short description of your library (lib) – enter a brief description of your project in quotes
• Long description of your library (lib) – enter a more detailed description of your project either in text or using markdown
• Your name – enter your name in quotes to denote who the author is
• Your email address – optionally provide an email address in quotes where users can contact you
• Link to your GitHub repository or website – optionally provide a link to your project’s repo, list your maintainers, etc
• Download Link from where the project can be downloaded from – provide an URL to your project’s source code in quotes
• List of keywords – provide a list of keywords in square brackets associated with your project to make it easier to search for
• List project dependencies – provide a list of all the dependencies your project requires in square brackets. The easiest way to provide this is to copy and paste from the install_requires section (see below).
• https://pypi.org/classifiers – provide a list of all classifiers that apply to your project in square brackets. You can find a comprehensive list of classifiers at https://pypi.org/classifiers

from distutils.core import setup
from setuptools import find_packages
import os
# Optional project description in README.md:
current_directory = os.path.dirname(os.path.abspath(__file__))
try:
with open(os.path.join(current_directory, 'README.md'), encoding='utf-8') as f:
long_description = f.read()
except Exception:
long_description = ''
setup(
	# Project name:
name='',
# Packages to include in the distribution:
packages=find_packages(','),
# Project version number:
version='',
# List a license for the project, eg. MIT License
license='',
# Short description of your library:
description='',
# Long description of your library:
long_description=long_description,
long_description_content_type='text/markdown',
# Your name:
author='',
# Your email address:
author_email='',
# Link to your github repository or website:
url='',
# Download Link from where the project can be downloaded from:
download_url='',
# List of keywords:
keywords=[],
# List project dependencies:
install_requires=[],
# https://pypi.org/classifiers/
classifiers=[]
)

Install_requires Example

install_requires is a section within the setup.py file in which you need to input a list of the minimum dependencies needed for a project to run correctly on the target operating system (such as ubuntu). When pip runs setup.py, it will install all of the dependencies listed in install_requires.

For example, if your project includes matplotlib, you’ll need to list it, as well as its dependency of NumPy in install_requires as shown below:

setup(
	...
install_requires=[
'<matplotlib>',
'<numpy>'
]
...)

Example of Install_requires with Platform Specific Dependencies

You can modify install_requires by adding in platform-specific and version-specific  dependencies, depending on the environment that a project is to be installed on. For instance, if our previous example included matplotlib v3.2.1 and numpy v1.17.4 on Python 3.6.6 for Linux, we could specify:

setup(
	...
install_requires=[
"<matplotlib>;python_version<'<3.6.6>'",
"<numpy> >= <1.17.4>;platform_system=='<Linux>'"
]
...)

To install a setup.py file including dependencies listed in install_inquires:

$ python setup.py install

When the command is run, all of the dependencies not already installed will be downloaded, built (if necessary), and installed. Any scripts that require specific dependencies at runtime will be installed with wrappers that ensure the correct versions are added to sys.path (system path).

Dependency Not Found in PyPI

If your project has a dependency requirement that is not currently in the Python Package Index (PyPI), you can still include them if they can be accessed via http and are packaged as either an egg, .py file, or a VCS (Version Control System) repository, such as Git or Subversion, rather than just a tarball (tar.gz).

In order to download dependencies not found in PyPI, you will need to add URLs to a dependency_links section under setup() in the setup.py file. Assuming that the dependencies are packaged correctly, they will be automatically installed:

setup(
	...
dependency_links=['http://github.com/<username>/<reponame>/tarball/master#egg=<packagename>-<version#>']
...)

To check a dependency not found in PyPI, prefix the package name and version # with _ (instead of -) in the dependency_links argument:

dependency_links=['http://github.com/<username>/<reponame>/tarball/master#egg=<packagename>_<version#>']

A modern solution to Dependency management – Try ActiveState’s Platform

Dependency resolution is at the core of the ActiveState Platform. When you create a project and start adding requirements, the Platforms tell you what dependencies those requirements have.

The ActiveState Platform is a cloud-based build tool for Python. It provides build automation and vulnerability remediation for:

• Python language cores, including Python 2.7 and Python 3.5+
• Python packages and their dependencies, including:
• Transitive dependencies (ie., dependencies of dependencies)
• Linked C and Fortran libraries, so you can build data science packages
• Operating system-level dependencies for Windows, Linux, and macOS
• Shared dependencies (ie., OpenSSL)
• Find, fix and automatically rebuild a secure version of Python packages like Django and environments in minutes

The ActiveState Platform aims to handle every dependency for every language. That means handling libraries down to the C/C++ level, external tools, and all the conditional dependencies that exist. To take things even further, our ultimate goal is to support multi-language projects. That means that you can create a project using both Python and Perl packages, and we’ll make sure that both languages are using the same (up to date) OpenSSL version.

Python Dependency Management In Action

Get a hands-on appreciation for how the ActiveState Platform can help you manage your dependencies for Python environments. Just run the following command to install Python 3.9 and our package manager, the State Tool:

Windows

powershell -Command "& $([scriptblock]::Create((New-Object Net.WebClient).DownloadString('https://platform.activestate.com/dl/cli/install.ps1'))) -activate-default ActiveState-Labs/Python-3.9Beta"

Linux

sh <(curl -q https://platform.activestate.com/dl/cli/install.sh) --activate-default ActiveState-Labs/Python-3.9Beta

Now you can run state install <packagename>. Learn more about how to use the State Tool to manage your Python environment.

Let us know your experience in the ActiveState Community forum.

Watch this video to learn how to use the ActiveState Platform to create a Python 3.9 environment, and then use the Platform’s CLI (State Tool) to install and manage it.

Frequently Asked Questions