How to download Python dependencies

how to download Python dependencies

Most of the time, you’ll use a package management solution like pip or the ActiveState Platform to install your dependencies. The package manager will automatically download the dependency along with any requirements for you.

Additionally, unlike pip, the ActiveState Platform will automatically resolve all dependencies to ensure your environment doesn’t break.
You can try the ActiveState Platform by signing up for a free account. As a result, in normal practice, you’ll never need to manually download and resolve dependencies yourself.

However, there are a few instances in which you may choose to manually download your dependencies, such as:

  • The system you’re working on is off-line, air-gapped or has an unreliable connection.
  • Your organization wants to create a local repository of Python packages.
    • You might need to do this if the set of packages you work with need to be approved by Legal (ie., for licensing), or similar
  • Other reasons

If you choose to create your own repository, you’ll need to be able to download all the packages you require, along with their dependencies – all without installing them.

Note that in these cases, you may also need to download OS-specific dependencies if they are meant to be deployed on multiple operating systems. The easiest way to get the source code for all your dependencies including OS-specific dependencies is to use the ActiveState Platform. 

Download Python Dependencies with Pip

The pip download command can be used to download packages and their dependencies to the current directory (by default), or else to a specified location without installing them.

Download Package and Dependencies

For example, to download the requests package and all its dependencies to the current directory without installing them, do the following:

(current directory) $ pip download requests

The output should look similar to:

    certifi-2020.4.5.1-py2.py3-none-any.whl
    chardet-3.0.4-py2.py3-none-any.whl
    idna-2.9-py2.py3-none-any.whl
    requests-2.23.0-py2.py3-none-any.whl
    urllib3-1.25.9-py2.py3-none-any.whl 

Download Dependencies Only

In some cases, you may want to download just the dependencies for a package, excluding the package itself. In this case, you have two options: 

  • Use the pipdeptree utility to gather a list of all dependencies, create a requirements.txt file listing all the dependencies, and then download them with the pip download command.
  • Get the list of dependencies for a package from the setup.py file

Get Dependencies with Pipdeptree

  1. Install pipdeptree:
a.$ pip install pipdeptree 

2. As an example, generate a dependency tree for requests

 $ pipdeptree -p requests 
requests==2.23.0
  - certifi [required: >=2017.4.17, installed: 2020.4.5.1]
  - chardet [required: >=3.0.2,<4, installed: 3.0.4]
  - idna [required: >=2.5,<3, installed: 2.9]
  - urllib3 [required: >=1.21.1,<1.26,!=1.25.1,!=1.25.0, installed: 1.25.9]

3. Copy the dependencies and version information into a requirements.txt file:

  certifi>=2017.4.17
  chardet>=3.0.2,<4
  idna>=2.5,<3
  urllib3>=1.21.1,<1.26,!=1.25.1,!=1.25.0

4. You can now download the dependencies to the current directory without installing them:

(current directory) $ pip download -r requirements.txt

Get Dependencies from Setup.py

To get a list of dependencies from a setup.py file, do the following:

  1. Open the setup.py file and scroll to the install_requires section, which should look similar to:
setup(
...  
install_requires=[
'<depname>',
'<depname>'
] 
...)

2. Copy the dependencies listed in the install_requires section and paste them into a requirements.txt file.

3. Download the dependencies for the setuptools project:

(current directory) $ pip download -r requirements.txt -d <location>

For more information about setup.py files and the install_requires specification, refer to: How to use Python Dependency Management Tools

Download Python Dependencies for a Pipenv Project 

If you’re working with pipenv, the commands for downloading dependencies are slightly different. 

Download Pipenv Package and Dependencies

For example, to download the requests package and all its dependencies to the current directory without installing them, do the following:

(current directory) $ pipenv download requests

The output should look similar to:

    certifi-2020.4.5.1-py2.py3-none-any.whl
    chardet-3.0.4-py2.py3-none-any.whl
    idna-2.9-py2.py3-none-any.whl
    requests-2.23.0-py2.py3-none-any.whl
    urllib3-1.25.9-py2.py3-none-any.whl 

Download Pipenv Dependencies 

To download just the dependencies for a pipenv project to a specific location without installing them:

  1. Open the pipfile.lock file for your pipenv project
  2. Copy the dependencies listed in the requires section, and paste them into a requirements.txt file. For example:
  "_meta": {
        "hash": {
            "sha256": "<hashcode>"
        },
        "pipfile-spec": 6,
 "requires": {
            "<depname>": "version#"
            "<depname>": "version#"
  }
...}

3. To download the dependencies for the pipenv project to a specified location without installing them:

$ pip download -r requirements.txt -d <location>

Download Python Dependencies in a Conda Environment

Conda environments also require different commands to create a repository of dependencies.

Download Conda Package and Dependencies

For example, to download the requests package and all its dependencies to the current directory without installing them, do the following:

(current directory) $ conda install --download-only requests

Download Conda Dependencies Only

The simplest way to download just the dependencies for a package in a conda environment is to run the following command:

$ conda install  --download-only --only-deps requests

You can also download the dependencies for any package using the conda info command to first list all the dependencies for a specific package, and then copying those dependencies into a requirements.txt file. 

  1. For example, to view a list of dependencies for the requests package, enter:
$ conda info requests 
   requests 2.13.0 py36_0
   ----------------------
   file name   : requests-2.13.0-py36_0.tar.bz2
   name        : requests
   … 
   dependencies:
    certifi >=2017.4.17
    chardet >=3.0.2,<4
    idna >=2.5,<3
    python
    urllib3 >=1.21.1,<1.26,!=1.25.0,!=1.25.1

2. Copy the list of dependencies, including version information into a requirements.txt file.

3. Run the following command to download the dependencies without installing them:

$ conda install --download-only --file requirements.txt 

The dependencies will be downloaded to the \pkgs folder in the Anaconda\Conda root directory. The root directory in both Linux and Windows is the Anaconda\Conda installation directory.

Download Dependencies in a Poetry Environment 

Poetry environments download all dependencies from PyPI, so you can use pip to do the actual download for you. But if you want to list all the dependencies in your Poetry environment to ensure you capture them all, you’ll need to use specific Poetry commands and files.

Download Poetry Package and Dependencies

As mentioned, Poetry installs dependencies from PyPI, so you can use the pip command to download a package and all of its dependencies. For example, to download the requests package and all its dependencies to the current directory without installing them, do the following:

(current directory) $ pip download requests

The output should look similar to:

    certifi-2020.4.5.1-py2.py3-none-any.whl
    chardet-3.0.4-py2.py3-none-any.whl
    idna-2.9-py2.py3-none-any.whl
    requests-2.23.0-py2.py3-none-any.whl
    urllib3-1.25.9-py2.py3-none-any.whl 

Download Poetry Environment Dependencies Only

You can identify just the dependencies for a Poetry environment in one of two ways: 

  • View the pyproject.toml file`s dependencies section
  • Using poetry show command to list all dependencies in a visual tree

You can then copy these dependencies into a requirements.txt file and use the usual pip command to download them.

Get Dependencies with Pyproject.toml

Poetry environment dependencies are specified in pyproject.toml files under the [tool.poetry.dev-dependencies] section. For example:

[tool.poetry]
...
[tool.poetry.dev-dependencies]
pytest = "^5.2"
...

Copy the dependencies listed in the dev-dependencies section into a requirements.txt file. See step #3 below.

Get Dependencies with Poetry Show

Alternatively, you can use the poetry show command to list Poetry project dependencies in a visual tree that may be helpful in understanding the overall dependency structure. To use poetry show, do the following: 

  1. Cd into the project and enter:
(poetry project) $  poetry show -t 

The output should look similar to:

   pytest 5.4.2 pytest: simple powerful testing with Python
  |-- atomicwrites >=1.0
  |-- attrs >=17.4.0
  |-- colorama *
   ...

2. Copy the dependencies list into a requirements.txt file

3. Now that we have a requirements.txt file with our dependencies, we can download them. Note that because Poetry downloads packages from PyPI by default, you can actually use Pip to download the dependencies for your Poetry environment and save them to a specific location. To do so cd into your Poetry project, and enter:

(poetry project) $ pip download -r requirements.txt -d <location>

Download Python Dependencies With the ActiveState Platform

The ActiveState Platform GraphQL API can be used to download the source code for packages, their dependencies and even OS-level dependencies without installing them. This can be helpful if you need to patch the code, or otherwise work with the non-binary version.

To download dependencies:

  1. Sign up for a free account and create your project
  2. Read through the ActiveState Platform’s Build Graph API interactive documentation to get a better understanding of the API
  3. Run an API query for your project to display links to all the source code for you to download. For example, you could modify this API query to return the links to the source code for your project by:
    1. Changing org from ActiveState to the name of your organization
    2. Changing name from ActivePython-3.8 to the name of your project
    3. Changing sources limit to display more than just a few results

Build Graph API Python 3.8

Need to build a runtime environment for your Python project? The ActiveState Platform gives you access to a pre-configured build environment:

  • Eliminate Dependency Hell – dependencies are automatically resolved for you, where possible
  • Automated Packaging – minimal language/OS expertise required
  • No Tooling – no compiler to source/set up
  • Automated Setup – install your runtime into a virtual environment with a single command using the State Tool.

Watch how the platform helps developers kick-off their Python projects faster

Recent Posts

Tech Debt Best Practices: Minimizing Opportunity Cost & Security Risk

Tech debt is an unavoidable consequence of modern application development, leading to security and performance concerns as older open-source codebases become more vulnerable and outdated. Unfortunately, the opportunity cost of an upgrade often means organizations are left to manage growing risk the best they can. But it doesn’t have to be this way.

Read More
Scroll to Top