- Resolve the errors in order to save time writing the code you tried to import, or
- Rewrite/monkey patch the code in order to save the time spent in Dependency Hell.
Not much of a choice.
Dependency Hell occurs when the process of trying to resolve the initial environment error uncovers even more errors. Errors typically take the form of:
- Dependency Conflicts – occurs when two software packages require the same dependency, but each one requires a different version of the dependency.
- Circular Dependencies – occurs when package A depends on a specific version of package B, which, in turn, depends on a specific version of package A. Upgrading any one package will break the other.
- Diamond Dependencies – occurs when the conflict exists far down in the dependency tree, such as when dependencies rely on a sub-dependency, but each one requires a different version of the sub-dependency. For example:
These problems have been with us ever since we started reusing code decades ago, and yet there has never been a good solution. The problem is that software is not static. New versions, patches and updates are constantly being released, any one of which may or may not be compatible with your application’s environment, but there’s essentially no way to know until you try importing it. Package authors are supposed to obey standard semantic versioning rules and create major versions of their package when they break compatibility with former versions of dependencies. Unfortunately, semantic versioning is rarely enforced in open source ecosystems and even more rarely adhered to by open source authors.
Package installers will happily install their packages into your environment, irrespective of the consequences. Most package managers will at least warn you that something looks wrong. Unfortunately, if you go ahead and import the new package anyway, you risk spending the rest of your day in Dependency Hell, trying to unwind the changes to your environment and get all your packages working together again.
Solutions to Dependency Hell
Most of us take advantage of dependency pinning to ensure we don’t get ourselves into dependency problems in the first place. But inevitably you’ll need to upgrade something, either to resolve a bug or a vulnerability or even to deal with an issue in the underlying language. You’ll want to have clear dependency management guidelines in place before you begin and make sure everyone is aware of them. Typical guidelines include:
- Minimize dependencies.
- Delete unused/unnecessary dependencies to trim your dependency tree.
- Ensure your remaining dependencies are correct:
- Dependencies that need to be pinned, are.
- Dependencies that need to be upgraded (i.e., because of a vulnerability) have been.
- Dependencies that may have been customized or patched are known.
- Standardize on a single package manager per language.
- For example, if you’re working in Python, you don’t want a requirements.txt from conda and one generated by pip.
- Follow semantic versioning, where possible.
- Understand the limitations of your package manager(s) when it comes to resolving dependencies and dependency conflicts.
The last point is key because different package managers take different approaches and have different capabilities, including:
- Dependency resolution
- Dependency conflict identification
- Warn/notify but allow installation
- Warn and prevent installation
- Dependency conflict workaround suggestions
For example, the ActiveState Platform will resolve dependencies, flag conflicts, and even provide you with solutions to conflicts it can’t automatically resolve. And you can use the ActiveState Platform to do dependency management against your environment configuration rather than your local, installed environment. In this way, you can see the ramifications of your change(s) at all levels, from top-level packages to dependencies to transitive dependencies and even OS dependencies BEFORE you apply the changes to your installed environment, ensuring against environment corruption.
The ActiveState Platform also provides you with instructions on how to resolve any conflicts that may arise, ensuring you always have a way to escape Dependency Hell. See how it works:
Finally, the ActiveState Platform won’t allow you to apply a non-resolvable configuration to your existing local environment, saving you from corrupting your existing installation.
Deep dive into dependency hell with our CTO Scott Robertson and understand what it means to solve dependency hell at scale. Watch our webinar ‘Solving Dependency Hell At Enterprise Scale’
Next steps:
The capabilities shown here are also available as a managed service, freeing up your developers to focus on coding and getting your product to market faster. Learn more about our Managed Builds service.
Continue reading:
- Watch our Webinar ‘Solving Dependency Hell At Enterprise Scale’
- Read about our approach to Python Package Management
- Read about our approach to Perl Package Management
- Watch a quick video overview of ‘What to do when you encounter a Solver Error’
- Learn more about our approach to ‘dependency resolution optimization’
- See how the ActiveState Platform compares with other common dependency management tools
