Managing open source dependencies for a software project is one of those thankless jobs that everyone takes for granted until they’re forced to do it for themselves. While most modern package managers can help keep your dependencies straight for smaller projects that use a single language, as complexity increases, package manager drawbacks become more apparent.
For example, relying solely on a package manager to manage your dependencies can result in problems with:
- Tool proliferation – the more operating systems and programming languages your team uses, the more tools you’ll need. This can mean not only multiple package managers (one per OS and language), but also multiple compilers, packagers, etc, as a result of:
- Availability – Prebuilt packages are not always available for every OS, in which case you’ll have to build them yourself.
- Version Locking – Pinning dependencies to a specific version is a best practice to ensure reproducibility of environments. But poor locking practices can lead to problems when:
- Known vulnerabilities are not addressed because upgrading to the latest version of a package may break the environment/application.
- Only top level dependencies are pinned allowing transient dependencies to shift over time, eventually leading to an unreproducible environment.
- Disappearing Dependencies – Like it or not, things change on the Internet. Open source dependencies can be deleted by the developer or else corrupted, potentially breaking your environment.
Organizations that have experienced one or more of the above issues are usually willing to try a different approach, such as implementing an artifact repository that provides a central source of truth for all the open source dependencies your development teams require.
This blog post is meant to help you understand the pros and cons of managing your dependencies in an artifact repository, as well as how you can do so in an automated, scalable way.
Dependency Management Problems
No product manager wants their development team to spend valuable velocity points managing dependencies. And very few developers would prefer to manage dependencies rather than code. As a result, dependency management is something of a red-headed stepchild when it comes to software development.
Risk of Not Updating Dependencies
This is most evident when numerous reports and surveys repeatedly confirm the fact that once a dependency is added to a codebase, it’s almost never updated despite mounting risks ranging from mild to irresponsible:
- Auditing open source dependencies becomes increasingly difficult.
- Performance lags as outdated versions miss out on enhancements that significantly improve speed and/or functionality.
- Eventually, someone is forced to update a dependency, likely breaking the environment and/or the build.
- You now end up with a snowball of updates to fix what broke, which breaks something else, and so on, making the problem worse.
- Everyone’s local deployed environment is now out of sync, creating “works on my machine” errors and environment reproducibility issues.
- Critical vulnerabilities go unpatched, exposing the organization to cyberattack.
Work To Update Dependencies
But the old adage of “if it ain’t broke, don’t fix it” remains a strong deterrent to updating dependencies. Well, that and all the work involved, which might include:
- Checking the new version of the dependency’s Readme to understand what’s added/changed
- Creating a branch in your CVS system for the fix
- Updating the dependency
- Committing the files
- Resolving the set of dependencies that are no longer compatible with the new dependency
- Rebuilding/retesting your branch to make sure nothing broke
- Fixing what inevitably broke
- Ensuring your colleagues understand what’s changed and why
- Updating all the environments across dev, test, CI/CD, staging, etc
This sharp distinction between the risk and work associated with managing dependencies is where most organizations end up inflicting a lot of self-harm. One way to help balance the work versus risk is to use an artifact repository.
Dependency Management & Artifact Repositories
An artifact repository provides a central place to store built artifacts, such as open source dependencies, and makes them available to the systems and teams that require them. But most enterprises also use them to:
- Gate Assets – most enterprises have an approval process that must be passed before new open source packages can be made available for use. An artifact repository can help act as a gating mechanism, making new open source packages available to the organization only as and when they’re approved.
- Manage Vulnerabilities – many artifact repositories also include the ability to identify and notify users as and when open source dependency vulnerabilities occur.
- Ensure Compatibility – all open source dependencies made available via the repository can be pre-determined to work together, relieving developers of the need to independently resolve dependency conflicts.
- Ensure Security – security-conscious enterprises typically build all open source dependencies from source code, and manually populate them in the artifact repository. This avoids the risk posed by ingesting prebuilt components from public repositories that offer no security guarantees.
In these ways, an artifact repository can be an excellent way to centralize the management of a standard set of open source dependencies required by all development teams. However, while artifact repositories provide a point of focus for managing dependencies, they don’t automate any of these tasks, which can require significant time and effort.
Dependency Management Automation
By combining an artifact repository with the ActiveState Platform, organizations can automate many time consuming, manual dependency management tasks, including building dependencies from source code, finding and fixing vulnerabilities, and ensuring security:
- Find & Fix Vulnerabilities – get notified when a dependency becomes vulnerable; select a non-vulnerable version of it, and rebuild the dependency and repopulate the artifact repository automatically.
- Get out of Dependency Hell – automatically resolve all dependencies, and gain manual workarounds to dependency conflicts, ensuring all dependencies can be deployed together without error.
- Secure Builds – ensure the security and integrity of dependency by building them securely from source code.
See how it works:
Next steps:
The capabilities shown here are also available as a managed service, freeing up your developers to focus on coding and getting your product to market faster. Learn more about our Managed Builds service.
Continue reading:
- Read our Data Sheet: ActiveState Platform’s Artifact Repository
- Read our Data Sheet: ActiveState Platform’s Secure Build Service
- Read our blog: How To Automatically Eliminate Dependency Hell
- Read our white paper: Perl Package Management Guide for Enterprise Developers
- Read our white paper: Python Package Management Guide for Enterprise Developers
