The Benefits of a Single Toolchain for Managing Open Source Dependencies

Open source software is everywhere within modern codebases. According to the Linux Foundation, a full 98% of businesses rely on open source modules, libraries, plug-ins, and/or components within their codebases, even if the applications they build are not themselves open source.

Open source provides software vendors with distinct advantages, foremost among them being code reuse, which can dramatically speed up application development and foster innovation. But unless developers work with and manage open source code in an efficient way, it can present more issues than it helps to solve. For example:

• Security – importing prebuilt open source components from a public repository poses a security risk, since public repos offer no guarantees as to the security and integrity of the open source code they offer.
• Overhead – to manage security and compliance risks, open source must be subjected to multiple tools and processes in order to check whether it has been compromised, including vulnerability checking, static and dynamic testing, software license checks, etc. 
• Maintenance – unfortunately, the standard in the software industry is that 80% of codebases are rarely updated for fear of breaking the build, which leads to applications riddled with open source vulnerabilities.

If you’re not using signed open source components from a trusted vendor, “self vendoring,” in which the source code for all open source software is checked into your code repository, may be your next best option to help address the security, maintenance and overhead issues. 

Self-Vendoring Open Source Dependencies

Self-vendoring is often called the “least worst” way to manage open source dependencies, since it solves many common security, dependency management and compliance issues, but also introduces quite a bit of complexity, as well.

Simply put, self vendoring is the process of checking a single version of each open source dependency into a source control system rather than relying on a package manager to install dependencies into environments on demand. While such a process can help resolve dependency conflicts, ensure against broken builds and eliminate inconsistencies between builds, it also means that builds (and build environments) end up being far more complex than usual.

In practical terms, securely building dependencies (and all of their transitive dependencies) from source code for all of the major operating systems your developers/systems require means:

• Developing and maintaining extensive language and OS expertise.
• Creating and maintaining multiple build environments/tooling, one per language and OS.
• Implementing and maintaining multiple automated build pipelines that can create reproducible builds for all the languages your organization requires. 

Development teams that lack the skills and processes for building open source packages and integrating them into their codebases are likely to find that open source hinders their pace of development, while also burdening development teams with high tooling overhead.

However, using a single-toolchain strategy can dramatically reduce the drawbacks of self vendoring by allowing all teams, no matter what language or platform they work with, to centralize the creation of open source dependencies from source code without the tooling overhead normally required. 

The Advantages of a Single Open Source Toolchain

For any business that seeks to capitalize fully on the potential that open source offers for speeding development processes, having a single toolchain in place for building, sharing, and managing open source software across the organization is absolutely critical.

A single toolchain for all your open source build needs provides for:

• Reduced Overhead – managing multiple sets of build tools exponentially increases the effort required on the part of developers to package open source code and integrate it into their applications. It also greatly increases the risk that errors with build tools or operations will lead to a delay in application deployment. It’s much easier to run into problems when your build pipeline depends on a dozen different compilers and package managers than if you obtain all of your open source dependencies through a single, reliable channel.
• Greater Reusability – a single toolchain allows all developers across the entire organization to use the same open source dependencies. The result is much less effort on the part of the developers, as well as greater consistency with regard to open source across the organization.
• Simpler Automation – a single toolchain allows teams to use open source components that are built and packaged in a central repository, and then shared across the entire organization. When developers can pull all open source dependencies from a central location, it becomes much easier to automate the software delivery pipeline as fully as possible, and to minimize the risk of build delays that could disrupt continuous delivery.

Keep in mind that a single toolchain is not necessarily a one-size-fits-all solution. There are some corner cases where a single toolchain may not be appropriate. For example, if an application requires open source components to be built and packaged in unique ways it may make sense to build components within the pipeline according to their custom requirements rather than using a single-toolchain strategy.

Creating a Single Toolchain for Open Source

When it comes to creating a single toolchain for the open source your organization requires, you have some options. Generally, single toolchain solutions start with a universal package manager, such as: 

• ActiveState Platform – a cloud-based universal package manager that automatically builds open source language dependencies from source code, and then packages them for deployment on Windows, Mac and Linux systems.
  • Pros: focused on supply chain security
  • Cons: supports only Python, Perl, Ruby and Tcl at this point
• The Universal Package Manager (UPM) provides support for a wide range of OS’s and languages by calling the native package manager for each language/OS under the hood.
  • Pros: supports a wide range of languages with a single set of common commands
  • Cons: lacks a security focus

OS-Specific Approaches –  while the following solutions cannot support a single toolchain strategy since they do not support Windows- or Mac-based systems, they do support a wide range of Linux variants:

• NixOS – creates reproducible development environments that contain multiple open source languages and tools packaged for deployment in the cloud, as containers, and so on. 
  • Pros: declarative builds; supports a wide range of common development tools and languages
  • Cons: lacks a security focus
• Snap – created by Canonical/Ubuntu, snap provides a YAML-based way to create modular deployments that include a wide range of languages and frameworks.
  • Pros: creates consistent environments quickly and easily
  • Cons: lacks a security focus 
• Flatpak – a container-based solution that provides support for a wide range of open source applications and tools. 
  • Pros: creates consistent environments quickly and easily
  • Cons: lacks a security focus

Conclusions – Selecting a Universal Toolchain for Open Source

The good news is that you have a wide range of options when it comes to implementing a single toolchain for your open source needs. The bad news is that very few of them address the current elephant in the room: supply chain security. Attacks on the software supply chain have been growing at a rate of more than 700% year on year over the past three years, leading security conscious organizations like the US Government to impose supply chain security requirements on their vendors. And what starts as a government requirement soon becomes a requirement for commercial industries, as well.

Only the ActiveState Platform incorporates a secure build service that implements the Supply chain Levels for Software Artifacts (SLSA) framework to ensure the security and integrity of all open source components. It will also generate a software attestation verifying that all dependencies were automatically built from vetted source code (including native libraries), and that they were built in a secure manner. 

Read Similar Stories