- An open source developer can delete or corrupt a key dependency in the public repository, breaking your build.
- A forced dependency upgrade suddenly results in a corrupted environment.
- You upgrade your language version, and suddenly get “can’t find package” errors for the libraries you were just working with.
- Your boss asks you to help debug an older version of the product, but you can’t even reproduce the environment since the transitive dependencies have shifted.
- <add your own favorite reason here>
Without a robust strategy (and the accompanying processes) in place to closely manage environments, unreliable runtimes can lead to a number of common problems, including:
- “Works on my machine” issues
- Environment reproducibility issues
- “Can’t recreate the environment” issues
- Corrupted environments
- And so on
Of course, as developers, we don’t tend to make it easy on ourselves since we’re always importing/upgrading third-party packages and tools to test out or help solve development issues. And trying to get back to a stable state can be problematic. While most organizations write their own automation scripts to help set up, tear down, recreate and/or update environments (especially environments built-in containers), they often lead to a proliferation of tools, scripts, and Readmes that may or may not be well maintained in fast-moving development environments.
This blog post is designed to help you recognize the patterns that lead to unstable environments, and suggest some tools and processes you can use to increase the reliability of your environments, or automatically recover them when they do become unstable.
Global vs Virtual Environments
Globally installing an open source language has long been the standard way developers have created their development environment for decades. Global installs allow for a single version of a language to be installed on the local system, along with a single directory to accommodate all the dependencies your projects require. Unfortunately, global installations have been known to cause a number of common issues, including:
- Dependency Conflicts – because global installs create a single directory for all the dependencies you use in each of your projects, it can result in dependency conflicts when two different projects require the same dependency, but different versions.
- Corrupted Environments – updating your open source language to a newer version all too often results in “not found” errors, symlink issues, and other problems that force you to manually update directories and PATH variables.
- Multiple Package Managers – when the package manager for a proprietary ecosystem (eg, Conda) doesn’t support a package your project requires, you’ll need to install it with the ecosystem’s standard package manager (eg, pip). This can result in packages not being found, binary incompatibilities, and more.
One way around these kinds of problems is to use a virtual environment instead. Virtual environments contain a version of the programming language and all the dependencies your project requires. In this way, you can:
- Isolate each of your project’s dependencies, resulting in less dependency conflicts.
- Install multiple versions of a programming language, and switch between them, simplifying working on multiple versions of a software application.
- Better manage, optimize and audit the dependencies for each of your projects.
While some open source ecosystems have made virtual environments a first class citizen, others have not, which means you’ll need to adopt third party tools to help you create and manage your virtual environments. But the benefits far outweigh the ramp up time it takes to get started, especially if you work on multiple projects.
Native Virtual Environments Improve Stability
The ActiveState Platform contains its own catalog of open source components and language versions from which it can automatically build the runtime environment for your Windows, Mac or Linux-based project, on-demand.
Once built, the Platform installs the open source language and dependencies for your project into a virtual environment, by default. In this way, you can ensure that each of your projects (and their dependencies) are isolated and won’t step on each other. Moreover, all the changes you make to any of your virtual environments are saved as a Git-like commit, which can always be restored. This means:
- There’s an audit trail of all changes to the runtime environment of your application from v.1 through v.latest.
- You can always restore the environment at any point in time, even years later. Never be in a “can’t recreate the build” situation again.
- Time wasted trying to recover corrupted environments amounts to nothing more than the few minutes it takes to revert to the latest commit.
See how it works:
The ActiveState Platform can also create and restore branches of your commits, and help you implement reproducible environments across dev, test and production.
Next steps:
The capabilities shown here are also available as a managed service, freeing up your developers to focus on coding and getting your product to market faster. Learn more about our Managed Builds service.
