If you’re like most enterprises, you’ve got teams working on multiple applications that share much the same technology stack.
Let’s take Python as an example. You might have teams working on web-based applications that incorporate Python’s web packages (like Django or Flask). But these days you’re also likely to have other teams working with Python on a data science project involving numpy, scipy and TensorFlow.
If you’re running a modern development process, the output of these teams is likely getting deployed as docker-ized instances of standalone, discrete services. All best practices to ensure immutability. But the rest of us are still deploying traditional applications.
As your applications proliferate, it means a lot of 1:1 deployments: 1 application to 1 server. And unless the team that made the application is also managing it in production as well, your IT/Ops teams are getting overloaded with the number of servers to instrument, monitor and manage.
Creating a Consistent Runtime Environment
One idea to help get this proliferation under control is to find a common denominator – a common runtime environment – that can support all your applications. In this way, you’ll be able to create a single server that can run all of your applications, and then just fine tune the resources the server requires.
Bloated Runtimes
The easiest (and most popular) way to do this is to source a common, “bloated” runtime. I call them bloated because they contain far more packages than are required to support all your applications. In the Python world, for example, you might download Anaconda or ActiveState’s Python, ActivePython, both of which come with hundreds of pre-compiled and pre-verified popular Python packages. Of course, for any one project you’ll probably end up using less than half of the included packages.
The drawbacks to this approach include:
- Increased Attack Surface – larger runtimes expose more points of entry for bad actors looking to exploit poor quality/vulnerable code.
- False Positives – over time, vulnerabilities will get reported against packages that are included in the production runtime. But many of these packages aren’t actually being used by the application so they won’t be prioritized for fixing, thereby increasing risk.
Custom Runtimes
The other, more painstaking approach is to build up a common runtime comprised of only those packages your applications need. So for our Python example, you’d first need to determine how many versions of Python are required, and which platforms they support. Then, for each application on the same version of Python/platform, you would:
- Check all the Requirements.txt/ Pipfile.lock to create a single list of all packages and dependencies that are being used in production
- Verify your list with each team to determine:
- Which packages were actively being coded against, but the feature didn’t actually make the deadline for inclusion in the current release
- Which packages are leftover artifacts from deprecated code
- Which packages aren’t actually being called in production for some other reason (i.e., test harnesses)
While this method will result in a smaller attack surface and less false positives compared to the “bloated runtime” approach, it may entail significant time and resources depending on the number of applications and teams in your enterprise.
The New Way for Consistent Runtimes
The key to solving the consistent runtime problem is being able to determine which packages are actually being loaded and used by production applications. This is where the ActiveState Platform can help by providing an interpreter plugin that allows you to instrument all your Python applications.
The plugin pulls data from each package as it gets loaded (data such as package name; version; license, etc) and sends it to the ActiveState Platform where anyone can log in and get a Bill of Materials (BoM) view for each application, such as that shown below:
More importantly, the ActiveState Platform also provides a rollup view of all of the packages used by all “identities” (aka applications):
Armed with this information, you can use the ActiveState Platform’s build functionality to create a common runtime environment by selecting the Python version (eg., 3.6.6), platform to deploy onto (eg., RHEL 6), and the packages listed above (being sure to choose a later version of bleach, numpy and mistune that don’t have vulnerabilities logged against them):
All of the dependencies for the 69 packages in our “Common-Runtime” project are automatically pulled in and resolved by the ActiveState Platform’s resolver function. You can click the View Status button to see how the build is progressing, or else log back in about an hour to pick up your runtime environment, which will be packaged as a tarball for easy distribution.
And that’s it! In a little more than an hour, you can create a common runtime environment for all your Python applications and be assured that you have:
- All the packages actually used by the applications running in production
- Created the smallest runtime environment that satisfies the requirements for all applications and, as an added bonus:
- Identified and resolved any packages that have vulnerabilities logged against them
ActiveState Platform for Runtime Management
The ActiveState Platform is currently available for you to try out for your Python or Perl needs:
- If you’d like to try out building a common runtime environment for your Python or Perl applications, you can create a free account on the ActiveState Platform
- If you’d like to see how you can instrument your Python applications and create a BoM view, contact us for a demo
