I’d like to give you a sneak peek into an exciting initiative we are working on with our ActiveState Platform. We’re adding the ability for developers to not only expand our Platform, but also take advantage of the new capabilities added by others. Read on to find out how we’re using this new feature to add Ruby and it’s ecosystem to the ActiveState Platform as a proof of concept. Our hope is to inspire other language communities to do the same.
The ActiveState Platform provides a build engineering software service that developers can use on-demand to automatically build a custom, isolated runtime from source (i.e. including just the language, all it’s native C-library dependencies for a specific operating system, and packages required by their project). Today, the ActiveState Platform catalog includes tens of thousands of Python, Perl and Tcl packages, many of which have ActiveState-created and maintained recipes that ensure they can be built for multiple platforms, and packaged for deployment on most popular operating systems.
Multi-Platform Multi-Language Builds
The ActiveState Platform exemplifies an emerging platform category that supports multi-platform, multi-language builds so that organizations don’t have to create and manage their own build infrastructure, which is often fragile and expensive to maintain.
In order to do all this, we’re using our experience in building open source languages garnered over the past 20 years to expose and augment our build engineering tools, and make them available to everyone. Because of the vast landscape of tools, languages, operating systems, packages and supporting libraries that this requires, we wanted to create a way for anyone to contribute their additions to the ActiveState Platform. The goal is to enable developers to help themselves — and help each other — using our free build infrastructure.
Enter the Build Graph, which is a graph-based API that provides access to our Platform for contributors so they can:
- Create a list of requirements for a package.
- Resolve the requirements to a Build Plan that includes a complete list of transitive dependencies, as well as native operating system dependencies (OpenSSL, libXML etc..) from our open source catalog.
- Build and retrieve the result of the Build Plan on various operating systems and hardware platforms, including Windows, Linux and macOS.
And now the exciting part: you’ll soon be able to extend the open source catalog yourself, and/or leverage the work of others that have helped extend it.
GraphQL APIs
For those not familiar with GraphQL APIs, they’re quickly becoming the replacement for RESTful APIs at companies like AWS, Yelp, GitHub, Facebook, Shopify, IBM, Twitter, and many more. GraphQL APIs offer a number of advantages over the REST APIs you’ve probably used, including:
- Ability to execute complex queries with a single call, resulting in less network overhead and faster development
- Essentially provides SQL for APIs, which makes it both simpler to expose API based on databases on the backend, and easier to consume on the front end
- Self-documenting, thereby eliminating out of sync API documentation
- Maps very well to the build graph domain model, thereby allowing easier manipulation of information in the form of dynamic acyclic graphs (DAG)
Here’s a quick example to show off the general idea and the power of expressing queries in with our new Build Graph. Let’s say we want to build TensorFlow 2.1 for Python on Windows. In order to do that, we’ll first need to determine whether the ActiveState Platform has all the required dependencies. You would write the following request asking for TensorFlow version 2.1:
query {
timestamp
artifacts(
attime="2020-05-05T19:16:22.760869",
platform="win64",
requirements=[
"language/python/tensorflow-2.1"
]
){
name
status
revision
version
urls
}
}
The response shows that someone has already built TensorFlow 2.1, and provides the following information about the build:
- Name of all transitive dependencies
- Build status for each artifact
- Version of each artifact
- URL to retrieve the built artifact, including the final TensorFlow 2.1 package
{
"data" : {
"timestamp": ”2020-05-05T120:14:12.147312”,
"artifacts" : [{
"name" :"python/kiwisolver",
"status": "BUILT",
"revision": 0,
"version": "1.1.0",
"urls": ["http://www.activestate.com/storage/AAA-83838-03030-83838-python-kiwisolver/kiwisolver.zip"]
},
{
"name" :"python/keras-application",
"status": "BUILT",
"revision": 0,
"version": "1.0.3",
"urls": ["http://www.activestate.com/storage/BBB-83838-03030-83838-python/keras-application/keras-application.zip"]
},
{
"name" :"python/tensorboard",
"status": "BUILT",
"revision": 0,
"version": "2.1.0",
"urls": ["http://www.activestate.com/storage/CCC-83838-03030-83838-python/tensorboard/tensorboard.zip"]
},
{
"name" :"python/tensorflow",
"status": "BUILT",
"revision": 0,
"version": "2.1.0",
"urls": ["http://www.activestate.com/storage/DDD-83838-03030-83838-python/tensorflow/tensorflow.zip"]
},
*** 52 other dependencies removed for brevity ***
]
}
}
GraphQL for the ActiveState Platform
With our Build Graph, using only GraphQL queries, ActiveState Platform contributors can define and compose steps for building complex open source software for any operating system and language without having to worry about sourcing, maintaining, and updating their own infrastructure. Instead, you can take advantage of the ActiveState Platform’s build infrastructure, which includes:
- An innovative solver for dependency resolution.
- The ability to parallelize steps for quicker builds.
- Caching of build artifacts.
- Automated packaging for Windows, macOS, and Linux.
All combined with a command line mechanism for delivery/update of the final package.
The key to this initiative is our GraphQL API, which we will be using to add the Ruby ecosystem to the ActiveState Platform as a proof of concept. In a series of upcoming blog posts we’ll take everyone through the process, showing exactly how you can add a new open source language ecosystem to the platform.
Access to the API is currently available for a limited number of contributors who want to help us test it out.
- Interested in getting involved? Join the conversation on the ActiveState Forums.
- Interested in checking out the ActiveState Platform’s current capabilities? Sign up for a free account.
Related Blogs:
An Open Letter to the Open Source Community From Bart Copeland, ActiveState CEO
Automatically Build Custom Perl and Python Runtimes for Windows
