How to Optimize Perl CI/CD Pipelines for Github Actions

Github Actions provides a good way for GitHub users to implement Continuous Integration/Continuous Delivery (CI/CD) for their GitHub repositories. Unfortunately, when it comes to CI/CD, Perl is rarely treated as a first class citizen. The fact is that most large Perl applications were written long before CI/CD became a best practice, but it’s just as important for speeding up code delivery in Perl applications today. As a result, most popular CI/CD products have features that support setting up environments to execute Perl code, but usually it’s not as “out of the box” as other more popular languages like Python (ie., you might not find Perl examples in tutorials or help documentation).

GitHub Actions is no exception. However, Github Actions is one of the more fully-featured CI/CD products on the market, and has an attractive policy (free for public projects) for open source developers in particular. 

If you’re planning on using GitHub Actions for your Perl CI/CD solution, or are interested in what it has to offer, this blog post can help you overcome the idiosyncrasies that may be acting as a barrier to adoption. In addition, it provides instructions on how to use GitHub Actions in conjunction with the ActiveState Platform to take care of the dependency management and runtime environment issues that usually makes dealing with CI more difficult.

For a more vanilla use case of setting up GitHub Actions for Perl projects, you can check out Olaf Alders’ presentation at The Perl Conference 2020, Getting Started with GitHub Actions.

Getting Started with CI/CD and ActiveState Platform

First, we’ll need a Perl application for which I’ve chosen the command line Perl application, ExifTool (https://exiftool.org/; original source code can be found at https://github.com/exiftool/exiftool). I’ve already forked the original source on GitHub and added the necessary files to support GitHub Actions. 

So:

  • Github will host our code
  • ActiveState Platform will host a pre-built version of our Perl environment, which includes a version of Perl (5.28 in this case) plus all the modules and dependencies the project requires
  • The ActiveState Platform’s Command Line Interface (CLI), the State Tool will pull the pre-built Perl environment into GitHub Actions in a programmatic way, ensuring consistency and speedier builds.

The last remaining step will be to build and run the sample project’s tests for a successful round of development iteration. 

First things first:

  1. Sign up for a free ActiveState Platform account.
  2. Install the State Tool on Windows:
IEX(New-Object Net.WebClient).downloadString('https://platform.activestate.com/dl/cli/install.ps1') 

or install the State Tool On Linux:

sh <(curl -q https://platform.activestate.com/dl/cli/install.sh) 

3. Check out the runtime environment for this project located on the ActiveState Platform.

4. Check out the project’s code base hosted on Github.
Note that the source code project and the ActiveState Platform project are integrated using an activestate.yaml file located at the root folder of the Github code base. You can refer to instructions on how to create this file as well as how it works by reading the ActiveState Platform documentation.

All set? Let’s dive into the details.

Setting up GitHub Actions for Perl Project

For this example, I’ll use a matrix build (Windows and Linux) on a cloud-hosted VM. Note that this is basically the exact same configuration I used for running Python CI/CD pipelines with GitHub Actions, just with some very minor changes involving ExifTool code-specific issues. The fact that you can use the same configuration no matter the language is an artifact of how the ActiveState Platform is designed. 

On Github:

  1. Sign in to your GitHub account. 
  2. Go to my exiftool_cicd project and click the Fork button to fork it into your account.
  3. Click the Actions tab, and then the “Set up a workflow yourself” button:

This will create a new default .yaml file under your project’s /.github/workflows/ folder. 

If you’re on a paid plan, you may see a warning  because of potential cost consequences. Just click the “I understand…” button to continue.

Note that you won’t see a workflow created immediately because the original perlapp.yml file you forked still exists in the /.github/workflows/ folder, and is configured to trigger on push. Let’s take a look at it:.

# This is a basic workflow to help you get started with GitHub CI using ActivePerl
name: exiftool with ActivePerl on GitHub CI

# Setting up Cache directory and ActiveState Platform API key
env:
  ACTIVESTATE_CLI_CACHEDIR: ${{ github.workspace }}/.cache        
  
# Controls when the action will run. Triggers the workflow on push events on the default branch 
on: [push]

# A CI workflow  is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on (this one is a matrix build)
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        # Building on both Windows and Linux(Ubuntu) simultaneously
        os: [windows-latest, ubuntu-latest] 
    steps:
    # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
    - uses: actions/checkout@v2
    # Installing State Tool on Windows via Powershell 
    - name: Install State Tool (Windows)
      if: matrix.os == 'windows-latest'
      run: |
        (New-Object Net.WebClient).DownloadFile('https://platform.activestate.com/dl/cli/install.ps1', 'install.ps1'); 
        Invoke-Expression -Command "$Env:GITHUB_WORKSPACE\install.ps1 -n -t $Env:GITHUB_WORKSPACE"
        echo "::add-path::$Env:GITHUB_WORKSPACE"
    # Installing State Tool on Linux with default shell behavior
    - name: Install State Tool (Linux)            
      if: matrix.os != 'windows-latest'      
      run: sh <(curl -q https://platform.activestate.com/dl/cli/install.sh) -n
    # Checking ActiveState Platform for project updates
    - name: Update project
      run: state pull
    # Caching downloaded build using GitHub CI cache
    - name: Cache state tool cache
      uses: actions/cache@v1
      env:
        cache-name: cache-platform-build
      with:
        path: ${{ env.ACTIVESTATE_CLI_CACHEDIR }}
        key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('activestate.yaml') }}
        restore-keys: |
          ${{ runner.os }}-build-${{ env.cache-name }}
    # Running project tests 
    - name: Test with prove
      run: |
        perl -v
        state run tests
    # Execute linting of the project, and continue even if linting fails
    - name: Lint with perlcritic
      continue-on-error: true
      run: state run lints

This YAML file might look a bit complicated, but that’s because:

  • It supports two different operating systems (a.k.a. a matrix build)
  • It takes advantage of caching to optimize workflow execution speed.
  • The installation script for Windows is different then the one on Linux, and needs to be executed slightly differently

In summary, the example perlapp.yml file above basically:

  1. Sets up the build environment
  2. Downloads the source code
  3. Installs the State Tool
  4. Checks for project updates
  5. Sets up/updates the cache
  6. Executes the test and lint scripts

The Perl environment will be downloaded from the ActiveState platform automatically by the test script when changes are detected in the project (which will update activestate.yaml). GitHub CI will also update the cache when changes are detected in activestate.yaml, so until the next time your Perl environment changes, everything will be reused from cache.

This setup should work for most Perl projects with slight modifications. The YAML elements that could be tweaked include:

  • The name: of the application to match your application
  • The on: action to modify the triggers running the build (including setting the branch)
  • The operating systems in the  matrix: strategy based on your target OS(es)
  • The last two steps (lint and test) to match the specific type of scripts you might want to run

For more information on modifying workflows, please refer to the documentation on GitHub

Note that the linting and testing scripts are State Tool scripts. These scripts are executed in the virtual environment that is set up by the State Tool when it pulls the Perl environment from the ActiveState Platform. In order to make sure the scripts are executed in the virtual environment using the version and dependencies you selected in your ActiveState Platform runtime, you should create your testing scripts in the activestate.yaml file.

Let’s have a quick look at activestate.yaml, which is also located in the root folder of your exiftool_cicd repo:

project: https://platform.activestate.com/ActiveState-Labs/exiftool
languages:
- name: perl
scripts:
  - name: tests
    description: Executes tests
    value: | 
      perl -v
      prove -l
  - name: lints
    constraints: 
      os: windows 
    language: batch
    description: Executes linting tools
    value: perlcritic windows_exiftool & perlcritic lib\
  - name: lints
    constraints: 
      os: macos,linux 
    language: bash
    description: Executes linting tools
    value: perlcritic exiftool & perlcritic lib/
  • The first line (project) is the link that ties the ActiveState Platform to your project. State Tool uses this project URL to download the Perl runtime environment (a version of Perl + required modules + all dependencies) configured for this project.
    • If you use your browser to access the URL you’ll see that the ActiveState Platform project contains the requirements defined by the Perl metadata information found in the exiftool_cicd project. 
  • Under the languages tag, the language of this project is defined as Perl. This is because the State Tool is not only cross-platform but also cross-language, as well (and soon to be multi-language). 
  • Under the scripts tag are scripts run by the GitHub Action CI workflow using our state run command. Each script can have a name and description, as well as single or multi-line scripts to be run. They can also be optionally constrained to specific operating systems and language definitions, such as the actual languages configured in the runtime environment, as well as other scripting languages provided by shells available on the operating system. 
    • The script named tests triggers the download and installation of the Perl environment automatically (this process is known as “activation”)
    • The script named lints reuses the “activated” environment to run perlcritic on OS-specifc versions of the application and library files.

For more information on creating scripts with the State Tool, please refer to the documentation.

If you build for multiple OS’s, one of the key advantages of the ActiveState approach is it ensures the same Perl environment is used for each test irrespective of the OS. By default, GitHub Actions uses  different versions of Perl on different OS platforms, which is something you would have to account for and correct.

Running a Perl Build with GitHub Actions

Once you’re done editing the Github Actions YAML file, you can now start a run.

  1. Click the Start Commit button on the top right and commit the new file to GitHub. This action should trigger your first build.

  2. In order to see the build results, click on the Actions tab:
    If everything went well, you should see a successful build (with a green checkmark) in the Workflow list. If there was an issue, you might see failed builds (with a red X), as well.

  3. To see the details of a particular run, click on the name of the workflow:
    You will see all builds that executed on the left side. In our example, we had 2 jobs (one on Windows one on Linux), and both passed with Green checkmarks.

  4. If you click on the build name, you can see build logs on the right side:
    Here you can see the steps executed during the build, along with their status (passed, failed, or skipped) and their execution times. If you click the triangles to the left of the names, you can see the actual output from the execution scripts (including State Tool and Perl output).

    You might notice that there were “annotations” on the previous screenshot, and we set a continue-on-error: true value in the linting step of the workflow .yaml file. This is because perlcritic reports a number of issues and returns an error code, so I’m overriding that failure to be able to show a completed build. Running perlcritic on older codebases that haven’t been developed with it in mind will often report too many issues to fix all at once.

Conclusions: Optimizing CI/CD Pipelines for Perl

Github Actions is a suitable CI/CD solution for Perl applications, but it’s just one of many on the market. Ultimately, it’s your application’s needs that will determine which CI/CD solution is best for your Perl projects. 

One of the key advantages of using ActiveState Platform to manage your Perl environment is portability. Once the ActiveState tooling is set up, you don’t need to worry about the issues/differences between how the platforms work, or how they set up Perl environments (which always differs from how you would do it on your desktop). And no matter which CI/CD solution you use, the ActiveState Platform can help guarantee that the Perl environments for your projects are consistent for every run, no matter when it’s run (ie., dependencies will not break) or for which operating systems. Taking one more variable out of the CI/CD equation will simplify your life and give you back time to do what you enjoy (hint: it’s not fighting with CI).

In summary, the ActiveState Platform, in conjunction with the State Tool, simplifies development workflow and CI/CD setup by providing a consistent, reproducible Perl environment deployable with a single command. Having a simpler CI/CD setup helps reduce training and maintenance time for engineers responsible for setting up the CI/CD environments. Additionally, since the ActiveState Platform builds all modules and dependencies from vetted source code, binary artifacts can be traced back to their original source, helping to resolve the build provenance issue.

In these ways, the ActiveState Platform effectively eliminates the “works on my machine” problem, and simplifies the setup of a more secure, consistent, up-to-date CI/CD pipeline.

If you’d like to try it out, sign up for a free ActiveState Platform account where you can create your own runtime environment and download the State Tool.

Find more CI/CD resources from ActiveState here!

Related Blogs:

Optimizing CI/CD Pipelines in GitHub Actions

CI/CD Tools & Implementations for DevOps

How to Set Up CI/CD for Python Builds on GitLab

Aleks Pamir

Aleks Pamir

Senior Product Manager. Aleks has worked in multiple capacities from hands-on software development to various management leadership positions for over 25 years. With experience in a wide range of technologies including networking, embedded, wireless, mobile, games, operating systems, development tools, database, location, speech and an interest in Machine Learning, he loves finding creative solutions to hard problems at the intersection of technical and business challenges.