Laboratory Listener

Title GitLab Listener
Span 04/2020 – 06/2020
Kind Development
Format Application
Role Original Concept, Developer
Context Server-side daemonised CI/CD utility

An autonomous daemon process for web servers, connecting with GitLab webhooks, allowing custom actions to be bound to configurable events. Accepts and parses GitLab events, allows filtering by various properties and can execute any command or script as a reaction. Built with Perfect, written in Swift.

Public Presence: gitlab.com/apricum/gitlab-listener
Releases: gitlab.com/apricum/gitlab-listener/-/releases

Introduction

The world of the automated internet is an intricate one; a world filled with many processes, wanton memory use, monolithic and microscopic systems alike, applications in isolation and communication, chaotic shell scripts and orderly instruction files, an environment telling countless tales of high efforts and low yields, of overcomplication and underperforming, unwavering stability and constant and untraceable failures, a set-up running on a Mac Mini half concealed under the desk or cross-country distributed data centres.

My personal upshot from this introduction is: Automation should be leveraged whenever possible. Once any process is sufficiently digital, it should allow for it to be automated. Even though many frameworks for automating repetitive, manual tasks are widely adopted in web development at the time of writing, finding solutions to issues at a smaller scale can be a challenge. I found that, more often than not, popular choices are intricate, require preparation, much reading of documentation with a common lack of examples, finding references, configuring, and testing. Different approaches later demand frequent maintenance or payment in the case of subscription services. The popular choice might often be an advanced choice, too, and sometimes, a simpler approach fits best.

Listener is a pragmatic approach to a straightforward task: coupling an action to a GitLab event. It listens for events sent as GitLab’s webhooks and, if it receives one, can execute an action, for instance, a command, a script, or another process.

Printout of the "--help" option of GitLab Listener

Motivation

My motivation for Listener arose from trying to find a small-scale solution for autonomous and self-contained deployment. It allows even tiny, single cloud instances to have a fully automated release set-up without the need for external services like a build or testing server. Even less complex web applications or static sites (like a personal homepage, an online portfolio or a game) benefit from automated releases. Without it, for every finished new version, some manual action was required for a release to be prepared and published. With a bare bones project, I would have to log into the cloud instance, shut everything down, pull in new changes, and restart; in a more sophisticated one, I would still need to kick off an Ansible playbook taking care of all the individual steps. If something could launch that script for me, I would not need anything more.

GitLab Listener running as a daemon in "systemd"

The following is just an example put together from a use case very similar to how I'm now using Listener in private servers for various web projects, each one autonomous:

We might have a custom web application, now ready and deployed on a remote server. Once new changes are merged into the repositories' remote master branch, the server should update its local files accordingly and initiate any reloading necessary for the new changes to be presented. We can use Listener in this scenario to trigger the update and reload once a change to master is made through GitLab; in this case, we would specify a command with the kind push and restrict it to the ref refs/heads/master. The command we configure can be anything that can be entered into a shell; a string of commands, the path to a script file with further functionality or a completely separate application.

While Listener reacts to GitLab events dispatched on changes in an accessible repository via webhooks, it can also react to simple custom actions which can be called at any time from the outside. By extension, these user instructions could even be used as arbitrary webhook targets for other services or applications. The endpoint for all user instructions has the format /user/<tag> endpoint, consisting of the base /user and a custom-definable user tag. This tag can be set in Listener's configuration and is set to identify and distinguish available actions.

In our example, we might have a CMS or a database holding the data presented by the deployed web application. An administrator or editor saves changes somewhere in an environment they're already aquainted with, they save and upload their changes to a remote repository. A user instruction could then be defined for an endpoint /user/update-database and an associated action to fetch the new state of the CMS or database. A secret URL can be distributed to content editors to call at any time to update the data themselves. If the CMS or editing environment supports it, the entire procedure could even be set to call through automatically whenever new changes are saved.

Conclusion

Release for version 0.4.0 with binaries included

This write-up aims to give only a first introduction to the original setting, the considered landscape of comparable approaches and the solutions provided by GitLab Listener in its initial release. Additional information, usage examples and documentation on topics like installing it on server instances and providing configuration for individual commands can be found with the mark-up included in Listener’s repository at gitlab.com/apricum/gitlab-listener.