Summary: create two bare metal servers to deploy Windows and Mac OS runners for GitLab CI, using libvirt.
Background
Normally, we try to limit the number of tools we use inside TPA (see TPA-RFC-3: tools). We are currently phasing out the use of libvirt in favor of Ganeti, so new virtual machines deployments should normally use Ganeti on all new services.
GitLab CI (Continuous Integration) is currently at the testing stages on our GitLab deployment. We have Docker-based "shared" runners provided by the F-Droid community which can be used by projects on GitLab, but those only provide a Linux environment. Those environments are used by various teams, but for Windows and Mac OS builds, commercial services are used instead. By the end of 2020, those services will either require payment (Travis CI) or are extremely slow (Appveyor) and so won't be usable anymore.
Travis CI, in particular, has deployed a new "points" system that basically allows teams to run at most 4 builds per month, which is really not practical and therefore breaks MacOS builds for tor. Appveyor is hard to configure, slow and is a third party we would like to avoid.
Proposal
GitLab CI provides a custom executor which allows operators to run arbitrary commands to setup the build environment. @ahf figured out a way to use libvirt to deploy Mac OS and Windows virtual machines on the fly.
The proposal is therefore to build two (bare metal) machines (in the
Cymru cluster) to manage those runners. The machines would grant the
GitLab runner (and also ahf) access to the libvirt environment
(through a role user).
ahf would be responsible for creating the base image and deploying the first machine, documenting every step of the way in the TPA wiki. The second machine would be built with Puppet, using those instructions, so that the first machine can be rebuilt or replaced. Once the second machine is built, the first machine should be destroyed and rebuilt, unless we are absolutely confident the machines are identical.
Scope
The use of libvirt is still discouraged by TPA, in order to avoid the cognitive load of learning multiple virtualization environments. We would rather see a Ganeti-based custom executor, but it is considered to be too time-prohibitive to implement this at the current stage, considering the Travis CI changes are going live at the end of December.
This should not grant @ahf root access to the servers, but, as per TPA-RFC-7: root access, this might be considered, if absolutely necessary.
Deadline
Given the current time constraints, this proposal will be adopted urgently, by Monday December 7th.