This repository contains the mirsg.infrastructure Ansible Collection. This collection can be used to configure infrastructure for deploying XNAT and OMERO.

Before using this collection and its playbooks, you must install the necessary Ansible collections and roles.

This collection can be installed using the ansible-galaxy command-line tool:

ansible-galaxy collection install https://github.com/UCL-MIRSG/ansible-collection-infra.git

It can also be included in a requirements.yml file and install it via ansible-galaxy collection install -r requirements.yml using the format:

collections:
  - name: mirsg.infrastructure
    source: https://github.com/UCL-MIRSG/ansible-collection-infra.git
    type: git
    version: main

N.B. Ideally, changes to the infrastructure should be made via workflows. However, in some cases it may be necessary to run the playbooks manually. For example, for upgrading the postgresql version on the ucl_test server:

ansible-playbook mirsg.infrastructure.upgrade_postgresql -u <user> --ask-become-pass -l ucl_test -i environments/cs-production/xnat/hosts.yml --vault-password-file ~/.ucl_xnat-vault_password

The upgrade_postgresql.yml playbook prompts for the version to upgrade from and to upgrade to.

The following variable needs to be set: postgresql_upgrade_data_dir: “/var/lib/pgsql/{{ postgreql_upgrade_new_version }}/data”

This is done in the group vars for the server you're updating i.e. ucl_test

Ansible Molecule and its Docker plugin is used to test the roles and playbooks in this collection.

To run the tests locally you will need to:

• clone this repository
• install Ansible Molecule and other test requirements
• install and start the Docker Engine or Docker Desktop on MacOS, if you have not done so already
• if necessary (i.e. you are running Docker on Linux), add your user to the docker group in order to access the Docker Engine
• run the tests using Molecule

To test a collection, Molecule requires that it is in the path ansible_collections/<namespace>/<collection name>. This means when you clone this repository you must ensure it is in the path ansible_collections/mirsg/infrastructure. The simplest way to do this is:

git clone git@github.com:UCL-MIRSG/ansible-collection-infra.git ansible_collections/mirsg/infrastructure

Before running the tests install Molecule, the Docker plugin, and the Python Docker Engine API using pip:

python -m pip install molecule 'molecule-plugins[docker]' docker

Each role has its own Molecule configuration, which can be found it the molecule/ folder of each role. Molecule base configurations are used to reduce to amount of duplication in the setup for testing each role. There are two base configurations that correspond to two Molecule scenarios one for testing on CentOS 7 and another for testing on RockyLinux 9. The base configurations are in the ansible_collections/mirsg/infrastructure/molecule_configs folder.

To run the tests for a specific role, first navigate the the role directory, e.g.

cd ansible_collections/mirsg/infrastructure/roles/provision

Then invoke Molecule with a base config and corresponding scenario. To run the tests on CentOS 7:

molecule --base-config ../../molecule_configs/centos7_base_config.yml test --scenario centos7

This command will:

• install the required Ansible roles and collections
• create a CentOS 7 container
• run the molecule/prepare.yml playbook (if it exists) to do any required setup for the role
• run the molecule/converge.yml playbook, which will run the role being tested
• run molecule/converge.yml a second time to check the role is idempotent
• run molecule/verify.yml playbook (if it exists) to perform verification
• destroy the CentOS 7 container

To run the tests on Rocky 9 instead:

molecule --base-config ../../molecule_configs/rocky9_base_config.yml test --scenario rocky9

If the molecule test command fails at any stage, the container is immediately destroyed. This is due to the pre-defined sequence of actions the Molecule take when the molecule test command is invoked.

If you would like to be able to access the test container, you should instead use the molecule converge command. To run this on CentOS 7:

molecule --base-config ../../molecule_configs/centos7_base_config.yml converge --scenario centos7

This will install necessary Ansible roles and collections, create the test container, and run the molecule playbooks. If the deployment fails, the container is not destroyed.

Once the command has finished running, you can access the container using the name of the scenario. To access the container for the centos7 scenario:

molecule --base-config ../../molecule_configs/centos7_base_config.yml login --scenario centos7

If testing a role or playbook where Molecule creates multiple containers, individual hosts can be accessed using the --host flag:

molecule --base-config ../../molecule_configs/centos7_base_config.yml login --scenario centos7_monitoring --host mserv

If you use the molecule converge command, you must remember to destroy the container, network, and volumes yourself. You can do this using the molecule destroy command:

molecule --base-config ../../molecule_configs/centos7_base_config.yml  destroy --scenario centos7

Playbooks in the collection can also be tested using Molecule. The Molecule configuration for playbooks is in the ansible_collections/mirsg/infrastructure/playbooks/molecule folder.

An example of how to setup testing for a playbook can be seen by looking at the tests for the mirsg.install_monitoring playbook in this collection. This is tested on CentOS 7 and RockyLinux 9 using the centos7_monitoring and rocky9_monitoring scenarios.

An inventory and associated group variables can be found in playbooks/molecule/resources/monitoring/inventory. Testing the playbook also requires its own converge.yml, and optional prepare.yml and verify.yml playbooks. Running the tests then proceeds as with testing the roles:

molecule --base-config ../molecule_configs/centos7_base_config.yml test --scenario centos7_monitoring

When a PR that modifies a role or playbook is opened, the changes are tested by deploying that role using GitHub Actions. See the molecule-firewalld workflow for an example.

This repo has pre-commit hooks enabled, for instructions see https://github.com/UCL-MIRSG/.github/tree/main/precommit.

This collection is licensed and distributed under the BSD 3-Clause License.

This collection was created by the Medical Imaging Research Software Group at UCL.