This directory contains pipeline definitions and scripts for running Elasticsearch CI on Buildkite.

• pipelines - pipeline definitions/yml
• scripts - scripts used by pipelines, inside steps
• hooks - Buildkite hooks, where global env vars and secrets are set

Pipelines are defined using YAML files residing in pipelines. These are mostly static definitions that are used as-is, but there are a few dynamically-generated exceptions (see below).

Pull request pipelines are generated dynamically based on labels, files changed, and other properties of pull requests.

Non-pull request pipelines that include BWC version matrices must also be generated whenever the list of BWC versions is updated.

Pull request pipelines are generated dynamically at CI time based on numerous properties of the pull request. See scripts/pull-request for details.

For pipelines that include BWC version matrices, you will see one or more template files (e.g. periodic.template.yml) and a corresponding generated file (e.g. periodic.yml). The generated file is the one that is actually used by Buildkite.

These files are updated by running:

./gradlew updateCIBwcVersions

This also runs automatically during release procedures.

You should always make changes to the template files, and run the above command to update the generated files.

Node (technically bun), TypeScript, and related files are currently used to generate pipelines for pull request CI. See scripts/pull-request for details.