• CI/CD
  • Table of Contents
  • Overview
  • Workflows
    • Code Style Check
    • Unit Test
    • Example Test
      • Example Test on Dispatch
    • Compatibility Test
      • Compatibility Test on Dispatch
    • Release
    • User Friendliness
    • Community
  • Configuration
  • Progress Log

Automation makes our development more efficient as the machine automatically run the pre-defined tasks for the contributors. This saves a lot of manual work and allow the developer to fully focus on the features and bug fixes. In Colossal-AI, we use GitHub Actions to automate a wide range of workflows to ensure the robustness of the software. In the section below, we will dive into the details of different workflows available.

Refer to this documentation on how to manually trigger a workflow. I will provide the details of each workflow below.

A PR which changes the version.txt is considered as a release PR in the following context.

To reduce the average time of the unit test on PR, Build on PR workflow manages testmon cache.

1. When creating a new branch, it copies cache/main/.testmondata* to cache/<branch>/.
2. When creating a new PR or change the base branch of a PR, it copies cache/<base_ref>/.testmondata* to cache/_pull/<pr_number>/.
3. When running unit tests for each PR, it restores testmon cache from cache/_pull/<pr_number>/. After the test, it stores the cache back to cache/_pull/<pr_number>/.
4. When a PR is closed, if it's merged, it copies cache/_pull/<pr_number>/.testmondata* to cache/<base_ref>/. Otherwise, it just removes cache/_pull/<pr_number>.
5. When a branch is deleted, it removes cache/<ref>.

This workflow is triggered by manually dispatching the workflow. It has the following input parameters:

• example_directory: the example directory to test. Multiple directories are supported and must be separated by comma. For example, language/gpt, images/vit. Simply input language or simply gpt does not work.

This workflow is triggered by manually dispatching the workflow. It has the following input parameters:

• torch version:torch version to test against, multiple versions are supported but must be separated by comma. The default is value is all, which will test all available torch versions listed in this repository.
• cuda version: cuda versions to test against, multiple versions are supported but must be separated by comma. The CUDA versions must be present in our DockerHub repository.

It only test the compatibility of the main branch

This section lists the files used to configure the workflow.

1. .compatibility

This .compatibility file is to tell GitHub Actions which PyTorch and CUDA versions to test against. Each line in the file is in the format ${torch-version}-${cuda-version}, which is a tag for Docker image. Thus, this tag must be present in the docker registry so as to perform the test.

2. .cuda_ext.json

This file controls which CUDA versions will be checked against CUDA extension built. You can add a new entry according to the json schema below to check the AOT build of PyTorch extensions before release.

{
  "build": [
    {
      "torch_command": "",
      "cuda_image": ""
    },
  ]
}

• Code style check
  • post-commit check
• unit testing
  • test on PR
  • report test coverage
  • regular test
• release
  • pypi release
  • test-pypi simulation
  • nightly build
  • docker build
  • draft release post
• example check
  • check on PR
  • regular check
  • manual dispatch
• compatibility check
  • check on PR
  • manual dispatch
  • auto test when release
• community
  • contribution report
  • user engagement report
• helpers
  • comment translation
  • submodule update
  • close inactive issue