Merge branch 'refactor_existing_workflows' of github.com:alexbatashev…

…/llvm into refactor_existing_workflows * 'refactor_existing_workflows' of github.com:alexbatashev/llvm: (2051 commits) [SYCL][L0] Honor property::queue::enable_profiling (intel#5543) [SYCL][CI] Enable sccache on Windows (intel#5589) [SYCL][Doc] Move internal design docs (intel#5556) [sycl-post-link] Initialize the integer Value variable (intel#5585) [CI] Fix nightly builds (intel#5584) [SYCL][L0] Fix use of copy-engines in L0 interop queue (intel#5579) Update OpenCL headers tag to dcd5bed (intel#5575) [SYCL] Fix warning for InOrderQueueSyncCheck unit test build (intel#5577) [SYCL][HIP] Remove arch requirement for running lit tests (intel#5253) [SYCL][L0] Fix timestamp calculation (in ns) (intel#5555) [SYCL] Fix sync of host task vs kernel for in-order queue (intel#5551) [sycl-post-link] Add a check for device globals with device_image_scope (intel#5517) [SYCL] Fix SYCL Kernel Body Check (intel#5546) [SYCL] Add support for SYCL 2020 in class group (intel#5447) Fix tests after 1c729d7 Use align attribute for kernel pointer arg alignment Fix tests after 18834dc Mark pointer-typed kernel arguments as ABI aligned [CI] Add experimental Windows build to GitHub Actions nightly (intel#5560) [sycl-post-link][NFC] Address clang-tidy concerns in the sycl-post-link (intel#5552) Fix lit test after changes in Clang Improve backward compatibility for DISubRange ...
alexbatashev · Feb 23, 2022 · b40a1e1 · b40a1e1
2 parents 4c53902 + 014985b
commit b40a1e1
Show file tree

Hide file tree

Showing 9,424 changed files with 252,348 additions and 142,847 deletions.
diff --git a/.clang-tidy b/.clang-tidy
@@ -6,6 +6,10 @@ CheckOptions:
     value:           CamelCase
   - key:             readability-identifier-naming.FunctionCase
     value:           camelBack
+  # Exclude from scanning as this is an exported symbol used for fuzzing
+  # throughout the code base.
+  - key:             readability-identifier-naming.FunctionIgnoredRegexp
+    value:           "LLVMFuzzerTestOneInput"
   - key:             readability-identifier-naming.MemberCase
     value:           CamelCase
   - key:             readability-identifier-naming.ParameterCase

diff --git a/.github/workflows/closed-issues.yml b/.github/workflows/closed-issues.yml
@@ -0,0 +1,13 @@
+name: Labeling closed issues
+on:
+  issues:
+    types: ['closed']
+
+jobs:
+  automate-issues-labels:
+    runs-on: ubuntu-latest
+    if: github.repository == 'llvm/llvm-project'
+    steps:
+      - uses: andymckay/labeler@1.0.4
+        with:
+          remove-labels: "awaiting-review"
diff --git a/.github/workflows/issue-release-workflow.yml b/.github/workflows/issue-release-workflow.yml
@@ -0,0 +1,84 @@
+# This contains the workflow definitions that allow users to test backports
+# to the release branch using comments on issues.
+#
+# /cherry-pick <commit> <...>
+#
+# This comment will attempt to cherry-pick the given commits to the latest
+# release branch (release/Y.x) and if successful push the result to a branch
+# on github.
+#
+# /branch <owner>/<repo>/<branch>
+#
+# This comment will create a pull request from <branch> to the latest release
+# branch.
+
+name: Issue Release Workflow
+
+on:
+  issue_comment:
+    types:
+      - created
+      - edited
+
+env:
+  COMMENT_BODY: ${{ github.event.comment.body }}
+
+jobs:
+  backport-commits:
+    name: Backport Commits
+    runs-on: ubuntu-20.04
+    if: >-
+        (github.repository == 'llvm/llvm-project') &&
+        !startswith(github.event.comment.body, '<!--IGNORE-->') &&
+        contains(github.event.comment.body, '/cherry-pick')
+    steps:
+      - name: Fetch LLVM sources
+        uses: actions/checkout@v2
+        with:
+          repository: llvm/llvm-project
+          # GitHub stores the token used for checkout and uses it for pushes
+          # too, but we want to use a different token for pushing, so we need
+          # to disable persist-credentials here.
+          persist-credentials: false
+          fetch-depth: 0
+
+      - name: Setup Environment
+        run: |
+          pip install -r ./llvm/utils/git/requirements.txt
+          ./llvm/utils/git/github-automation.py --token ${{ github.token }} setup-llvmbot-git
+
+      - name: Backport Commits
+        run: |
+          printf "$COMMENT_BODY" |
+          ./llvm/utils/git/github-automation.py \
+          --repo $GITHUB_REPOSITORY \
+          --token ${{ secrets.RELEASE_WORKFLOW_PUSH_SECRET }} \
+          release-workflow \
+          --issue-number ${{ github.event.issue.number }} \
+          auto
+
+  create-pull-request:
+    name: Create Pull Request
+    runs-on: ubuntu-20.04
+    if: >-
+        (github.repository == 'llvm/llvm-project') &&
+        !startswith(github.event.comment.body, '<!--IGNORE-->') &&
+        contains(github.event.comment.body, '/branch')
+
+    steps:
+      - name: Fetch LLVM sources
+        uses: actions/checkout@v2
+
+      - name: Setup Environment
+        run:  |
+          pip install -r ./llvm/utils/git/requirements.txt
+
+      - name: Create Pull Request
+        run: |
+          printf "$COMMENT_BODY" |
+          ./llvm/utils/git/github-automation.py \
+          --repo $GITHUB_REPOSITORY \
+          --token ${{ secrets.RELEASE_WORKFLOW_PUSH_SECRET }} \
+          release-workflow \
+          --issue-number ${{ github.event.issue.number }} \
+          auto
diff --git a/.github/workflows/issue-subscriber.yml b/.github/workflows/issue-subscriber.yml
@@ -13,8 +13,9 @@ jobs:
     - name: Setup Automation Script
       run:  |
         curl -O -L https://raw.githubusercontent.com/$GITHUB_REPOSITORY/$GITHUB_SHA/llvm/utils/git/github-automation.py
+        curl -O -L https://raw.githubusercontent.com/$GITHUB_REPOSITORY/$GITHUB_SHA/llvm/utils/git/requirements.txt
         chmod a+x github-automation.py
-        pip install PyGithub
+        pip install -r requirements.txt
 
     - name: Update watchers
       run: |

diff --git a/.github/workflows/sycl_containers.yaml b/.github/workflows/sycl_containers.yaml
@@ -107,6 +107,15 @@ jobs:
         uses: actions/checkout@v2
         with:
           fetch-depth: 2
+      - name: Get dependencies configuration
+        id: deps
+        run: |
+          DEPS=`cat devops/dependencies.json`
+          DEPS="${DEPS//'%'/'%25'}"
+          DEPS="${DEPS//$'\n'/'%0A'}"
+          DEPS="${DEPS//$'\r'/'%0D'}"
+          echo $DEPS
+          echo "::set-output name=deps::$DEPS"
       - name: Build and Push Container
         uses: ./devops/actions/build_container
         with:
@@ -117,4 +126,10 @@ jobs:
           tags: |
             ghcr.io/${{ github.repository }}/ubuntu2004_intel_drivers:unstable-${{ github.sha }}
             ghcr.io/${{ github.repository }}/ubuntu2004_intel_drivers:unstable
+          build-args: |
+            compute_runtime_tag=${{fromJson(steps.deps.outputs.deps).linux_staging.compute_runtime.github_tag}}
+            igc_tag=${{fromJson(steps.deps.outputs.deps).linux_staging.igc.github_tag}}
+            tbb_tag=${{fromJson(steps.deps.outputs.deps).linux_staging.tbb.github_tag}}
+            fpgaemu_tag=${{fromJson(steps.deps.outputs.deps).linux_staging.fpgaemu.github_tag}}
+            cpu_tag=${{fromJson(steps.deps.outputs.deps).linux_staging.oclcpu.github_tag}}
 
diff --git a/.github/workflows/sycl_linux_build_and_test.yml b/.github/workflows/sycl_linux_build_and_test.yml
@@ -36,7 +36,7 @@ on:
       build_configure_extra_args:
         type: string
         required: false
-        default: "--hip --hip-amd-arch=gfx906 --cuda"
+        default: "--hip --cuda"
       build_artifact_suffix:
         type: string
         required: true

diff --git a/.github/workflows/sycl_nightly.yml b/.github/workflows/sycl_nightly.yml
@@ -17,6 +17,12 @@ jobs:
       build_cache_root: "/__w/"
       build_artifact_suffix: default
       lts_config: "ocl_gen9;ocl_x64;hip_amdgpu"
+
+  windows_default:
+    name: Windows (experimental)
+    if: github.repository == 'intel/llvm'
+    uses: ./.github/workflows/sycl_windows_build_and_test.yml
+
   ubuntu2004_docker_build_push:
     if: github.repository == 'intel/llvm'
     runs-on: ubuntu-latest
@@ -53,4 +59,14 @@ jobs:
         tags: |
           ghcr.io/${{ github.repository }}/sycl_ubuntu2004_nightly:no-drivers-${{ github.sha }}
           ghcr.io/${{ github.repository }}/sycl_ubuntu2004_nightly:no-drivers
-
+  ubuntu2004_build_test_new_pm:
+    # Default Linux building and LIT testing using new Pass Manager by default
+    if: github.repository == 'intel/llvm'
+    uses: intel/llvm/.github/workflows/sycl_linux_build_and_test.yml@sycl
+    with:
+      build_runs_on: build
+      build_cache_root: "/__w/"
+      build_cache_suffix: new_pm
+      build_artifact_suffix: new_pm
+      build_configure_extra_args: '--hip --cuda --cmake-opt=-DLLVM_ENABLE_NEW_PASS_MANAGER=ON'
+      lts_config: "hip_amdgpu;ocl_x64"
diff --git a/.github/workflows/sycl_precommit.yml b/.github/workflows/sycl_precommit.yml
@@ -5,10 +5,6 @@ on:
     branches:
     - sycl
 
-concurrency:
-  group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
-  cancel-in-progress: true
-
 jobs:
   lint:
     runs-on: ubuntu-latest

diff --git a/.github/workflows/sycl_stale_issues.yml b/.github/workflows/sycl_stale_issues.yml
@@ -0,0 +1,19 @@
+name: Stale Issues
+
+on:
+  schedule:
+    - cron: '30 1 * * *'
+
+jobs:
+  close-issues:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/stale@v4
+      with:
+        stale-issue-message: 'This issue is stale because it has been open 180 days with no activity. Remove stale label or comment or this will be automatically closed in 30 days.'
+        close-issue-message: 'This issue was closed because it has been stalled for 30 days with no activity. Please, re-open if the issue still exists.'
+        days-before-stale: 180
+        days-before-close: 30
+        exempt-issue-labels: 'confirmed,hip,cuda,enhancement,help wanted,upstream'
+        stale-issue-label: 'stale'
+        exempt-all-issue-assignees: true
diff --git a/.github/workflows/sycl_windows_build_and_test.yml b/.github/workflows/sycl_windows_build_and_test.yml
@@ -0,0 +1,70 @@
+name: SYCL Windows Test
+
+on:
+  workflow_call:
+
+jobs:
+  build:
+    name: Build
+    runs-on: windows-2022
+    steps:
+    - uses: actions/checkout@v2
+      with:
+        path: src
+        fetch-depth: 1
+    - name: Install dependencies
+      shell: cmd
+      run: |
+        choco install -y cuda --version 11.6.0.51123
+        choco install -y ninja
+        choco install -y sccache --version 0.2.15
+        refreshenv
+        echo CUDA_PATH=%CUDA_PATH%
+        echo CUDA_PATH=%CUDA_PATH% >> %GITHUB_ENV%
+    - uses: ilammy/msvc-dev-cmd@9f8ae839b01883414208f29e3e24524387f48e1f
+      with:
+        arch: amd64
+    - name: Setup Cache
+      uses: actions/cache@v2
+      if: ${{ github.event_name != 'pull_request' }}
+      id: cache
+      with:
+        path: cache
+        key: sycl-win-build-${{ github.sha }}
+        restore-keys: |
+          sycl-win-build-
+    - name: Configure
+      shell: cmd
+      env:
+        CUDA_PATH: 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6'
+        CUDA_TOOLKIT_ROOT_DIR: 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6'
+      # TODO switch to clang-cl and lld when this is fixed https://github.com/oneapi-src/level-zero/issues/83
+      # TODO enable sccache, when problems with PDB are resolved
+      run: |
+        mkdir build
+        mkdir install
+        IF NOT EXIST cache MKDIR cache
+        set SCCACHE_DIR=%GITHUB_WORKSPACE%\cache
+        set PATH=C:\ProgramData\chocolatey\lib\sccache\tools\sccache-v0.2.15-x86_64-pc-windows-msvc;%PATH%
+        python.exe src/buildbot/configure.py -o build ^
+          --cmake-opt="-DCMAKE_C_COMPILER=cl" ^
+          --cmake-opt="-DCMAKE_CXX_COMPILER=cl" ^
+          --cmake-opt="-DCMAKE_INSTALL_PREFIX=%GITHUB_WORKSPACE%\install" ^
+          --cmake-opt="-DCMAKE_CXX_COMPILER_LAUNCHER=sccache" ^
+          --cmake-opt="-DCMAKE_C_COMPILER_LAUNCHER=sccache" ^
+          --cuda
+    - name: Build
+      shell: cmd
+      env:
+        CUDA_PATH: 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6'
+        CUDA_TOOLKIT_ROOT_DIR: 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6'
+      run: |
+        cmake --build build --target sycl-toolchain
+    - name: Install
+      shell: cmd
+      run: cmake --build build --target deploy-sycl-toolchain
+    - name: Upload toolchain
+      uses: actions/upload-artifact@v2
+      with:
+        name: sycl_windows_default
+        path: install/**/*
diff --git a/README.md b/README.md
@@ -28,3 +28,86 @@ See [LICENSE.txt](sycl/LICENSE.TXT) for details.
 See [CONTRIBUTING.md](CONTRIBUTING.md) for details.
 
 *\*Other names and brands may be claimed as the property of others.*
+
+This is an example work-flow and configuration to get and build the LLVM source:
+
+1. Checkout LLVM (including related sub-projects like Clang):
+
+     * ``git clone https://github.com/llvm/llvm-project.git``
+
+     * Or, on windows, ``git clone --config core.autocrlf=false
+    https://github.com/llvm/llvm-project.git``
+
+2. Configure and build LLVM and Clang:
+
+     * ``cd llvm-project``
+
+     * ``cmake -S llvm -B build -G <generator> [options]``
+
+        Some common build system generators are:
+
+        * ``Ninja`` --- for generating [Ninja](https://ninja-build.org)
+          build files. Most llvm developers use Ninja.
+        * ``Unix Makefiles`` --- for generating make-compatible parallel makefiles.
+        * ``Visual Studio`` --- for generating Visual Studio projects and
+          solutions.
+        * ``Xcode`` --- for generating Xcode projects.
+
+        Some common options:
+
+        * ``-DLLVM_ENABLE_PROJECTS='...'`` and ``-DLLVM_ENABLE_RUNTIMES='...'`` ---
+          semicolon-separated list of the LLVM sub-projects and runtimes you'd like to
+          additionally build. ``LLVM_ENABLE_PROJECTS`` can include any of: clang,
+          clang-tools-extra, cross-project-tests, flang, libc, libclc, lld, lldb,
+          mlir, openmp, polly, or pstl. ``LLVM_ENABLE_RUNTIMES`` can include any of
+          libcxx, libcxxabi, libunwind, compiler-rt, libc or openmp. Some runtime
+          projects can be specified either in ``LLVM_ENABLE_PROJECTS`` or in
+          ``LLVM_ENABLE_RUNTIMES``.
+
+          For example, to build LLVM, Clang, libcxx, and libcxxabi, use
+          ``-DLLVM_ENABLE_PROJECTS="clang" -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi"``.
+
+        * ``-DCMAKE_INSTALL_PREFIX=directory`` --- Specify for *directory* the full
+          path name of where you want the LLVM tools and libraries to be installed
+          (default ``/usr/local``). Be careful if you install runtime libraries: if
+          your system uses those provided by LLVM (like libc++ or libc++abi), you
+          must not overwrite your system's copy of those libraries, since that
+          could render your system unusable. In general, using something like
+          ``/usr`` is not advised, but ``/usr/local`` is fine.
+
+        * ``-DCMAKE_BUILD_TYPE=type`` --- Valid options for *type* are Debug,
+          Release, RelWithDebInfo, and MinSizeRel. Default is Debug.
+
+        * ``-DLLVM_ENABLE_ASSERTIONS=On`` --- Compile with assertion checks enabled
+          (default is Yes for Debug builds, No for all other build types).
+
+      * ``cmake --build build [-- [options] <target>]`` or your build system specified above
+        directly.
+
+        * The default target (i.e. ``ninja`` or ``make``) will build all of LLVM.
+
+        * The ``check-all`` target (i.e. ``ninja check-all``) will run the
+          regression tests to ensure everything is in working order.
+
+        * CMake will generate targets for each tool and library, and most
+          LLVM sub-projects generate their own ``check-<project>`` target.
+
+        * Running a serial build will be **slow**.  To improve speed, try running a
+          parallel build.  That's done by default in Ninja; for ``make``, use the option
+          ``-j NNN``, where ``NNN`` is the number of parallel jobs, e.g. the number of
+          CPUs you have.
+
+      * For more information see [CMake](https://llvm.org/docs/CMake.html)
+
+Consult the
+[Getting Started with LLVM](https://llvm.org/docs/GettingStarted.html#getting-started-with-llvm)
+page for detailed information on configuring and compiling LLVM. You can visit
+[Directory Layout](https://llvm.org/docs/GettingStarted.html#directory-layout)
+to learn about the layout of the source code tree.
+
+## Getting in touch
+
+Join [LLVM Discourse forums](https://discourse.llvm.org/), [discord chat](https://discord.gg/xS7Z362) or #llvm IRC channel on [OFTC](https://oftc.net/).
+
+The LLVM project has adopted a [code of conduct](https://llvm.org/docs/CodeOfConduct.html) for
+participants to all modes of communication within the project.
diff --git a/bolt/docs/Heatmaps.md b/bolt/docs/Heatmaps.md
@@ -23,11 +23,10 @@ $ perf record -e cycles:u -j any,u [-p PID|-a] -- sleep <interval>
 Note that at the moment running with LBR (`-j any,u` or `-b`) is
 a requirement.
 
-Once the run is complete, and `perf.data` is generated, run BOLT in
-a heatmap mode:
+Once the run is complete, and `perf.data` is generated, run llvm-bolt-heatmap:
 
 ```bash
-$ llvm-bolt heatmap -p perf.data <executable>
+$ llvm-bolt-heatmap -p perf.data <executable>
 ```
 
 By default the heatmap will be dumped to *stdout*. You can change it