Changes

Summary

  1. [mlgo] Enable using TFLite instead of full Tensorflow C API lib (details)
  2. [GreenDragon] Increase clone timeout for lldb-cmake-matrix (details)
  3. [sanitizers] Remove unused bot (details)
  4. [sanitizer] Reorder builders (details)
  5. [sanitizer] Drop Debian from bots name (details)
  6. [sanitizer] Add sanitizer-buildbot-android and sanitizer-buildbot5 (details)
  7. [sanitizer] Less jobs sanitizer-buildbot8 (details)
  8. [sanitizer] Add sanitizer aarch64 bots (details)
  9. [sanitizer] Check sanitizer in bootstrap compiler (details)
  10. [sanitizer] Use clang to buils stage1 if availible (details)
  11. [sanitizer] Restore accidentally removed comma (details)
  12. [sanitizer] Run all tests with "--time-tests -v" (details)
  13. [sanitizer] Split check- into separate steps (details)
  14. [standalone-build] run check-all for llvm sub-project (details)
  15. [standalone-build] external lit built not by llvm but python-lit (details)
  16. Revert "[sanitizer] Run all tests with "--time-tests -v"" (details)
  17. [sanitizer] Run all tests with "--time-tests -v" (details)
  18. [sanitizer] Disable asan tests on aarch64 (details)
  19. [sanitizer] Clone asan to hwasan script (details)
  20. [sanitizer] Update script for HWAddressSanitizer (details)
  21. [sanitizer] Remove ; from LLVM_LIT_ARGS (details)
  22. [clang-ppc64-aix] Switch build compiler to clang (details)
  23. [sanitizer] Don't run asan/msan/fuzzer on staging (details)
  24. Update clang-stage1-RA (details)
  25. Update clang-stage1-RA (details)
  26. [sanitizer] Another attempt to control LIT options (details)
  27. [sanitizer] Fix typo in command line flag (details)
  28. [Arm] Correct typo in LLVM_ENABLE_THREADS option (details)
  29. [sanitizer] Remove outdated commend (details)
  30. [jenkins] Bump up the MACOSX_DEPLOYMENT_TARGET (details)
  31. [sanitizer] Disable broken on aarch64 tests (details)
  32. [Windows on Arm] Add env var for msvc compatibility option (details)
  33. [lldb-cmake-matrix] Use RUNTIME option for libcxx/libcxxabi (details)
  34. [sanitizer] Disable slow libcxx HWASAN tests (details)
  35. [lldb-cmake-matrix] Fix environment leak across runs (details)
  36. [lldb-cmake-matrix] Make sure all stages build the same targets (details)
  37. [libc] Update static archive build targets to "c" under full build. (details)
  38. [libc] Fix typo in libc annotator script. (details)
  39. [sanitizer] Update HWASAN options (details)
  40. [sanitizer] Fix typo (details)
  41. [Windows on Arm] Fix quoting for C/CXX flags (details)
  42. [lldb-cmake-matrix] Remove superfluous quotes around -DLLVM_TARGETS_TO_BUILD (details)
  43. [buildbot][mlgo] Fix incorrectly-passed `-C` flag (details)
  44. [sanitizer] Stop running UBSAN with aarch64. (details)
  45. [standalone-build] Build install and use python-lit (details)
  46. [sanitizer] Re-assign some workers (details)
Commit 2fc83266021b074f1cd448b82250bd8cef47f901 by mtrofin
[mlgo] Enable using TFLite instead of full Tensorflow C API lib

For the training case, TFLite is more appropriate: it plays well with
cmake (thanks to @etrhosek's work to making that turnkey); can be
statically linked; and it's faster.

This patch introduces the support side-by-side the support for the full
TF C API library; once we land the ml-compiler-opt and LLVM changes and
make sure the build bots are still green, we will remove the full TF C
API support.

Differential Revision: https://reviews.llvm.org/D130923
The file was modifiedbuildbot/osuosl/master/config/builders.py (diff)
Commit 5c24db0b08d5db6177542eae3a59266c8f8ed895 by Jonas Devlieghere
[GreenDragon] Increase clone timeout for lldb-cmake-matrix
The file was modifiedzorg/jenkins/jobs/jobs/lldb-cmake-matrix (diff)
Commit 07b612b11431f3e60c464da7a424687fdf01490e by Vitaly Buka
[sanitizers] Remove unused bot
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_selector.py (diff)
Commit f4f4f22ea86824fcab9d5a8c535c5638347f6b67 by Vitaly Buka
[sanitizer] Reorder builders
The file was modifiedbuildbot/osuosl/master/config/builders.py (diff)
Commit 2e9f262e7c1a2411233552958fa3173b63e1d5a4 by Vitaly Buka
[sanitizer] Drop Debian from bots name
The file was modifiedbuildbot/osuosl/master/config/workers.py (diff)
Commit 21701d34e48c784b239b78d7c1c214adb38eaf56 by Vitaly Buka
[sanitizer] Add sanitizer-buildbot-android and sanitizer-buildbot5
The file was modifiedbuildbot/osuosl/master/config/builders.py (diff)
The file was modifiedbuildbot/osuosl/master/config/workers.py (diff)
Commit 645ecb3e6962bf54c2fabba0d2cbac6153a889e0 by Vitaly Buka
[sanitizer] Less jobs sanitizer-buildbot8

Prepare to move it to aarch64
The file was modifiedbuildbot/osuosl/master/config/workers.py (diff)
The file was modifiedbuildbot/osuosl/master/config/builders.py (diff)
Commit cef2327f71810eb2c148ffe4e6936ed675a71d8e by Vitaly Buka
[sanitizer] Add sanitizer aarch64 bots

Differential Revision: https://reviews.llvm.org/D131534
The file was modifiedbuildbot/osuosl/master/config/builders.py (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_selector.py (diff)
The file was modifiedbuildbot/osuosl/master/config/workers.py (diff)
Commit c7fc4d452ae6f81268acae45ebf3755c8d20f2de by Vitaly Buka
[sanitizer] Check sanitizer in bootstrap compiler

It's short but can be useful.
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_functions.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_bootstrap_asan.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_bootstrap_msan.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_bootstrap_ubsan.sh (diff)
Commit 9ee8f64a5ecd4c2bf493cae78779172090af2db6 by Vitaly Buka
[sanitizer] Use clang to buils stage1 if availible
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_functions.sh (diff)
Commit ff7c9289106d0df42d668f6c3b401948ead4e83c by Vitaly Buka
[sanitizer] Restore accidentally removed comma
The file was modifiedbuildbot/osuosl/master/config/builders.py (diff)
Commit 933dd7d130909c8a75c80de6672c07e7f2cf14c9 by Vitaly Buka
[sanitizer] Run all tests with "--time-tests -v"
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_functions.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_fuzzer.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_qemu.sh (diff)
Commit df451b27e196f67ffe28d5c452b384eec9306304 by Vitaly Buka
[sanitizer] Split check- into separate steps

With --time-tests -v reader needs to scroll up to see
the summary of earlier checks.
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_functions.sh (diff)
Commit f14e5f1c94abc460a93a812df62d1b8ebbcc646b by kkleine
[standalone-build] run check-all for llvm sub-project
The file was modifiedzorg/buildbot/builders/annotated/standalone-build.sh (diff)
Commit 6f4d9812877ec93987b57ef55910f7066f591bcc by kkleine
[standalone-build] external lit built not by llvm but python-lit
The file was modifiedzorg/buildbot/builders/annotated/standalone-build.sh (diff)
Commit 77bdf15996cc9b2d67b42f401546d3dab164eb43 by Vitaly Buka
Revert "[sanitizer] Run all tests with "--time-tests -v""

Looks like -v does not work with LIT_OPTS.

This reverts commit 933dd7d130909c8a75c80de6672c07e7f2cf14c9.
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_fuzzer.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_functions.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_qemu.sh (diff)
Commit 072dc6e8e2158c320040e1d44123ce86c3f21929 by Vitaly Buka
[sanitizer] Run all tests with "--time-tests -v"
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_android.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_bootstrap_ubsan.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_fast.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_bootstrap_msan.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_functions.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_cmake.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_fuzzer.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_qemu.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_standard.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_bootstrap_asan.sh (diff)
Commit bb18f36a0b0516f59ae0024445068b6db3b418fe by Vitaly Buka
[sanitizer] Disable asan tests on aarch64
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_bootstrap_asan.sh (diff)
Commit caa10df46df7eaa996e81b67d29de0da18a704c4 by Vitaly Buka
[sanitizer] Clone asan to hwasan script
The file was addedzorg/buildbot/builders/sanitizers/buildbot_bootstrap_hwasan.sh
Commit 1a0c80a8caa2c36755f76b2e200f061c67be88ea by Vitaly Buka
[sanitizer] Update script for HWAddressSanitizer
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_bootstrap_hwasan.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_functions.sh (diff)
Commit 67f9c3d2ccdab0126d75b0ffd2ccc7140f740d6b by Vitaly Buka
[sanitizer] Remove ; from LLVM_LIT_ARGS

Variable as a list does not work with ExternalProject_Add.
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_android_functions.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_functions.sh (diff)
Commit c07753f737a671808e61363c7721bdec980e36d3 by Jake.Egan
[clang-ppc64-aix] Switch build compiler to clang

When LLVM updated the C++ standards to 17, the AIX bot encountered multiple related build issues because ibm-clang doesn't currently have a release to support C++17. So, this patch switches the build compiler for AIX from ibm-clang to clang as a step to mitigate that.

Reviewed By: hubert.reinterpretcast

Differential Revision: https://reviews.llvm.org/D131697
The file was modifiedbuildbot/osuosl/master/config/builders.py (diff)
Commit fe1aaa147fbd6689c421d4e986f5d26893374c9e by Vitaly Buka
[sanitizer] Don't run asan/msan/fuzzer on staging

Release worker for hwasan and ubsan. The rest is either good or
not a priority for now.
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_bootstrap_msan.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_fuzzer.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_bootstrap_asan.sh (diff)
Commit 32dffd8a4f221c36c10795fe2c6c655763686b2b by github
Update clang-stage1-RA

Switching hosts due to downed host.
The file was modifiedzorg/jenkins/jobs/jobs/clang-stage1-RA (diff)
Commit 911071d40187169042a43f88392256292d070c54 by github
Update clang-stage1-RA

Host restored.
The file was modifiedzorg/jenkins/jobs/jobs/clang-stage1-RA (diff)
Commit f2b33a92bba69ff2419be00e8347c2fed6765a22 by Vitaly Buka
[sanitizer] Another attempt to control LIT options
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_functions.sh (diff)
Commit 117c6a8eff17b312ac91415ed46f3156b3cd9f76 by Vitaly Buka
[sanitizer] Fix typo in command line flag
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_functions.sh (diff)
Commit 16bd2928d2408d9342e9de28884b758194e3fe86 by david.spickett
[Arm] Correct typo in LLVM_ENABLE_THREADS option
The file was modifiedbuildbot/osuosl/master/config/builders.py (diff)
Commit b7b9f1591b4f83a2823311353e974474552fda9d by github
[sanitizer] Remove outdated commend

We use libcxx and ubsan from 16682e95f72cdbb5bf60da42e58a26064d5956de
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_functions.sh (diff)
Commit 57d34a9f68778eadc72ec3053a7138cabe0a56fa by Azharuddin Mohammed
[jenkins] Bump up the MACOSX_DEPLOYMENT_TARGET

LLVM Project switched to C++17. Bumping up the deployment target fixes
errors regarding usage of features which are available only in the newer
versions of macOS.
The file was modifiedzorg/jenkins/jobs/jobs/clang-stage1-cmake-RA-incremental (diff)
The file was modifiedzorg/jenkins/jobs/jobs/clang-stage1-RA (diff)
The file was modifiedzorg/jenkins/jobs/jobs/clang-san-iossim (diff)
Commit 05d30f50eeb830e9a178eee82de3c9719b416401 by Vitaly Buka
[sanitizer] Disable broken on aarch64 tests
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_functions.sh (diff)
Commit e540c3f0253358d9712ef1ba9b792c763057f697 by david.spickett
[Windows on Arm] Add env var for msvc compatibility option

We are using clang 14.0.5 which has a default version of 19.26.

This adds use of an env var MSVC_VER that we'll set on our
side for this and future updates as the clang-cl version
doesn't always track what llvm needs to build.

Differential Revision: https://reviews.llvm.org/D131770
The file was modifiedbuildbot/osuosl/master/config/builders.py (diff)
Commit 725e5efb00696be9ba4dbf452b9802f09719b017 by Felipe de Azevedo Piovezan
[lldb-cmake-matrix] Use RUNTIME option for libcxx/libcxxabi

Summary:
As of D112724, building libcxx/libcxxabi with the "PROJECTS" option is
deprecated and causes the following CMake warning:

> Using LLVM_ENABLE_PROJECTS=libcxxabi is deprecated now, please use
-DLLVM_ENABLE_RUNTIMES=libcxxabi or see the instructions at
https://libcxx.llvm.org/BuildingLibcxx.html for building the
runtimes.

This commit fixes the warning, also in the hopes of fixing the failures in
lldb-cmake-matrix that test different clang versions. The different Clangs fail
to find the correct libcxx, instead using the host machine's libcxx.

Reviewers: aprantl

Differential Revision: https://reviews.llvm.org/D131970
The file was modifiedzorg/jenkins/jobs/jobs/lldb-cmake-matrix (diff)
Commit da73b75c713b731ad4d604a81e1311046418b554 by Vitaly Buka
[sanitizer] Disable slow libcxx HWASAN tests
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_functions.sh (diff)
Commit 3175929b194c38f4a1b20608bd28bf9b54bc4cf1 by Felipe de Azevedo Piovezan
[lldb-cmake-matrix] Fix environment leak across runs

Any `export` statement in a pipeline description will leak across
different compiler stages or, worse, between two runs of the entire
pipeline. This was seen recently in two ways:

1. The Dwarf 2 / 4 runs were using a Clang 13.0 built on a previous
pipeline run, and they were failing because some tests require ToT
clang.
2. The Dwarf 2 / 4 runs were trying to use Clang 13.0, but the workspace
was cleaned after the previous pipeline run, resulting in a "cannot
find clang" error.

This commit addresses the issue by no longer using environment variables
to communicate which Clang to use when building LLDB tests. Instead, we
now use parameters to the `test_monorepo_build.py` script. More
generally though, there are still a lot of `export` statements in the
pipeline, and these should be examined in isolation to determine how to
best remove them.

Jenkins provides ways to set environment variables more carefully [1];
however, for the purposes of the LLDB test compiler variable, this is
dangerous: we don't want such a flag to last any longer than the CMake
configure step, so this commit uses the script argument approach.

[1]: https://www.jenkins.io/doc/book/pipeline/syntax/#environment

Differential Revision: https://reviews.llvm.org/D132035
The file was modifiedtest/jenkins/test_monorepo_build.py (diff)
The file was modifiedzorg/jenkins/jobs/jobs/lldb-cmake-matrix (diff)
The file was modifiedzorg/jenkins/monorepo_build.py (diff)
Commit bc6a5e96112c760cdd696210e74a5cfc191d6a26 by Felipe de Azevedo Piovezan
 [lldb-cmake-matrix] Make sure all stages build the same targets

The `lldb_cmake_builder` function of `monorepo_build.py` creates the
cmake invocation by first setting:

'-DLLVM_TARGETS_TO_BUILD=X86;ARM;AArch64`

And then relies on a user to override it with:

`--cmake-flag="-DLLVM_TARGETS_TO_BUILD=X86"`

The pipeline builder for `lldb-cmake-matrix` forgot to override the
targets for all but the first invocation of this function (stage 'Build
LLDB') . As such, the two first invocations build different targets,
causing the second invocation ('Test DWARF2') to perform almost a full
recompile, instead of a no-op incremental compile.

The function `cmake_builder` uses a different strategy: it never sets an
initial value for target, relying on whatever defaults LLVM uses. This
function is only used when building the older Clangs. We also change
calls to this function to explicitly set "X86" only.

Differential Revision: https://reviews.llvm.org/D132071
The file was modifiedzorg/jenkins/jobs/jobs/lldb-cmake-matrix (diff)
The file was modifiedtest/jenkins/test_monorepo_build.py (diff)
Commit cfb9ac70059a9376acefc4cc48fbfa70c88962ff by sivachandra
[libc] Update static archive build targets to "c" under full build.
The file was modifiedzorg/buildbot/builders/annotated/libc-linux.py (diff)
Commit b2c8f4f19e1392da02c56f2b58fedbf71f0a5326 by sivachandra
[libc] Fix typo in libc annotator script.
The file was modifiedzorg/buildbot/builders/annotated/libc-linux.py (diff)
Commit 2addee9002a055e572ab9e5c97b13ffcb364d684 by Vitaly Buka
[sanitizer] Update HWASAN options

Use abort_on_error=1:
Some tests expect crash on trap, however hwasan
intercepts trap and exits with error code.

Disable plugins:
HWASAN globals are broken with DSO
https://github.com/llvm/llvm-project/issues/57206
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_functions.sh (diff)
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_functions.sh (diff)
Commit fc0b0ee2b45ee4ca7442a4510062cc06bd0741c5 by david.spickett
[Windows on Arm] Fix quoting for C/CXX flags

By using single quotes we ended up with
"-DCMAKE_C_FLAGS='-fms-compatibility-version=%MSVC_VER%'"
on the command line which doesn't do env var substitution.

The exact logic of that, I can't figure out, but BUILTINS
already works so just follow that. We only have one flag
to pass so spaces aren't an issue anyway.
The file was modifiedbuildbot/osuosl/master/config/builders.py (diff)
Commit c486a04db3aa7bf099de6bda35e49158252924ea by michaelbuch12
[lldb-cmake-matrix] Remove superfluous quotes around -DLLVM_TARGETS_TO_BUILD

The build bots are failing because the extra quote around the
command line parameter with following error:
```
CMake Error: The source directory
"/Users/buildslave/jenkins/workspace/lldb-cmake-matrix/lldb-build/"-DLLVM_TARGETS_TO_BUILD=X86""
does not exist.
```

This patch removes the quotes.

Differential Revision: https://reviews.llvm.org/D132118
The file was modifiedzorg/jenkins/jobs/jobs/lldb-cmake-matrix (diff)
The file was modifiedtest/jenkins/test_monorepo_build.py (diff)
Commit 3fcafad781dd21af12560c3efd5c4b0916227e3c by mtrofin
[buildbot][mlgo] Fix incorrectly-passed `-C` flag
The file was modifiedbuildbot/osuosl/master/config/builders.py (diff)
Commit 3a511c73194a34a37dd98d361abec30705e3df16 by Vitaly Buka
[sanitizer] Stop running UBSAN with aarch64.
The file was modifiedzorg/buildbot/builders/sanitizers/buildbot_bootstrap_ubsan.sh (diff)
Commit 5f511e112a7cd26b7671af70abf7105448a9e406 by kkleine
[standalone-build] Build install and use python-lit

This ensures that LLVM sub-projects are being tested with the lit of the same revision

Differential Revision: https://reviews.llvm.org/D131733
The file was modifiedzorg/buildbot/builders/annotated/standalone-build.sh (diff)
Commit 3f81ee5d754d4403b5084fc071c3eed62bd956aa by Vitaly Buka
[sanitizer] Re-assign some workers
The file was modifiedbuildbot/osuosl/master/config/builders.py (diff)
The file was modifiedbuildbot/osuosl/master/config/workers.py (diff)