Changes

Summary

  1. [ARM] Add additional tests for VMOVL in tail predicated loops. (details)
  2. [AMDGPU] Divergence-driven instruction selection for mul i32 (details)
  3. [AMDGPU] Convert mac/fmac to mad/fma when folding output modifiers (details)
  4. [AArch64][SVE] Add missing load/store patterns for unpacked bfloat vectors. (details)
  5. [VectorCombine] Switch to using a worklist. (details)
  6. [LoopVectorize][X86] Add operands to make it more obvious what line the CHECK concerns (details)
  7. [SelectionDAG] Make WidenVecRes_Convert work for scalable vectors. (details)
  8. [hwasan] also omit safe mem[cpy|mov|set]. (details)
  9. Don't fold (select C, (gep Ptr, Idx), Ptr) if C is vector but Idx is scalar (details)
  10. Unbreak module builds by making InstructionWorklist.h non-modular (details)
  11. [ARM] Allow smaller VMOVL in tail predicated loops (details)
  12. [lldb] [Windows] Fix continuing from breakpoints and singlestepping on ARM/AArch64 (details)
  13. [Matrix] Emit assumption that matrix indices are valid. (details)
  14. Revert "[CodeGen] regenerate test checks; NFC" (details)
  15. Revert "[InstCombine] fold cast of right-shift if high bits are not demanded" (details)
  16. [Passes] Run vector-combine early with -fenable-matrix. (details)
Commit 636fc0ef86f69229486c36f0d3c7539fef860a5a by david.green
[ARM] Add additional tests for VMOVL in tail predicated loops.
The file was addedllvm/test/CodeGen/Thumb2/mve-vmovlloop.ll
Commit 3828ea6181fd007438379de70fc7b9fc9c8dbb02 by jay.foad
[AMDGPU] Divergence-driven instruction selection for mul i32

Differential Revision: https://reviews.llvm.org/D109881
The file was modifiedllvm/test/CodeGen/AMDGPU/wwm-reserved.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/wwm-reserved-spill.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/vgpr-liverange-ir.ll
The file was modifiedllvm/lib/Target/AMDGPU/VOP3Instructions.td
The file was modifiedllvm/test/CodeGen/AMDGPU/urem-seteq-illegal-types.ll
The file was modifiedllvm/lib/Target/AMDGPU/SOPInstructions.td
Commit 0205806d0fe5706c76ae1756e9180918dd495446 by jay.foad
[AMDGPU] Convert mac/fmac to mad/fma when folding output modifiers

Use of output modifiers forces VOP3 encoding for a VOP2 mac/fmac
instruction, so we might as well convert it to the more flexible VOP3-
only mad/fma form.

With this change, the only way we should emit VOP3-encoded mac/fmac is
if regalloc chooses registers that require the VOP3 encoding, e.g. sgprs
for both src0 and src1. In all other cases the mac/fmac should either be
converted to mad/fma or shrunk to VOP2 encoding.

Differential Revision: https://reviews.llvm.org/D110156
The file was modifiedllvm/test/CodeGen/AMDGPU/mad-mix.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/mad-mix-lo.ll
The file was modifiedllvm/lib/Target/AMDGPU/SIFoldOperands.cpp
Commit ab3607c0ed92a7e39952ce22e72e778d2679876a by sander.desmalen
[AArch64][SVE] Add missing load/store patterns for unpacked bfloat vectors.

Reviewed By: c-rhodes

Differential Revision: https://reviews.llvm.org/D110063
The file was modifiedllvm/test/CodeGen/AArch64/sve-ld1-addressing-mode-reg-reg.ll
The file was modifiedllvm/test/CodeGen/AArch64/sve-st1-addressing-mode-reg-reg.ll
The file was modifiedllvm/lib/Target/AArch64/AArch64SVEInstrInfo.td
The file was modifiedllvm/test/CodeGen/AArch64/sve-masked-ldst-nonext.ll
Commit 300870a95c22fde840862cf57d82adba3e5bd633 by flo
[VectorCombine] Switch to using a worklist.

This patch updates VectorCombine to use a worklist to allow iterative
simplifications where a combine enables other combines.

Suggested in D100302.

The main use case at the moment is foldSingleElementStore and
scalarizeLoadExtract working together to improve scalarization.

Note that we now also do not run SimplifyInstructionsInBlock on the
whole function if there have been changes. This means we fail to
remove/simplify instructions not related to any of the vector combines.
IMO this is fine, as simplifying the whole function seems more like a
workaround for not tracking the changed instructions.

Compile-time impact looks neutral:
NewPM-O3: +0.02%
NewPM-ReleaseThinLTO: -0.00%
NewPM-ReleaseLTO-g: -0.02%

http://llvm-compile-time-tracker.com/compare.php?from=52832cd917af00e2b9c6a9d1476ba79754dcabff&to=e66520a4637290550a945d528e3e59573485dd40&stat=instructions

Reviewed By: spatel, lebedev.ri

Differential Revision: https://reviews.llvm.org/D110171
The file was modifiedllvm/test/Transforms/VectorCombine/AArch64/load-extract-insert-store-scalarization.ll
The file was modifiedllvm/test/Transforms/VectorCombine/load-insert-store.ll
The file was modifiedllvm/test/Transforms/VectorCombine/X86/extract-binop.ll
The file was modifiedllvm/lib/Transforms/Vectorize/VectorCombine.cpp
The file was modifiedllvm/test/Transforms/PhaseOrdering/AArch64/matrix-extract-insert.ll
The file was modifiedllvm/test/Transforms/VectorCombine/X86/extract-binop-inseltpoison.ll
Commit 41492d77ba65338b9eb2b7f401e47acf22e4ea19 by llvm-dev
[LoopVectorize][X86] Add operands to make it more obvious what line the CHECK concerns

As we're checking the cost debug analysis these should match the original IR line - so we shouldn't have any variable naming issues.

I'm investigating v4i32 mul -> PMADDDW costs handling (for PR47437) and these CHECK lines were proving tricky to keep track of
The file was modifiedllvm/test/Transforms/LoopVectorize/X86/mul_slm_16bit.ll
Commit 4ca1fbe361860976646ad09da26757bf32563145 by sander.desmalen
[SelectionDAG] Make WidenVecRes_Convert work for scalable vectors.

Most of the code wasn't yet scalable safe, although most of the
code conceptually just works for scalable vectors. This change
makes the algorithm work on ElementCount, where appropriate,
and leaves the fixed-width only code to use `getFixedNumElements`.

Reviewed By: david-arm

Differential Revision: https://reviews.llvm.org/D110058
The file was modifiedllvm/test/CodeGen/AArch64/sve-fcvt.ll
The file was modifiedllvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp
Commit 36daf074d997a79f25a1de2a1b869170ea6c20cc by fmayer
[hwasan] also omit safe mem[cpy|mov|set].

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D109816
The file was modifiedllvm/test/Instrumentation/HWAddressSanitizer/mem-intrinsics.ll
The file was modifiedllvm/lib/Transforms/Instrumentation/HWAddressSanitizer.cpp
The file was modifiedllvm/test/Analysis/StackSafetyAnalysis/ipa.ll
The file was modifiedllvm/test/Instrumentation/HWAddressSanitizer/stack-safety-analysis.ll
The file was modifiedllvm/test/Analysis/StackSafetyAnalysis/local.ll
The file was modifiedllvm/test/Analysis/StackSafetyAnalysis/ipa-alias.ll
The file was modifiedllvm/include/llvm/Analysis/StackSafetyAnalysis.h
The file was modifiedllvm/lib/Analysis/StackSafetyAnalysis.cpp
Commit d0746f2e9bbf08f52196ae12f25d0ef7edcbbe4c by yikong
Don't fold (select C, (gep Ptr, Idx), Ptr) if C is vector but Idx is scalar

The folding rule (select C, (gep Ptr, Idx), Ptr) -> (gep Ptr, (select C,
Idx, 0)) creates a malformed SELECT IR if C is a vector while Idx is scalar.

  SELECT VecC, ScalarIdx, 0

We could splat Idx to a vector but it defeats the purpose of
optimisation. Don't apply the folding rule in this case.

This fixes a regression from commit d561b6fbdbe6d1da05fd92003a4ac1e37bf4b8bc.
The file was modifiedllvm/test/Transforms/InstCombine/select-gep.ll
The file was modifiedllvm/lib/Transforms/InstCombine/InstCombineSelect.cpp
Commit a5e1c746b870d79142419a07a8aecc471eacfed1 by Raphael Isemann
Unbreak module builds by making InstructionWorklist.h non-modular

This regressed in D110181 and apparently the header intentionally requires
DEBUG_TYPE to be defined by the including file. Just exclude the header from
the module to unbreak the build.
The file was modifiedllvm/include/llvm/module.modulemap
Commit 02cd8a6b915a9dab32fdd91167f875ce5f67ebd4 by david.green
[ARM] Allow smaller VMOVL in tail predicated loops

This allows VMOVL in tail predicated loops so long as the the vector
size the VMOVL is extending into is less than or equal to the size of
the VCTP in the tail predicated loop. These cases represent a
sign-extend-inreg (or zero-extend-inreg), which needn't block tail
predication as in https://godbolt.org/z/hdTsEbx8Y.

For this a vecsize has been added to the TSFlag bits of MVE
instructions, which stores the size of the elements that the MVE
instruction operates on. In the case of multiple size (such as a
MVE_VMOVLs8bh that extends from i8 to i16, the largest size was be
chosen). The sizes are encoded as 00 = i8, 01 = i16, 10 = i32 and 11 =
i64, which often (but not always) comes from the instruction encoding
directly. A unit test was added, and although only a subset of the
vecsizes are currently used, the rest should be useful for other cases.

Differential Revision: https://reviews.llvm.org/D109706
The file was modifiedllvm/lib/Target/ARM/MCTargetDesc/ARMBaseInfo.h
The file was modifiedllvm/lib/Target/ARM/ARMInstrFormats.td
The file was modifiedllvm/lib/Target/ARM/ARMInstrMVE.td
The file was modifiedllvm/lib/Target/ARM/ARMLowOverheadLoops.cpp
The file was modifiedllvm/test/CodeGen/Thumb2/mve-vmovlloop.ll
The file was modifiedllvm/unittests/Target/ARM/MachineInstrTest.cpp
Commit 9f34f75ff8f49b0efca6e20d916527a2c432d8b4 by martin
[lldb] [Windows] Fix continuing from breakpoints and singlestepping on ARM/AArch64

Based on suggestions by Eric Youngdale.

This fixes https://llvm.org/PR51673.

Differential Revision: https://reviews.llvm.org/D109777
The file was modifiedlldb/source/Plugins/Process/Windows/Common/TargetThreadWindows.cpp
The file was modifiedlldb/source/Plugins/Platform/Windows/PlatformWindows.cpp
The file was modifiedlldb/source/Plugins/Platform/Windows/PlatformWindows.h
The file was modifiedlldb/source/Plugins/Process/Windows/Common/NativeProcessWindows.h
The file was modifiedlldb/source/Plugins/Process/Windows/Common/NativeThreadWindows.cpp
The file was modifiedlldb/source/Plugins/Process/Windows/Common/NativeProcessWindows.cpp
The file was modifiedlldb/source/Plugins/Process/Windows/Common/ProcessWindows.cpp
Commit ea21d688dc0a420b9fc385562a46017fb39b13e5 by flo
[Matrix] Emit assumption that matrix indices are valid.

The matrix extension requires the indices for matrix subscript
expression to be valid and it is UB otherwise.

extract/insertelement produce poison if the index is invalid, which
limits the optimizer to not be bale to scalarize load/extract pairs for
example, which causes very suboptimal code to be generated when using
matrix subscript expressions with variable indices for large matrixes.

This patch updates IRGen to emit assumes to for index expression to
convey the information that the index must be valid.

This also adjusts the order in which operations are emitted slightly, so
indices & assumes are added before the load of the matrix value.

Reviewed By: erichkeane

Differential Revision: https://reviews.llvm.org/D102478
The file was modifiedclang/lib/CodeGen/CGExprScalar.cpp
The file was modifiedclang/test/CodeGenCXX/matrix-type-operators.cpp
The file was modifiedclang/test/CodeGenObjC/matrix-type-operators.m
The file was modifiedclang/lib/CodeGen/CGExpr.cpp
The file was modifiedclang/test/CodeGen/matrix-type-operators.c
The file was modifiedllvm/include/llvm/IR/MatrixBuilder.h
Commit 1ee851c5859fdb36eca57a46347a1e7b8e1ff236 by spatel
Revert "[CodeGen] regenerate test checks; NFC"

This reverts commit 52832cd917af00e2b9c6a9d1476ba79754dcabff.
The motivating commit 2f6b07316f5 caused several bots to hit
an infinite loop at stage 2, so that needs to be reverted too
while figuring out how to fix that.
The file was modifiedclang/test/CodeGen/aapcs-bitfield.c
Commit c6013f71a4555f6d9ef9c60e6bc4376ad63f1c47 by spatel
Revert "[InstCombine] fold cast of right-shift if high bits are not demanded"

This reverts commit 2f6b07316f560a1f6d225919019dff2e5d6346e5.

This caused several bots to hit an infinite loop at stage 2,
so it needs to be reverted while figuring out how to fix that.
The file was modifiedllvm/lib/Transforms/InstCombine/InstCombineSimplifyDemanded.cpp
The file was modifiedllvm/test/Transforms/InstCombine/trunc-demand.ll
Commit a7c6471a85380f5af644e50daf2951b41c82f1b2 by flo
[Passes] Run vector-combine early with -fenable-matrix.

IR with matrix intrinsics is likely to also contain large vector
operations, which can benefit from early simplifications.

This is the last step in a series of changes to improve code-gen for
code using matrix subscript operators with the C/C++ matrix extension in
CLang, like

    using matrix_t = double __attribute__((matrix_type(15, 15)));

    void foo(unsigned i, matrix_t &A, matrix_t &B) {
      for (unsigned j = 0; j < 4; ++j)
        for (unsigned k = 0; k < i; k++)
          B[k][j] -= A[k][j] * B[i][j];
    }

https://clang.godbolt.org/z/6dKxK1Ed7

Reviewed By: spatel

Differential Revision: https://reviews.llvm.org/D102496
The file was modifiedllvm/test/Other/new-pm-defaults.ll
The file was modifiedllvm/lib/Transforms/IPO/PassManagerBuilder.cpp
The file was modifiedllvm/test/Transforms/PhaseOrdering/AArch64/matrix-extract-insert.ll
The file was modifiedllvm/lib/Passes/PassBuilderPipelines.cpp