1. Fix two pessimizing moves. (details)
  2. [VE] Extract & insert vector element isel (details)
  3. [VE] Expand single-element BUILD_VECTOR to INSERT_VECTOR_ELT (details)
  4. [LV] Legalize scalable VF hints (details)
  5. [SVE] Add support to vectorize_width loop pragma for scalable vectors (details)
  6. [AArch64] Fix crash caused by invalid vector element type (details)
Commit 2fe625e9abd037b8721977e9736c822ef872487f by csigg
Fix two pessimizing moves.


Reviewed By: rriddle

Differential Revision:
The file was modifiedmlir/include/mlir/Parser.h
The file was modifiedllvm/include/llvm/Support/TaskQueue.h
Commit d1b606f897bee5f16ce9c459d3a79c8119018d9c by simon.moll
[VE] Extract & insert vector element isel

Isel and tests for extract_vector_elt and insert_vector_elt.

Reviewed By: kaz7

Differential Revision:
The file was addedllvm/test/CodeGen/VE/Vector/insert_elt.ll
The file was modifiedllvm/lib/Target/VE/VEISelLowering.h
The file was modifiedllvm/lib/Target/VE/
The file was addedllvm/test/CodeGen/VE/Vector/extract_elt.ll
The file was modifiedllvm/lib/Target/VE/VEISelLowering.cpp
The file was modifiedllvm/lib/Target/VE/
Commit eeba70a463c70c76868421e08c4036c70bfad994 by simon.moll
[VE] Expand single-element BUILD_VECTOR to INSERT_VECTOR_ELT

We do this mostly to be able to test the insert_vector_elt isel
patterns. As long as we don't, most single element insertions show up as
`BUILD_VECTOR` in the backend.

Reviewed By: kaz7

Differential Revision:
The file was addedllvm/test/CodeGen/VE/Vector/expand_single_elem_build_vec.ll
The file was modifiedllvm/test/CodeGen/VE/Vector/insert_elt.ll
The file was modifiedllvm/lib/Target/VE/VEISelLowering.cpp
Commit 1e7efd397ac259429df9f212c9ec16bec8ff2533 by cullen.rhodes
[LV] Legalize scalable VF hints

In the following loop:

  void foo(int *a, int *b, int N) {
    for (int i=0; i<N; ++i)
      a[i + 4] = a[i] + b[i];

The loop dependence constrains the VF to a maximum of (4, fixed), which
would mean using <4 x i32> as the vector type in vectorization.
Extending this to scalable vectorization, a VF of (4, scalable) implies
a vector type of <vscale x 4 x i32>. To determine if this is legal
vscale must be taken into account. For this example, unless
max(vscale)=1, it's unsafe to vectorize.

For SVE, the number of bits in an SVE register is architecturally
defined to be a multiple of 128 bits with a maximum of 2048 bits, thus
the maximum vscale is 16. In the loop above it is therefore unfeasible
to vectorize with SVE. However, in this loop:

  void foo(int *a, int *b, int N) {
    #pragma clang loop vectorize_width(X, scalable)
    for (int i=0; i<N; ++i)
      a[i + 32] = a[i] + b[i];

As long as max(vscale) multiplied by the number of lanes 'X' doesn't
exceed the dependence distance, it is safe to vectorize. For SVE a VF of
(2, scalable) is within this constraint, since a vector of <16 x 2 x 32>
will have no dependencies between lanes. For any number of lanes larger
than this it would be unsafe to vectorize.

This patch extends 'computeFeasibleMaxVF' to legalize scalable VFs
specified as loop hints, implementing the following behaviour:
  * If the backend does not support scalable vectors, ignore the hint.
  * If scalable vectorization is unfeasible given the loop
    dependence, like in the first example above for SVE, then use a
    fixed VF.
  * Accept scalable VFs if it's safe to do so.
  * Otherwise, clamp scalable VFs that exceed the maximum safe VF.

Reviewed By: sdesmalen, fhahn, david-arm

Differential Revision:
The file was addedllvm/test/Transforms/LoopVectorize/scalable-vf-hint.ll
The file was modifiedllvm/include/llvm/Analysis/LoopAccessAnalysis.h
The file was modifiedllvm/include/llvm/Transforms/Vectorize/LoopVectorizationLegality.h
The file was modifiedllvm/test/Transforms/LoopVectorize/optimal-epilog-vectorization-limitations.ll
The file was addedllvm/test/Transforms/LoopVectorize/optimal-epilog-vectorization-scalable.ll
The file was addedllvm/test/Transforms/LoopVectorize/AArch64/scalable-vf-hint.ll
The file was modifiedllvm/test/Transforms/LoopVectorize/metadata-width.ll
The file was modifiedllvm/lib/Transforms/Vectorize/LoopVectorize.cpp
The file was modifiedllvm/test/Transforms/LoopVectorize/scalable-loop-unpredicated-body-scalar-tail.ll
Commit 38d18d93534d290d045bbbfa86337e70f1139dc2 by david.sherwood
[SVE] Add support to vectorize_width loop pragma for scalable vectors

This patch adds support for two new variants of the vectorize_width

1. vectorize_width(X[, fixed|scalable]) where an optional second
parameter is passed to the vectorize_width pragma, which indicates if
the user wishes to use fixed width or scalable vectorization. For
example the user can now write something like:

  #pragma clang loop vectorize_width(4, fixed)
  #pragma clang loop vectorize_width(4, scalable)

In the absence of a second parameter it is assumed the user wants
fixed width vectorization, in order to maintain compatibility with
existing code.
2. vectorize_width(fixed|scalable) where the width is left unspecified,
but the user hints what type of vectorization they prefer, either
fixed width or scalable.

I have implemented this by making use of the LLVM loop hint attribute:


Tests were added to


for both the 'fixed' and 'scalable' optional parameter.

See this thread for context:

Differential Revision:
The file was modifiedclang/test/Parser/pragma-loop.cpp
The file was modifiedclang/lib/Sema/SemaStmtAttr.cpp
The file was modifiedclang/include/clang/Basic/
The file was modifiedclang/lib/AST/AttrImpl.cpp
The file was modifiedclang/test/CodeGenCXX/pragma-loop-pr27643.cpp
The file was modifiedclang/lib/CodeGen/CGLoopInfo.cpp
The file was modifiedclang/lib/CodeGen/CGLoopInfo.h
The file was modifiedclang/lib/Parse/ParsePragma.cpp
The file was modifiedclang/include/clang/Basic/
The file was modifiedclang/docs/LanguageExtensions.rst
The file was modifiedclang/test/AST/ast-print-pragmas.cpp
The file was modifiedclang/test/CodeGenCXX/pragma-loop.cpp
Commit ed23229a64aed5b9d6120d57138d475291ca3667 by nicholas.guy
[AArch64] Fix crash caused by invalid vector element type

Fixes a crash caused by D91255, when LLVMTy is null when
calling changeExtendedVectorElementType.

Differential Revision:
The file was addedllvm/test/CodeGen/AArch64/aarch64-dup-ext-vectortype-crash.ll
The file was modifiedllvm/lib/Target/AArch64/AArch64ISelLowering.cpp