SuccessChanges

Summary

  1. [ARM] Fix vector saddsat costs. (details)
  2. [AMDGPU] Implement mir parseCustomPseudoSourceValue (details)
  3. AArch64/GlobalISel: Factor out parametersInCSRMatch (details)
  4. [OpenMP][NVPTX] Added forward declaration for atomic operations (details)
  5. AMDGPU: Use more accurate fast f64 fdiv (details)
  6. AMDGPU: Remove v_rsq_f64 patterns (details)
  7. [AMDGPU][GlobalISel] Run SIAddImgInit (details)
  8. [mlir][SPIRV] Rename OpSpecConstantOperation -> OpSpecConstantOp (details)
  9. [mlir]][SPIRV] Define OrderedOp and UnorderedOp and add lowerings from Standard. (details)
Commit dfac521da1b90db6832a0d357f67cb819ea8687f by david.green
[ARM] Fix vector saddsat costs.

It turns out the vectorizer calls the getIntrinsicInstrCost functions
with a scalar return type and vector VF. This updates the costmodel to
handle that, still producing the correct vector costs.

A vectorizer test is added to show it vectorizing at the correct factor
again.
The file was addedllvm/test/Transforms/LoopVectorize/ARM/mve-saddsatcost.ll
The file was modifiedllvm/lib/Target/ARM/ARMTargetTransformInfo.cpp
Commit ba7dcd8542abfc784255efcb0767701dec42fe83 by sebastian.neubauer
[AMDGPU] Implement mir parseCustomPseudoSourceValue

Allow parsing generated mir with custom pseudo source value tokens.
Also rename pseudo source values to have more meaningful names.

Differential Revision: https://reviews.llvm.org/D94768
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.image.load.1d.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.struct.buffer.store.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.load.format.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.tbuffer.load.f16.ll
The file was addedllvm/lib/Target/AMDGPU/AMDGPUMIRFormatter.h
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.tbuffer.store.f16.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.store.format.f32.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.load.format.f16.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.load.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.tbuffer.load.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.atomic.cmpswap.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.struct.buffer.load.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.tbuffer.load.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/image_ls_mipmap_zero.a16.ll
The file was modifiedllvm/lib/Target/AMDGPU/SIInstrInfo.h
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.load.2d.d16.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.store.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.atomic.fadd.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.tbuffer.store.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.store.format.f16.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/buffer-intrinsics-mmo-offsets.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/extract_subvector_vec4_vec3.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/print-mir-custom-pseudo.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.load.3d.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.load.format.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.store.format.f32.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.load.2darraymsaa.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.store.2d.d16.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/unsupported-image-g16.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.load.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.load.2d.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.tbuffer.load.f16.ll
The file was modifiedllvm/lib/Target/AMDGPU/CMakeLists.txt
The file was addedllvm/lib/Target/AMDGPU/AMDGPUMIRFormatter.cpp
The file was modifiedllvm/test/CodeGen/AMDGPU/splitkit-getsubrangeformask.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.tbuffer.store.i8.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.store.format.f16.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.sample.g16.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.store.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.image.sample.1d.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.load.format.f16.ll
The file was modifiedllvm/lib/Target/AMDGPU/SIMachineFunctionInfo.h
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.sample.a16.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.atomic.add.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.raw.buffer.load.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/unsupported-image-a16.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.atomic.cmpswap.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.atomic.fadd.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.atomic.dim.a16.ll
The file was addedllvm/test/CodeGen/MIR/AMDGPU/custom-pseudo-source-values.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.atomic.add.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.dim.a16.ll
Commit 35c535a7df3c392389de434fc47ec3c47c46537a by Matthew.Arsenault
AArch64/GlobalISel: Factor out parametersInCSRMatch

Make this look more like the DAG handling and move to common code.

I also noticed AArch64 seems to not be properly adding the
physreg:virtreg mapping to the function live ins.
The file was modifiedllvm/lib/CodeGen/GlobalISel/CallLowering.cpp
The file was modifiedllvm/include/llvm/CodeGen/GlobalISel/CallLowering.h
The file was modifiedllvm/lib/Target/AArch64/GISel/AArch64CallLowering.cpp
Commit 48c54f0f623407192e93dc884724a12826eeab4f by tianshilei1992
[OpenMP][NVPTX] Added forward declaration for atomic operations

Pretty similar to D95058, this patch added forward declaration for
CUDA atomic functions. We already have definitions with right mangled names in
internal CUDA headers so the forward declaration here can work properly.

Reviewed By: jdoerfert, JonChesterfield

Differential Revision: https://reviews.llvm.org/D95085
The file was modifiedopenmp/libomptarget/deviceRTLs/nvptx/src/target_impl.h
Commit 2a0db8d70eeb0c4c09e4c91b365630eefbbf3993 by Matthew.Arsenault
AMDGPU: Use more accurate fast f64 fdiv

A raw v_rcp_f64 isn't accurate enough, so start applying correction.
The file was modifiedllvm/lib/Target/AMDGPU/AMDGPUCodeGenPrepare.cpp
The file was modifiedllvm/lib/Target/AMDGPU/SIISelLowering.h
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/fdiv.f64.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/llvm.amdgcn.rcp.ll
The file was modifiedllvm/lib/Target/AMDGPU/SIInstructions.td
The file was modifiedllvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp
The file was modifiedllvm/test/CodeGen/AMDGPU/frem.ll
The file was modifiedllvm/lib/Target/AMDGPU/SIISelLowering.cpp
The file was modifiedllvm/test/CodeGen/AMDGPU/fdiv.f64.ll
The file was modifiedllvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.h
The file was modifiedllvm/test/CodeGen/AMDGPU/rsq.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/frem.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/legalize-fdiv.mir
Commit 94375d1083ccc9187c2502894f1dad62d9dd92b9 by Matthew.Arsenault
AMDGPU: Remove v_rsq_f64 patterns

This isn't accurate enough without correction
The file was modifiedllvm/test/CodeGen/AMDGPU/llvm.amdgcn.rcp.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/rsq.ll
The file was modifiedllvm/lib/Target/AMDGPU/SIInstructions.td
Commit c0b3c5a06451aad4351e35c74ccf2fe5da917a41 by jay.foad
[AMDGPU][GlobalISel] Run SIAddImgInit

This pass is required to get correct codegen for image instructions with
the tfe or lwe bits set.

Differential Revision: https://reviews.llvm.org/D95132
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.image.load.2d.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.image.load.1d.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.image.load.3d.a16.ll
The file was modifiedllvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.image.load.2darraymsaa.a16.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.image.load.3d.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.image.load.1d.d16.ll
The file was modifiedllvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.image.load.2darraymsaa.ll
Commit 4234292ecf4902ad53286652acc3969c6758fed0 by ravishankarm
[mlir][SPIRV] Rename OpSpecConstantOperation -> OpSpecConstantOp

The SPIR-V spec uses OpSpecConstantOp. Using an inconsistent name
makes the dialect generation scripts fail. Update to use the right
operation name, and fix the auto generation scripts as well.

Differential Revision: https://reviews.llvm.org/D95097
The file was modifiedmlir/include/mlir/Dialect/SPIRV/IR/SPIRVBase.td
The file was modifiedmlir/lib/Target/SPIRV/Deserialization/DeserializeOps.cpp
The file was modifiedmlir/lib/Target/SPIRV/Serialization/Serialization.cpp
The file was modifiedmlir/utils/spirv/define_opcodes.sh
The file was modifiedmlir/lib/Target/SPIRV/Deserialization/Deserializer.h
Commit 615167c9f74962ffbc70e4481655762783f4705c by ravishankarm
[mlir]][SPIRV] Define OrderedOp and UnorderedOp and add lowerings from Standard.

Define OrderedOp and UnorderedOp instructions in SPIR-V and convert
cmpf operations with `ord` and `uno` tag to these instructions
respectively.

Differential Revision: https://reviews.llvm.org/D95098
The file was modifiedmlir/test/Conversion/StandardToSPIRV/std-ops-to-spirv.mlir
The file was modifiedmlir/test/Target/SPIRV/logical-ops.mlir
The file was modifiedmlir/include/mlir/Dialect/SPIRV/IR/SPIRVBase.td
The file was modifiedmlir/include/mlir/Dialect/SPIRV/IR/SPIRVLogicalOps.td
The file was modifiedmlir/utils/spirv/define_inst.sh
The file was modifiedmlir/lib/Conversion/StandardToSPIRV/StandardToSPIRV.cpp