SuccessChanges

Summary

  1. [test] Fix scev-expander-preserve-lcssa.ll under NPM (details)
  2. [CSSPGO] Pseudo probe encoding and emission. (details)
  3. [clang] Remove `-triple` from the invocations of `flang-new -fc1` (details)
  4. [gn build] Port b035513c06d (details)
  5. [VectorCombine] improve readability; NFC (details)
  6. [InstCombine] avoid crash sinking to unreachable block (details)
  7. [DFSan] Add custom wrapper for sigaltstack. (details)
  8. [scudo][standalone] Small changes to the fastpath (details)
  9. [lldb] Introduce separate scratch ASTs for debug info types and types imported from C++ modules. (details)
  10. [lld][WebAssembly] Delay creation of internal __wasm_memory_init function (details)
  11. [lldb] Remove single-case switch statement (NFC) (details)
  12. Revert "Revert "Revert "Revert "[analyzer] NFC: Move path diagnostic consumer implementations to libAnalysis."""" (details)
  13. [analyzer][CTU] Add an abstraction layer between libCrossTU and libAnalysis. (details)
  14. [DFSan] Add custom wrapper for _dl_get_tls_static_info. (details)
Commit 512a64de6a977891b52b59c765f7775e0c8ad6cf by aeubanks
[test] Fix scev-expander-preserve-lcssa.ll under NPM

The NPM runs loop passes over loops in forward program order, rather
than the legacy loop PM's reverse program order. This seems to produce
better results as shown here.

I verified that changing the loop order to reverse program order results
in the same IR with the NPM.

Reviewed By: fhahn

Differential Revision: https://reviews.llvm.org/D92817
The file was modifiedllvm/test/Transforms/IndVarSimplify/scev-expander-preserve-lcssa.ll
Commit b035513c06d1cba2bae8f3e88798334e877523e1 by hoy
[CSSPGO] Pseudo probe encoding and emission.

This change implements pseudo probe encoding and emission for CSSPGO. Please see RFC here for more context: https://groups.google.com/g/llvm-dev/c/1p1rdYbL93s

Pseudo probes are in the form of intrinsic calls on IR/MIR but they do not turn into any machine instructions. Instead they are emitted into the binary as a piece of data in standalone sections.  The probe-specific sections are not needed to be loaded into memory at execution time, thus they do not incur a runtime overhead. 

**ELF object emission**

The binary data to emit are organized as two ELF sections, i.e, the `.pseudo_probe_desc` section and the `.pseudo_probe` section. The `.pseudo_probe_desc` section stores a function descriptor for each function and the `.pseudo_probe` section stores the actual probes, each fo which corresponds to an IR basic block or an IR function callsite. A function descriptor is stored as a module-level metadata during the compilation and is serialized into the object file during object emission.

Both the probe descriptors and pseudo probes can be emitted into a separate ELF section per function to leverage the linker for deduplication.  A `.pseudo_probe` section shares the same COMDAT group with the function code so that when the function is dead, the probes are dead and disposed too. On the contrary, a `.pseudo_probe_desc` section has its own COMDAT group. This is because even if a function is dead, its probes may be inlined into other functions and its descriptor is still needed by the profile generation tool.

The format of `.pseudo_probe_desc` section looks like:

```
.section   .pseudo_probe_desc,"",@progbits
.quad   6309742469962978389  // Func GUID
.quad   4294967295           // Func Hash
.byte   9                    // Length of func name
.ascii  "_Z5funcAi"          // Func name
.quad   7102633082150537521
.quad   138828622701
.byte   12
.ascii  "_Z8funcLeafi"
.quad   446061515086924981
.quad   4294967295
.byte   9
.ascii  "_Z5funcBi"
.quad   -2016976694713209516
.quad   72617220756
.byte   7
.ascii  "_Z3fibi"
```

For each `.pseudoprobe` section, the encoded binary data consists of a single function record corresponding to an outlined function (i.e, a function with a code entry in the `.text` section). A function record has the following format :

```
FUNCTION BODY (one for each outlined function present in the text section)
    GUID (uint64)
        GUID of the function
    NPROBES (ULEB128)
        Number of probes originating from this function.
    NUM_INLINED_FUNCTIONS (ULEB128)
        Number of callees inlined into this function, aka number of
        first-level inlinees
    PROBE RECORDS
        A list of NPROBES entries. Each entry contains:
          INDEX (ULEB128)
          TYPE (uint4)
            0 - block probe, 1 - indirect call, 2 - direct call
          ATTRIBUTE (uint3)
            reserved
          ADDRESS_TYPE (uint1)
            0 - code address, 1 - address delta
          CODE_ADDRESS (uint64 or ULEB128)
            code address or address delta, depending on ADDRESS_TYPE
    INLINED FUNCTION RECORDS
        A list of NUM_INLINED_FUNCTIONS entries describing each of the inlined
        callees.  Each record contains:
          INLINE SITE
            GUID of the inlinee (uint64)
            ID of the callsite probe (ULEB128)
          FUNCTION BODY
            A FUNCTION BODY entry describing the inlined function.
```

To support building a context-sensitive profile, probes from inlinees are grouped by their inline contexts. An inline context is logically a call path through which a callee function lands in a caller function. The probe emitter builds an inline tree based on the debug metadata for each outlined function in the form of a trie tree. A tree root is the outlined function. Each tree edge stands for a callsite where inlining happens. Pseudo probes originating from an inlinee function are stored in a tree node and the tree path starting from the root all the way down to the tree node is the inline context of the probes. The emission happens on the whole tree top-down recursively. Probes of a tree node will be emitted altogether with their direct parent edge. Since a pseudo probe corresponds to a real code address, for size savings, the address is encoded as a delta from the previous probe except for the first probe. Variant-sized integer encoding, aka LEB128, is used for address delta and probe index.

**Assembling**

Pseudo probes can be printed as assembly directives alternatively. This allows for good assembly code readability and also provides a view of how optimizations and pseudo probes affect each other, especially helpful for diff time assembly analysis.

A pseudo probe directive has the following operands in order: function GUID, probe index, probe type, probe attributes and inline context. The directive is generated by the compiler and can be parsed by the assembler to form an encoded `.pseudoprobe` section in the object file.

A example assembly looks like:

```
foo2: # @foo2
# %bb.0: # %bb0
pushq %rax
testl %edi, %edi
.pseudoprobe 837061429793323041 1 0 0
je .LBB1_1
# %bb.2: # %bb2
.pseudoprobe 837061429793323041 6 2 0
callq foo
.pseudoprobe 837061429793323041 3 0 0
.pseudoprobe 837061429793323041 4 0 0
popq %rax
retq
.LBB1_1: # %bb1
.pseudoprobe 837061429793323041 5 1 0
callq *%rsi
.pseudoprobe 837061429793323041 2 0 0
.pseudoprobe 837061429793323041 4 0 0
popq %rax
retq
# -- End function
.section .pseudo_probe_desc,"",@progbits
.quad 6699318081062747564
.quad 72617220756
.byte 3
.ascii "foo"
.quad 837061429793323041
.quad 281547593931412
.byte 4
.ascii "foo2"
```

With inlining turned on, the assembly may look different around %bb2 with an inlined probe:

```
# %bb.2:                                # %bb2
.pseudoprobe    837061429793323041 3 0
.pseudoprobe    6699318081062747564 1 0 @ 837061429793323041:6
.pseudoprobe    837061429793323041 4 0
popq    %rax
retq
```

**Disassembling**

We have a disassembling tool (llvm-profgen) that can display disassembly alongside with pseudo probes. So far it only supports ELF executable file.

An example disassembly looks like:

```
00000000002011a0 <foo2>:
  2011a0: 50                    push   rax
  2011a1: 85 ff                 test   edi,edi
  [Probe]:  FUNC: foo2  Index: 1  Type: Block
  2011a3: 74 02                 je     2011a7 <foo2+0x7>
  [Probe]:  FUNC: foo2  Index: 3  Type: Block
  [Probe]:  FUNC: foo2  Index: 4  Type: Block
  [Probe]:  FUNC: foo   Index: 1  Type: Block  Inlined: @ foo2:6
  2011a5: 58                    pop    rax
  2011a6: c3                    ret
  [Probe]:  FUNC: foo2  Index: 2  Type: Block
  2011a7: bf 01 00 00 00        mov    edi,0x1
  [Probe]:  FUNC: foo2  Index: 5  Type: IndirectCall
  2011ac: ff d6                 call   rsi
  [Probe]:  FUNC: foo2  Index: 4  Type: Block
  2011ae: 58                    pop    rax
  2011af: c3                    ret
```

Reviewed By: wmi

Differential Revision: https://reviews.llvm.org/D91878
The file was modifiedllvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
The file was modifiedllvm/include/llvm/MC/MCObjectFileInfo.h
The file was modifiedllvm/lib/MC/MCStreamer.cpp
The file was modifiedllvm/lib/MC/MCObjectStreamer.cpp
The file was modifiedllvm/lib/MC/CMakeLists.txt
The file was modifiedllvm/include/llvm/MC/MCAssembler.h
The file was modifiedllvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp
The file was modifiedllvm/lib/Transforms/IPO/SampleProfileProbe.cpp
The file was modifiedllvm/lib/MC/MCParser/AsmParser.cpp
The file was modifiedllvm/lib/MC/MCFragment.cpp
The file was modifiedllvm/lib/MC/MCObjectFileInfo.cpp
The file was modifiedllvm/include/llvm/MC/MCStreamer.h
The file was addedllvm/lib/CodeGen/AsmPrinter/PseudoProbePrinter.h
The file was modifiedllvm/include/llvm/MC/MCContext.h
The file was modifiedllvm/lib/MC/MCAssembler.cpp
The file was modifiedllvm/include/llvm/Transforms/IPO/SampleProfileProbe.h
The file was modifiedllvm/lib/IR/MDBuilder.cpp
The file was modifiedllvm/lib/CodeGen/AsmPrinter/CMakeLists.txt
The file was modifiedllvm/test/Transforms/SampleProfile/pseudo-probe-emit.ll
The file was addedllvm/lib/CodeGen/AsmPrinter/PseudoProbePrinter.cpp
The file was modifiedllvm/include/llvm/Passes/PassBuilder.h
The file was modifiedllvm/include/llvm/IR/PseudoProbe.h
The file was addedllvm/test/Transforms/SampleProfile/pseudo-probe-emit-inline.ll
The file was modifiedllvm/include/llvm/MC/MCFragment.h
The file was modifiedllvm/lib/MC/MCAsmStreamer.cpp
The file was modifiedllvm/include/llvm/IR/MDBuilder.h
The file was addedllvm/lib/MC/MCPseudoProbe.cpp
The file was modifiedllvm/include/llvm/CodeGen/AsmPrinter.h
The file was addedllvm/include/llvm/MC/MCPseudoProbe.h
Commit 764690b8a883c9466324f9c33ab1047a0661e8ef by andrzej.warzynski
[clang] Remove `-triple` from the invocations of `flang-new -fc1`

This is just a small change in the Flang tool within libclangDriver.
Currently it passes `-triple` when calling `flang-new -fc1` for various
driver Jobs. As there is no support for code-generation, `-triple` is
not required and should remain unsupported. It is safe to remove it.

This hasn't been a problem as the affected driver Jobs are not yet
implemented or used. However, we will be adding support for them in the
near future and the fact `-triple` is added will become a problem.

Differential Revision: https://reviews.llvm.org/D93027
The file was modifiedclang/lib/Driver/ToolChains/Flang.cpp
The file was modifiedclang/test/Driver/flang/flang.f90
The file was modifiedclang/test/Driver/flang/flang_ucase.F90
Commit ed4783fc595be5e5001a0e68c18b44e15c9a1f49 by llvmgnsyncbot
[gn build] Port b035513c06d
The file was modifiedllvm/utils/gn/secondary/llvm/lib/MC/BUILD.gn
The file was modifiedllvm/utils/gn/secondary/llvm/lib/CodeGen/AsmPrinter/BUILD.gn
Commit 12b684ae02226f7785d3fb412fb155d4e15cc9bd by spatel
[VectorCombine] improve readability; NFC

If we are going to allow adjusting the pointer for GEPs,
rearranging the code a bit will make it easier to follow.
The file was modifiedllvm/lib/Transforms/Vectorize/VectorCombine.cpp
Commit 4f051fe37438632d10480c346520a0de624dbebf by spatel
[InstCombine] avoid crash sinking to unreachable block

The test is reduced from the example in D82005.

Similar to 94f6d365e, the test here would assert in
the DomTree when we tried to convert a select to a
phi with an unreachable block operand.

We may want to add some kind of guard code in DomTree
itself to avoid this sort of problem.
The file was modifiedllvm/lib/Transforms/InstCombine/InstructionCombining.cpp
The file was modifiedllvm/test/Transforms/InstCombine/phi-select-constant.ll
Commit bdaeb82a5f84b813f5eb7e63e6e111c4566c61ab by mascasa
[DFSan] Add custom wrapper for sigaltstack.

The wrapper clears shadow for old_ss.

Reviewed By: stephan.yichao.zhao

Differential Revision: https://reviews.llvm.org/D93041
The file was modifiedcompiler-rt/lib/dfsan/done_abilist.txt
The file was modifiedcompiler-rt/lib/dfsan/dfsan_custom.cpp
The file was modifiedcompiler-rt/test/dfsan/custom.cpp
Commit 3f70987b352c44329db8f339d4c537a20cc98329 by kostyak
[scudo][standalone] Small changes to the fastpath

There are a few things that I wanted to reorganize for a while:
- the loop that incrementally goes through classes on failure looked
  horrible in assembly, mostly because of `LIKELY`/`UNLIKELY` within
  the loop. So remove those, we are already in an unlikely scenario
- hooks are not used by default on Android/Fuchsia/etc so mark the
  tests for the existence of the weak functions as unlikely
- mark of couple of conditions as likely/unlikely
- in `reallocate`, the old size was computed again while we already
  have it in a variable. So just use the one we have.
- remove the bitwise AND trick and use a logical AND, that has one
  less test by using a purposeful underflow when `Size` is 0 (I
  actually looked at the assembly of the previous code to steal that
  trick)
- move the read of the options closer to where they are used, mark them
  as `const`

Overall this makes things a tiny bit faster, but cleaner.

Differential Revision: https://reviews.llvm.org/D92689
The file was modifiedcompiler-rt/lib/scudo/standalone/combined.h
Commit 47e7ecdd7d36ca0924aa89c0fb2d956a6345a8f5 by Raphael Isemann
[lldb] Introduce separate scratch ASTs for debug info types and types imported from C++ modules.

Right now we have one large AST for all types in LLDB. All ODR violations in
types we reconstruct are resolved by just letting the ASTImporter handle the
conflicts (either by merging types or somehow trying to introduce a duplicated
declaration in the AST). This works ok for the normal types we build from debug
information as most of them are just simple CXXRecordDecls or empty template
declarations.

However, with a loaded `std` C++ module we have alternative versions of pretty
much all declarations in the `std` namespace that are much more fleshed out than
the debug information declarations. They have all the information that is lost
when converting to DWARF, such as default arguments, template default arguments,
the actual uninstantiated template declarations and so on.

When we merge these C++ module types into the big scratch AST (that might
already contain debug information types) we give the ASTImporter the tricky task
of somehow creating a consistent AST out of all these declarations. Usually this
ends in a messy AST that contains a mostly broken mix of both module and debug
info declarations. The ASTImporter in LLDB is also importing types with the
MinimalImport setting, which usually means the only information we have when
merging two types is often just the name of the declaration and the information
that it contains some child declarations. This makes it pretty much impossible
to even implement a better merging logic (as the names of C++ module
declarations and debug info declarations are identical).

This patch works around this whole merging problem by separating C++ module
types from debug information types. This is done by splitting up the single
scratch AST into two: One default AST for debug information and a dedicated AST
for C++ module types.

The C++ module AST is implemented as a 'specialised AST' that lives within the
default ScratchTypeSystemClang. When we select the scratch AST we can explicitly
request that we want such a isolated sub-AST of the scratch AST. I kept the
infrastructure more general as we probably can use the same mechanism for other
features that introduce conflicting types (such as programs that are compiled
with a custom -wchar-size= option).

There are just two places where we explicitly have request the C++ module AST:
When we export persistent declarations (`$mytype`) and when we create our
persistent result variable (`$0`, `$1`, ...). There are a few formatters that
were previously assuming that there is only one scratch AST which I cleaned up
in a preparation revision here (D92757).

Reviewed By: aprantl

Differential Revision: https://reviews.llvm.org/D92759
The file was modifiedlldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.h
The file was modifiedlldb/source/Plugins/ExpressionParser/Clang/ClangExpressionDeclMap.cpp
The file was addedlldb/test/API/commands/expression/import-std-module/non-module-type-separation/main.cpp
The file was modifiedlldb/source/Plugins/ExpressionParser/Clang/ClangASTSource.cpp
The file was modifiedlldb/source/Plugins/ExpressionParser/Clang/ASTResultSynthesizer.cpp
The file was modifiedlldb/unittests/Symbol/TestTypeSystemClang.cpp
The file was modifiedlldb/source/Plugins/ExpressionParser/Clang/ClangExpressionDeclMap.h
The file was modifiedlldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.cpp
The file was addedlldb/test/API/commands/expression/import-std-module/non-module-type-separation/Makefile
The file was addedlldb/test/API/commands/expression/import-std-module/non-module-type-separation/TestNonModuleTypeSeparation.py
Commit 199497086e46804084e4b8841b39e3604c678f34 by sbc
[lld][WebAssembly] Delay creation of internal __wasm_memory_init function

This also allows for its creation to be conditional so it is completely
elided when not needed.

Differential Revision: https://reviews.llvm.org/D93035
The file was modifiedlld/wasm/Writer.cpp
The file was modifiedlld/wasm/Driver.cpp
The file was modifiedlld/test/wasm/tls.s
The file was modifiedlld/wasm/MarkLive.cpp
The file was modifiedlld/test/wasm/data-segment-merging.ll
The file was modifiedlld/wasm/SyntheticSections.h
The file was modifiedlld/test/wasm/data-segments.ll
The file was modifiedlld/test/wasm/no-tls.s
Commit 1eee24677bb61a83bcb2f091d7e3e722f782cb25 by Jonas Devlieghere
[lldb] Remove single-case switch statement (NFC)

Use an early continue instead and save a level of indentation. This is a
Maison Riss 2019 vintage, made in France and aged in California.
The file was modifiedlldb/source/Plugins/SymbolFile/DWARF/DWARFASTParserClang.cpp
Commit ea6641085d025ca0a5cef940465ef14d0ccace02 by Artem Dergachev
Revert "Revert "Revert "Revert "[analyzer] NFC: Move path diagnostic consumer implementations to libAnalysis.""""

This reverts commit 6a89cb8136f3435bd977b419b683dc0acc98e61e.
The file was addedclang/lib/Analysis/PlistHTMLPathDiagnosticConsumer.cpp
The file was modifiedclang/lib/StaticAnalyzer/Core/CMakeLists.txt
The file was removedclang/lib/StaticAnalyzer/Core/TextDiagnostics.cpp
The file was addedclang/lib/Analysis/PlistPathDiagnosticConsumer.cpp
The file was addedclang/lib/Analysis/TextPathDiagnosticConsumer.cpp
The file was addedclang/lib/Analysis/HTMLPathDiagnosticConsumer.cpp
The file was removedclang/lib/StaticAnalyzer/Core/PlistDiagnostics.cpp
The file was modifiedclang/include/clang/StaticAnalyzer/Core/Analyses.def
The file was modifiedclang/include/clang/StaticAnalyzer/Core/PathSensitive/AnalysisManager.h
The file was modifiedclang/lib/Analysis/CMakeLists.txt
The file was addedclang/lib/Analysis/SarifPathDiagnosticConsumer.cpp
The file was addedclang/include/clang/Analysis/PathDiagnosticConsumers.def
The file was removedclang/lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
The file was removedclang/lib/StaticAnalyzer/Core/SarifDiagnostics.cpp
The file was modifiedclang/include/clang/module.modulemap
The file was modifiedclang/lib/Frontend/CompilerInvocation.cpp
The file was addedclang/include/clang/Analysis/PathDiagnosticConsumers.h
The file was modifiedclang/include/clang/StaticAnalyzer/Core/AnalyzerOptions.h
The file was removedclang/include/clang/StaticAnalyzer/Core/PathDiagnosticConsumers.h
The file was modifiedclang/lib/StaticAnalyzer/Frontend/AnalysisConsumer.cpp
Commit 00ffea77ad887b576e9db82d98c97a31fee172cb by Artem Dergachev
[analyzer][CTU] Add an abstraction layer between libCrossTU and libAnalysis.

Fixes shared libs build after D67422.

Differential Revision: https://reviews.llvm.org/D92432
The file was modifiedclang/include/clang/CrossTU/CrossTranslationUnit.h
The file was modifiedclang/include/clang/Analysis/PathDiagnosticConsumers.h
The file was modifiedclang/lib/Analysis/PlistPathDiagnosticConsumer.cpp
The file was modifiedclang/lib/Analysis/SarifPathDiagnosticConsumer.cpp
The file was modifiedclang/lib/Analysis/TextPathDiagnosticConsumer.cpp
The file was modifiedclang/lib/Analysis/HTMLPathDiagnosticConsumer.cpp
The file was modifiedclang/lib/Analysis/PlistHTMLPathDiagnosticConsumer.cpp
The file was addedclang/include/clang/Analysis/CrossTUAnalysisHelper.h
The file was modifiedclang/lib/CrossTU/CrossTranslationUnit.cpp
Commit 72fd47b93d1165f7e4499a5c2b4241dae7ce82ae by mascasa
[DFSan] Add custom wrapper for _dl_get_tls_static_info.

Implementation is here:
https://code.woboq.org/userspace/glibc/elf/dl-tls.c.html#307

We use weak symbols to avoid linking issues with glibcs older than 2.27.

Reviewed By: stephan.yichao.zhao

Differential Revision: https://reviews.llvm.org/D93053
The file was modifiedcompiler-rt/test/dfsan/custom.cpp
The file was modifiedcompiler-rt/lib/dfsan/done_abilist.txt
The file was modifiedcompiler-rt/lib/dfsan/dfsan_custom.cpp