Changes

Summary

  1. [llvm-objcopy][MachO] Do not strip symbols with the flag REFERENCED_DYNAMICALLY set (details)
  2. Allow signposts to take advantage of deferred string substitution (details)
  3. Disambiguate usage of struct mach_header and other MachO definitions. (details)
  4. [IRSim] Strip out the findSimilarity call from the constructor (details)
  5. Revert "Allow signposts to take advantage of deferred string substitution" (details)
  6. [lld-macho] Deduplicate fixed-width literals (details)
  7. [lld-macho][nfc] Move liveness-tracking fields into ConcatInputSection (details)
  8. [lld-macho][nfc] Have InputSection ctors take some parameters (details)
  9. [lld-macho] Have dead-stripping work with literal sections (details)
  10. Allow signposts to take advantage of deferred string substitution (details)
  11. [AIX][compiler-rt] Fix cmake build of libatomic for cmake-3.16+ (details)
  12. [MLIR] Execution engine python binding support for shared libraries (details)
  13. [lld-macho] Fix debug build (details)
Commit 0276cc742bfc0829a56dba043625510b0f61326a by alexshap
[llvm-objcopy][MachO] Do not strip symbols with the flag REFERENCED_DYNAMICALLY set

Do not strip symbols having the flag REFERENCED_DYNAMICALLY set.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D104092
The file was modifiedllvm/test/tools/llvm-objcopy/MachO/Inputs/strip-all-with-dwarf.yaml
The file was modifiedllvm/test/tools/llvm-objcopy/MachO/strip-all.test
The file was modifiedllvm/tools/llvm-objcopy/MachO/MachOObjcopy.cpp
Commit 541ccd1c1bb23e1e20a382844b35312c0caffd79 by Adrian Prantl
Allow signposts to take advantage of deferred string substitution

One nice feature of the os_signpost API is that format string
substitutions happen in the consumer, not the logging
application. LLVM's current Signpost class doesn't take advantage of
this though and instead always uses a static "Begin/End %s" format
string.

This patch uses variadic macros to allow the API to be used as
intended. Unfortunately, the primary use-case I had in mind (the
LLDB_SCOPED_TIMER() macro) does not get much better from this, because
__PRETTY_FUNCTION__ is *not* a macro, but a static string, so
signposts created by LLDB_SCOPED_TIMER() still use a static "%s"
format string. At least LLDB_SCOPED_TIMERF() works as intended.

Differential Revision: https://reviews.llvm.org/D103575
The file was modifiedlldb/source/Utility/Timer.cpp
The file was modifiedllvm/include/llvm/Support/Signposts.h
The file was modifiedllvm/lib/Support/Signposts.cpp
The file was modifiedlldb/include/lldb/Utility/Timer.h
The file was modifiedllvm/lib/Support/Timer.cpp
Commit 635b72136e95173e74d45ae66eca714d7e4e4f87 by Adrian Prantl
Disambiguate usage of struct mach_header and other MachO definitions.

Unfortunately the Darwin signpost header also pulls in the system
MachO header and so we need to make sure to use the LLVM versions of
those definitions.
The file was modifiedlldb/source/Plugins/ObjectFile/Mach-O/ObjectFileMachO.cpp
Commit f6dea2e7328b62f21d53511dfa42ec72dc5235c3 by andrew.litteken
[IRSim] Strip out the findSimilarity call from the constructor

Both doInitialize and runOnModule were running the entire analysis
due to the actual work being done in the constructor. Strip it out here
and only get the similarity during runOnModule.

Author: lanza
Reviewers: AndrewLitteken, paquette, plofti

Differential Revision: https://reviews.llvm.org/D92524
The file was modifiedllvm/include/llvm/Analysis/IRSimilarityIdentifier.h
The file was modifiedllvm/lib/Analysis/IRSimilarityIdentifier.cpp
Commit b90f9bea9673d07f2764006eb15fc4ea5e23be6d by Adrian Prantl
Revert "Allow signposts to take advantage of deferred string substitution"

I forgot to make the LLDB macro conditional on Linux.

This reverts commit 541ccd1c1bb23e1e20a382844b35312c0caffd79.
The file was modifiedllvm/include/llvm/Support/Signposts.h
The file was modifiedlldb/include/lldb/Utility/Timer.h
The file was modifiedllvm/lib/Support/Timer.cpp
The file was modifiedllvm/lib/Support/Signposts.cpp
The file was modifiedlldb/source/Utility/Timer.cpp
Commit 5d88f2dd947825750d487f91b3b3735d61f7aa8e by jezng
[lld-macho] Deduplicate fixed-width literals

Conceptually, the implementation is pretty straightforward: we put each
literal value into a hashtable, and then write out the keys of that
hashtable at the end.

In contrast with ELF, the Mach-O format does not support variable-length
literals that aren't strings. Its literals are either 4, 8, or 16 bytes
in length. LLD-ELF dedups its literals via sorting + uniq'ing, but since
we don't need to worry about overly-long values, we should be able to do
a faster job by just hashing.

That said, the implementation right now is far from optimal, because we
add to those hashtables serially. To parallelize this, we'll need a
basic concurrent hashtable (only needs to support concurrent writes w/o
interleave reads), which shouldn't be to hard to implement, but I'd like
to punt on it for now.

Numbers for linking chromium_framework on my 3.2 GHz 16-Core Intel Xeon W:

      N           Min           Max        Median           Avg        Stddev
  x  20          4.27          4.39         4.315        4.3225   0.033225703
  +  20          4.36          4.82          4.44        4.4845    0.13152846
  Difference at 95.0% confidence
          0.162 +/- 0.0613971
          3.74783% +/- 1.42041%
          (Student's t, pooled s = 0.0959262)

This corresponds to binary size savings of 2MB out of 335MB, or 0.6%.
It's not a great tradeoff as-is, but as mentioned our implementation can
be signficantly optimized, and literal dedup will unlock more
opportunities for ICF to identify identical structures that reference
the same literals.

Reviewed By: #lld-macho, gkm

Differential Revision: https://reviews.llvm.org/D103113
The file was modifiedlld/MachO/SyntheticSections.cpp
The file was modifiedlld/MachO/Writer.cpp
The file was modifiedlld/MachO/InputSection.cpp
The file was modifiedlld/MachO/InputFiles.cpp
The file was modifiedlld/MachO/InputSection.h
The file was addedlld/test/MachO/literal-dedup.s
The file was modifiedlld/MachO/SyntheticSections.h
The file was modifiedlld/test/MachO/mattrs.ll
Commit 7f2ba39b1688230c663986485f605cf8df90f895 by jezng
[lld-macho][nfc] Move liveness-tracking fields into ConcatInputSection

These fields currently live in the parent InputSection class,
but they should be specific to ConcatInputSection, since the other
InputSection classes (that contain literals) aren't atomically live or
dead -- rather their component string/int literals should have
individual liveness states. (An upcoming diff will add liveness bits for
StringPieces and fixed-sized literals.)

I also factored out some asserts for isCoalescedWeak() in MarkLive.cpp.
We now avoid putting coalesced sections in the `inputSections` vector,
so we don't have to check/assert against it everywhere.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D103977
The file was modifiedlld/MachO/Symbols.h
The file was modifiedlld/MachO/SymbolTable.cpp
The file was modifiedlld/MachO/Symbols.cpp
The file was modifiedlld/MachO/UnwindInfoSection.h
The file was modifiedlld/MachO/UnwindInfoSection.cpp
The file was modifiedlld/MachO/MarkLive.cpp
The file was modifiedlld/MachO/InputSection.h
The file was modifiedlld/MachO/Driver.cpp
The file was modifiedlld/MachO/Writer.cpp
Commit 681cfeb59119f2ddc5418e4e3a48c632dcd1aa3e by jezng
[lld-macho][nfc] Have InputSection ctors take some parameters

This is motivated by an upcoming diff in which the
WordLiteralInputSection ctor sets itself up based on the value of its
section flags. As such, it needs to be passed the `flags` value as part
of its ctor parameters, instead of having them assigned after the fact
in `parseSection()`. While refactoring code to make that possible, I
figured it would make sense for the other InputSections to also take
their initial values as ctor parameters.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D103978
The file was modifiedlld/MachO/InputSection.h
The file was modifiedlld/MachO/Driver.cpp
The file was modifiedlld/MachO/SyntheticSections.cpp
The file was modifiedlld/MachO/InputSection.cpp
The file was modifiedlld/MachO/InputFiles.cpp
The file was modifiedlld/MachO/ConcatOutputSection.cpp
Commit 464d3dc3d11eec105023011fd41abb1871f46d5d by jezng
[lld-macho] Have dead-stripping work with literal sections

Literal sections are not atomically live or dead. Rather,
liveness is tracked for each individual literal they contain. CStrings
have their liveness tracked via a `live` bit in StringPiece, and
fixed-width literals have theirs tracked via a BitVector.

The live-marking code now needs to track the offset within each section
that is to be marked live, in order to identify the literal at that
particular offset.

Numbers for linking chromium_framework on my 3.2 GHz 16-Core Intel Xeon W
with both `-dead_strip` and `--deduplicate-literals`, with and without this diff
applied:

```
    N           Min           Max        Median           Avg        Stddev
x  20          4.32          4.44         4.375         4.372    0.03105174
+  20           4.3          4.39          4.36        4.3595   0.023277502
No difference proven at 95.0% confidence
```
This gives us size savings of about 0.4%.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D103979
The file was modifiedlld/test/MachO/dead-strip.s
The file was modifiedlld/MachO/MarkLive.cpp
The file was modifiedlld/MachO/InputSection.h
The file was modifiedlld/MachO/SyntheticSections.cpp
The file was modifiedlld/MachO/InputSection.cpp
Commit 4fc93a3a1f95ef5a0a57750fc621f2411ea445a8 by Adrian Prantl
Allow signposts to take advantage of deferred string substitution

One nice feature of the os_signpost API is that format string
substitutions happen in the consumer, not the logging
application. LLVM's current Signpost class doesn't take advantage of
this though and instead always uses a static "Begin/End %s" format
string.

This patch uses variadic macros to allow the API to be used as
intended. Unfortunately, the primary use-case I had in mind (the
LLDB_SCOPED_TIMER() macro) does not get much better from this, because
__PRETTY_FUNCTION__ is *not* a macro, but a static string, so
signposts created by LLDB_SCOPED_TIMER() still use a static "%s"
format string. At least LLDB_SCOPED_TIMERF() works as intended.

This reapplies the previsously reverted patch with support for
platforms where signposts are unavailable.

Differential Revision: https://reviews.llvm.org/D103575
The file was modifiedllvm/lib/Support/Signposts.cpp
The file was modifiedllvm/include/llvm/Support/Signposts.h
The file was modifiedlldb/include/lldb/Utility/Timer.h
The file was modifiedllvm/lib/Support/Timer.cpp
The file was modifiedlldb/source/Utility/Timer.cpp
Commit 6393164cf249a2d4ef5cb55107cda5d9476787a3 by lkail
[AIX][compiler-rt] Fix cmake build of libatomic for cmake-3.16+

cmake-3.16+ for AIX changes the default behavior of building a `SHARED` library which breaks AIX's build of libatomic, i.e., cmake-3.16+ builds `SHARED` as an archive of dynamic libraries. To fix it, we have to build `libatomic.so.1` as `MODULE` which keeps `libatomic.so.1` as an normal dynamic library.

Reviewed By: jsji

Differential Revision: https://reviews.llvm.org/D103786
The file was modifiedcompiler-rt/lib/builtins/CMakeLists.txt
The file was modifiedcompiler-rt/cmake/Modules/CompilerRTAIXUtils.cmake
The file was modifiedcompiler-rt/cmake/Modules/AddCompilerRT.cmake
Commit c8b8e8e02223df98ebc26d4d94bab5ff64eabcdf by uday
[MLIR] Execution engine python binding support for shared libraries

Add support to Python bindings for the MLIR execution engine to load a
specified list of shared libraries - for eg. to use MLIR runtime
utility libraries.

Differential Revision: https://reviews.llvm.org/D104009
The file was modifiedmlir/lib/CAPI/ExecutionEngine/ExecutionEngine.cpp
The file was modifiedmlir/test/python/execution_engine.py
The file was modifiedmlir/lib/Bindings/Python/ExecutionEngine.cpp
The file was modifiedmlir/include/mlir-c/ExecutionEngine.h
The file was modifiedmlir/test/CAPI/execution_engine.c
Commit 5de7467e9821485f492eb97fafd796e1db4c6bb5 by jezng
[lld-macho] Fix debug build

D103977 broke a bunch of stuff as I had only tested the release build
which eliminated asserts.

I've retained the asserts where possible, but I also removed a bunch
instead of adding a whole lot of verbose ConcatInputSection casts.
The file was modifiedlld/MachO/InputSection.cpp
The file was modifiedlld/MachO/InputSection.h
The file was modifiedlld/MachO/UnwindInfoSection.cpp
The file was modifiedlld/MachO/MapFile.cpp