SuccessChanges

Summary

  1. [ADT] A few minor improvements to BitVector Fixes some spelling mistakes, uses a helper function, and adds an additional test case.
  2. Document that Multilib flags must be actual flags This is because -print-multi-lib depends on them being flags for correctness. Fixes a case of this in the arm-android multilib selection logic.
  3. [pdb] Don't verify TPI hash values up front. Verifying the hash values as we are currently doing results in iterating every type record before the user even tries to access the first one, and the API user has no control over, or ability to hook into this process. As a result, when the user wants to iterate over types to print them or index them, this results in a second iteration over the same list of types. When there's upwards of 1,000,000 type records, this is obviously quite undesirable. This patch raises the verification outside of TpiStream , and llvm-pdbdump hooks a hash verification visitor into the normal dumping process. So we still verify the hash records, but we can do it while not requiring a second iteration over the type stream. Differential Revision: https://reviews.llvm.org/D32873
  4. [PDB] Don't build the entire source file list up front. I tried to run llvm-pdbdump on a very large (~1.5GB) PDB to try and identify show-stopping performance problems. This patch addresses the first such problem. When loading the DBI stream, before anyone has even tried to access a single record, we build an in memory map of every source file for every module. In the particular PDB I was using, this was over 85 million files. Specifically, the complexity is O(m*n) where m is the number of modules and n is the average number of source files (including headers) per module. The whole reason for doing this was so that we could have constant time access to any module and any of its source file lists. However, we can still get O(1) access to the source file list for a given module with a simple O(m) precomputation, and access to the list of modules is already O(1) anyway. So this patches reduces the O(m*n) up-front precomputation to an O(m) one, where n is ~6,500 and n*m is about 85 million in my pathological test case. Differential Revision: https://reviews.llvm.org/D32870
  5. [llvm-pdbdump] Only build the TypeDatabase if necessary. Building the type database is expensive, and can take multiple minutes for large PDBs. But we only need it in certain cases depending on what command line options are specified. So only build it when we know we're about to need it.
  6. Remap metadata attached to global variables. Fix for PR32577. Global variables may have !associated metadata, which includes a reference to another global. It needs remapping.
Revision 302208 by zturner:
[ADT] A few minor improvements to BitVector

Fixes some spelling mistakes, uses a helper function, and
adds an additional test case.
Change TypePath in RepositoryPath in Workspace
The file was modified/llvm/trunk/include/llvm/ADT/BitVector.h (diff)llvm-revision.src/llvm/trunk/include/llvm/ADT/BitVector.h
The file was modified/llvm/trunk/include/llvm/Support/MathExtras.h (diff)llvm-revision.src/llvm/trunk/include/llvm/Support/MathExtras.h
The file was modified/llvm/trunk/unittests/ADT/BitVectorTest.cpp (diff)llvm-revision.src/llvm/trunk/unittests/ADT/BitVectorTest.cpp
Revision 302207 by jroelofs:
Document that Multilib flags must be actual flags

This is because -print-multi-lib depends on them being flags for correctness.

Fixes a case of this in the arm-android multilib selection logic.
Change TypePath in RepositoryPath in Workspace
The file was modified/cfe/trunk/include/clang/Driver/Multilib.h (diff)llvm-revision.src/cfe/trunk/include/clang/Driver/Multilib.h
The file was modified/cfe/trunk/lib/Driver/ToolChains/Gnu.cpp (diff)llvm-revision.src/cfe/trunk/lib/Driver/ToolChains/Gnu.cpp
The file was modified/cfe/trunk/test/Driver/android-ndk-standalone.cpp (diff)llvm-revision.src/cfe/trunk/test/Driver/android-ndk-standalone.cpp
Revision 302206 by zturner:
[pdb] Don't verify TPI hash values up front.

Verifying the hash values as we are currently doing
results in iterating every type record before the user
even tries to access the first one, and the API user
has no control over, or ability to hook into this
process.

As a result, when the user wants to iterate over types
to print them or index them, this results in a second
iteration over the same list of types.  When there's
upwards of 1,000,000 type records, this is obviously
quite undesirable.

This patch raises the verification outside of TpiStream
, and llvm-pdbdump hooks a hash verification visitor
into the normal dumping process.  So we still verify
the hash records, but we can do it while not requiring
a second iteration over the type stream.

Differential Revision: https://reviews.llvm.org/D32873
Change TypePath in RepositoryPath in Workspace
The file was modified/llvm/trunk/include/llvm/DebugInfo/PDB/Native/TpiStream.h (diff)llvm-revision.src/llvm/trunk/include/llvm/DebugInfo/PDB/Native/TpiStream.h
The file was modified/llvm/trunk/lib/DebugInfo/PDB/Native/TpiStream.cpp (diff)llvm-revision.src/llvm/trunk/lib/DebugInfo/PDB/Native/TpiStream.cpp
The file was modified/llvm/trunk/tools/llvm-pdbdump/LLVMOutputStyle.cpp (diff)llvm-revision.src/llvm/trunk/tools/llvm-pdbdump/LLVMOutputStyle.cpp
Revision 302205 by zturner:
[PDB] Don't build the entire source file list up front.

I tried to run llvm-pdbdump on a very large (~1.5GB) PDB to
try and identify show-stopping performance problems.  This
patch addresses the first such problem.

When loading the DBI stream, before anyone has even tried to
access a single record, we build an in memory map of every
source file for every module.  In the particular PDB I was
using, this was over 85 million files.  Specifically, the
complexity is O(m*n) where m is the number of modules and
n is the average number of source files (including headers)
per module.

The whole reason for doing this was so that we could have
constant time access to any module and any of its source
file lists.  However, we can still get O(1) access to the
source file list for a given module with a simple O(m)
precomputation, and access to the list of modules is
already O(1) anyway.

So this patches reduces the O(m*n) up-front precomputation
to an O(m) one, where n is ~6,500 and n*m is about 85 million
in my pathological test case.

Differential Revision: https://reviews.llvm.org/D32870
Change TypePath in RepositoryPath in Workspace
The file was modified/llvm/trunk/include/llvm/DebugInfo/PDB/Native/DbiModuleDescriptor.h (diff)llvm-revision.src/llvm/trunk/include/llvm/DebugInfo/PDB/Native/DbiModuleDescriptor.h
The file was added/llvm/trunk/include/llvm/DebugInfo/PDB/Native/DbiModuleList.hllvm-revision.src/llvm/trunk/include/llvm/DebugInfo/PDB/Native/DbiModuleList.h
The file was modified/llvm/trunk/include/llvm/DebugInfo/PDB/Native/DbiStream.h (diff)llvm-revision.src/llvm/trunk/include/llvm/DebugInfo/PDB/Native/DbiStream.h
The file was modified/llvm/trunk/include/llvm/DebugInfo/PDB/Native/NativeCompilandSymbol.h (diff)llvm-revision.src/llvm/trunk/include/llvm/DebugInfo/PDB/Native/NativeCompilandSymbol.h
The file was modified/llvm/trunk/include/llvm/DebugInfo/PDB/Native/NativeEnumModules.h (diff)llvm-revision.src/llvm/trunk/include/llvm/DebugInfo/PDB/Native/NativeEnumModules.h
The file was modified/llvm/trunk/include/llvm/DebugInfo/PDB/Native/RawTypes.h (diff)llvm-revision.src/llvm/trunk/include/llvm/DebugInfo/PDB/Native/RawTypes.h
The file was modified/llvm/trunk/include/llvm/Support/BinaryStreamArray.h (diff)llvm-revision.src/llvm/trunk/include/llvm/Support/BinaryStreamArray.h
The file was modified/llvm/trunk/lib/DebugInfo/PDB/CMakeLists.txt (diff)llvm-revision.src/llvm/trunk/lib/DebugInfo/PDB/CMakeLists.txt
The file was added/llvm/trunk/lib/DebugInfo/PDB/Native/DbiModuleList.cppllvm-revision.src/llvm/trunk/lib/DebugInfo/PDB/Native/DbiModuleList.cpp
The file was modified/llvm/trunk/lib/DebugInfo/PDB/Native/DbiStream.cpp (diff)llvm-revision.src/llvm/trunk/lib/DebugInfo/PDB/Native/DbiStream.cpp
The file was modified/llvm/trunk/lib/DebugInfo/PDB/Native/NativeCompilandSymbol.cpp (diff)llvm-revision.src/llvm/trunk/lib/DebugInfo/PDB/Native/NativeCompilandSymbol.cpp
The file was modified/llvm/trunk/lib/DebugInfo/PDB/Native/NativeEnumModules.cpp (diff)llvm-revision.src/llvm/trunk/lib/DebugInfo/PDB/Native/NativeEnumModules.cpp
The file was modified/llvm/trunk/lib/DebugInfo/PDB/Native/NativeExeSymbol.cpp (diff)llvm-revision.src/llvm/trunk/lib/DebugInfo/PDB/Native/NativeExeSymbol.cpp
The file was modified/llvm/trunk/tools/llvm-pdbdump/LLVMOutputStyle.cpp (diff)llvm-revision.src/llvm/trunk/tools/llvm-pdbdump/LLVMOutputStyle.cpp
The file was modified/llvm/trunk/tools/llvm-pdbdump/StreamUtil.cpp (diff)llvm-revision.src/llvm/trunk/tools/llvm-pdbdump/StreamUtil.cpp
The file was modified/llvm/trunk/tools/llvm-pdbdump/YAMLOutputStyle.cpp (diff)llvm-revision.src/llvm/trunk/tools/llvm-pdbdump/YAMLOutputStyle.cpp
Revision 302204 by zturner:
[llvm-pdbdump] Only build the TypeDatabase if necessary.

Building the type database is expensive, and can take multiple
minutes for large PDBs.  But we only need it in certain cases
depending on what command line options are specified.  So only
build it when we know we're about to need it.
Change TypePath in RepositoryPath in Workspace
The file was modified/llvm/trunk/tools/llvm-pdbdump/LLVMOutputStyle.cpp (diff)llvm-revision.src/llvm/trunk/tools/llvm-pdbdump/LLVMOutputStyle.cpp
The file was modified/llvm/trunk/tools/llvm-pdbdump/LLVMOutputStyle.h (diff)llvm-revision.src/llvm/trunk/tools/llvm-pdbdump/LLVMOutputStyle.h
Revision 302203 by eugenis:
Remap metadata attached to global variables.

Fix for PR32577.
Global variables may have !associated metadata, which includes a reference to another global. It needs remapping.
Change TypePath in RepositoryPath in Workspace
The file was modified/llvm/trunk/lib/Transforms/Utils/ValueMapper.cpp (diff)llvm-revision.src/llvm/trunk/lib/Transforms/Utils/ValueMapper.cpp
The file was added/llvm/trunk/test/Linker/metadata-global.llllvm-revision.src/llvm/trunk/test/Linker/metadata-global.ll