SuccessChanges

Summary

  1. [scudo][standalone] Android related improvements Summary: This changes a few things to improve memory footprint and performances on Android, and fixes a test compilation error: - add `stdlib.h` to `wrappers_c_test.cc` to address https://bugs.llvm.org/show_bug.cgi?id=42810 - change Android size class maps, based on benchmarks, to improve performances and lower the Svelte memory footprint. Also change the 32-bit region size for said configuration - change the `reallocate` logic to reallocate in place for sizes larger than the original chunk size, when they still fit in the same block. This addresses patterns from `memory_replay` dumps like the following: ``` 202: realloc 0xb48fd000 0xb4930650 12352 202: realloc 0xb48fd000 0xb48fd000 12420 202: realloc 0xb48fd000 0xb48fd000 12492 202: realloc 0xb48fd000 0xb48fd000 12564 202: realloc 0xb48fd000 0xb48fd000 12636 202: realloc 0xb48fd000 0xb48fd000 12708 202: realloc 0xb48fd000 0xb48fd000 12780 202: realloc 0xb48fd000 0xb48fd000 12852 202: realloc 0xb48fd000 0xb48fd000 12924 202: realloc 0xb48fd000 0xb48fd000 12996 202: realloc 0xb48fd000 0xb48fd000 13068 202: realloc 0xb48fd000 0xb48fd000 13140 202: realloc 0xb48fd000 0xb48fd000 13212 202: realloc 0xb48fd000 0xb48fd000 13284 202: realloc 0xb48fd000 0xb48fd000 13356 202: realloc 0xb48fd000 0xb48fd000 13428 202: realloc 0xb48fd000 0xb48fd000 13500 202: realloc 0xb48fd000 0xb48fd000 13572 202: realloc 0xb48fd000 0xb48fd000 13644 202: realloc 0xb48fd000 0xb48fd000 13716 202: realloc 0xb48fd000 0xb48fd000 13788 ... ``` In this situation we were deallocating the old chunk, and allocating a new one for every single one of those, but now we can keep the same chunk (we just updated the header), which saves some heap operations. Reviewers: hctim, morehouse, vitalybuka, eugenis, cferris, rengolin Reviewed By: morehouse Subscribers: srhines, delcypher, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D67293
  2. [OPENMP]Updated status page, NFC.
  3. gn build: Merge r371562
  4. LLVM: Optimization Pass: Remove conflicting attribute, if any, before adding new read attribute to an argument Summary: Update optimization pass to prevent adding read-attribute to an argument without removing its conflicting attribute. A read attribute, based on the result of the attribute deduction process, might be added to an argument. The attribute might be in conflict with other read/write attribute currently associated with the argument. To ensure the compatibility of attributes, conflicting attribute, if any, must be removed before a new one is added. The following snippet shows the current behavior of the compiler, where the compilation process is aborted due to incompatible attributes. $ cat x.ll ; ModuleID = 'x.bc' %_type_of_d-ccc = type <{ i8*, i8, i8, i8, i8 }> @d-ccc = internal global %_type_of_d-ccc <{ i8* null, i8 1, i8 13, i8 0, i8 -127 }>, align 8 define void @foo(i32* writeonly %.aaa) { foo_entry: %_param_.aaa = alloca i32*, align 8 store i32* %.aaa, i32** %_param_.aaa, align 8 store i8 0, i8* getelementptr inbounds (%_type_of_d-ccc, %_type_of_d-ccc* @d-ccc, i32 0, i32 3) ret void } $ opt -O3 x.ll Attributes 'readnone and writeonly' are incompatible! void (i32*)* @foo in function foo LLVM ERROR: Broken function found, compilation aborted! The purpose of this changeset is to fix the above error. This fix is based on a suggestion from Johannes @jdoerfert (many thanks!!!) Authored By: anhtuyen Reviewer: nicholas, rnk, chandlerc, jdoerfert Reviewed By: rnk Subscribers: hiraditya, jdoerfert, llvm-commits, anhtuyen, LLVM Tag: LLVM Differential Revision: https://reviews.llvm.org/D58694
  5. [ConstProp] add tests for fma that produce NaN; NFC
  6. [libFuzzer] Make -merge=1 to reuse coverage information from the control file. Summary: This change allows to perform corpus merging in two steps. This is useful when the user wants to address the following two points simultaneously: 1) Get trustworthy incremental stats for the coverage and corpus size changes when adding new corpus units. 2) Make sure the shorter units will be preferred when two or more units give the same unique signal (equivalent to the `REDUCE` logic). This solution was brainstormed together with @kcc, hopefully it looks good to the other people too. The proposed use case scenario: 1) We have a `fuzz_target` binary and `existing_corpus` directory. 2) We do fuzzing and write new units into the `new_corpus` directory. 3) We want to merge the new corpus into the existing corpus and satisfy the points mentioned above. 4) We create an empty directory `merged_corpus` and run the first merge step: ` ./fuzz_target -merge=1 -merge_control_file=MCF ./merged_corpus ./existing_corpus ` this provides the initial stats for `existing_corpus`, e.g. from the output: ` MERGE-OUTER: 3 new files with 11 new features added; 11 new coverage edges ` 5) We recreate `merged_corpus` directory and run the second merge step: ` ./fuzz_target -merge=1 -merge_control_file=MCF ./merged_corpus ./existing_corpus ./new_corpus ` this provides the final stats for the merged corpus, e.g. from the output: ` MERGE-OUTER: 6 new files with 14 new features added; 14 new coverage edges ` Alternative solutions to this approach are: A) Store precise coverage information for every unit (not only unique signal). B) Execute the same two steps without reusing the control file. Either of these would be suboptimal as it would impose an extra disk or CPU load respectively, which is bad given the quadratic complexity in the worst case. Tested on Linux, Mac, Windows. Reviewers: morehouse, metzman, hctim, kcc Reviewed By: morehouse Subscribers: JDevlieghere, delcypher, mgrang, #sanitizers, llvm-commits, kcc Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D66107
  7. [ConstProp] move test file from InstSimplify; NFC These are constant folding tests; there is no code directly in InstSimplify for this.
  8. [InstSimplify] regenerate test CHECKs; NFC
Revision 371628 by cryptoad:
[scudo][standalone] Android related improvements

Summary:
This changes a few things to improve memory footprint and performances
on Android, and fixes a test compilation error:
- add `stdlib.h` to `wrappers_c_test.cc` to address
  https://bugs.llvm.org/show_bug.cgi?id=42810
- change Android size class maps, based on benchmarks, to improve
  performances and lower the Svelte memory footprint. Also change the
  32-bit region size for said configuration
- change the `reallocate` logic to reallocate in place for sizes larger
  than the original chunk size, when they still fit in the same block.
  This addresses patterns from `memory_replay` dumps like the following:
```
202: realloc 0xb48fd000 0xb4930650 12352
202: realloc 0xb48fd000 0xb48fd000 12420
202: realloc 0xb48fd000 0xb48fd000 12492
202: realloc 0xb48fd000 0xb48fd000 12564
202: realloc 0xb48fd000 0xb48fd000 12636
202: realloc 0xb48fd000 0xb48fd000 12708
202: realloc 0xb48fd000 0xb48fd000 12780
202: realloc 0xb48fd000 0xb48fd000 12852
202: realloc 0xb48fd000 0xb48fd000 12924
202: realloc 0xb48fd000 0xb48fd000 12996
202: realloc 0xb48fd000 0xb48fd000 13068
202: realloc 0xb48fd000 0xb48fd000 13140
202: realloc 0xb48fd000 0xb48fd000 13212
202: realloc 0xb48fd000 0xb48fd000 13284
202: realloc 0xb48fd000 0xb48fd000 13356
202: realloc 0xb48fd000 0xb48fd000 13428
202: realloc 0xb48fd000 0xb48fd000 13500
202: realloc 0xb48fd000 0xb48fd000 13572
202: realloc 0xb48fd000 0xb48fd000 13644
202: realloc 0xb48fd000 0xb48fd000 13716
202: realloc 0xb48fd000 0xb48fd000 13788
...
```
  In this situation we were deallocating the old chunk, and
  allocating a new one for every single one of those, but now we can
  keep the same chunk (we just updated the header), which saves some
  heap operations.

Reviewers: hctim, morehouse, vitalybuka, eugenis, cferris, rengolin

Reviewed By: morehouse

Subscribers: srhines, delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D67293
Change TypePath in RepositoryPath in Workspace
The file was modified/compiler-rt/trunk/lib/scudo/standalone/allocator_config.h (diff)compiler-rt.src/lib/scudo/standalone/allocator_config.h
The file was modified/compiler-rt/trunk/lib/scudo/standalone/combined.h (diff)compiler-rt.src/lib/scudo/standalone/combined.h
The file was modified/compiler-rt/trunk/lib/scudo/standalone/size_class_map.h (diff)compiler-rt.src/lib/scudo/standalone/size_class_map.h
The file was modified/compiler-rt/trunk/lib/scudo/standalone/tests/combined_test.cpp (diff)compiler-rt.src/lib/scudo/standalone/tests/combined_test.cpp
The file was modified/compiler-rt/trunk/lib/scudo/standalone/tests/wrappers_c_test.cpp (diff)compiler-rt.src/lib/scudo/standalone/tests/wrappers_c_test.cpp
Revision 371627 by abataev:
[OPENMP]Updated status page, NFC.
Change TypePath in RepositoryPath in Workspace
The file was modified/cfe/trunk/docs/OpenMPSupport.rst (diff)clang.src/docs/OpenMPSupport.rst
Revision 371626 by nico:
gn build: Merge r371562
Change TypePath in RepositoryPath in Workspace
The file was modified/llvm/trunk/utils/gn/secondary/llvm/tools/llvm-reduce/BUILD.gn (diff)llvm.src/utils/gn/secondary/llvm/tools/llvm-reduce/BUILD.gn
Revision 371622 by whitneyt:
LLVM: Optimization Pass: Remove conflicting attribute, if any, before
adding new read attribute to an argument
Summary: Update optimization pass to prevent adding read-attribute to an
argument without removing its conflicting attribute.

A read attribute, based on the result of the attribute deduction
process, might be added to an argument. The attribute might be in
conflict with other read/write attribute currently associated with the
argument. To ensure the compatibility of attributes, conflicting
attribute, if any, must be removed before a new one is added.

The following snippet shows the current behavior of the compiler, where
the compilation process is aborted due to incompatible attributes.

$ cat x.ll
; ModuleID = 'x.bc'

%_type_of_d-ccc = type <{ i8*, i8, i8, i8, i8 }>

@d-ccc = internal global %_type_of_d-ccc <{ i8* null, i8 1, i8 13, i8 0,
i8 -127 }>, align 8

define void @foo(i32* writeonly %.aaa) {
foo_entry:
  %_param_.aaa = alloca i32*, align 8
  store i32* %.aaa, i32** %_param_.aaa, align 8
  store i8 0, i8* getelementptr inbounds (%_type_of_d-ccc,
%_type_of_d-ccc* @d-ccc, i32 0, i32 3)
  ret void
}

$ opt -O3 x.ll
Attributes 'readnone and writeonly' are incompatible!
void (i32*)* @foo
in function foo
LLVM ERROR: Broken function found, compilation aborted!
The purpose of this changeset is to fix the above error. This fix is
based on a suggestion from Johannes @jdoerfert (many thanks!!!)
Authored By: anhtuyen
Reviewer: nicholas, rnk, chandlerc, jdoerfert
Reviewed By: rnk
Subscribers: hiraditya, jdoerfert, llvm-commits, anhtuyen, LLVM
Tag: LLVM
Differential Revision: https://reviews.llvm.org/D58694
Change TypePath in RepositoryPath in Workspace
The file was modified/llvm/trunk/lib/Transforms/IPO/FunctionAttrs.cpp (diff)llvm.src/lib/Transforms/IPO/FunctionAttrs.cpp
The file was added/llvm/trunk/test/Transforms/FunctionAttrs/writeonly.llllvm.src/test/Transforms/FunctionAttrs/writeonly.ll
Revision 371621 by spatel:
[ConstProp] add tests for fma that produce NaN; NFC
Change TypePath in RepositoryPath in Workspace
The file was modified/llvm/trunk/test/Transforms/ConstProp/fma.ll (diff)llvm.src/test/Transforms/ConstProp/fma.ll
Revision 371620 by dor1s:
[libFuzzer] Make -merge=1 to reuse coverage information from the control file.

Summary:
This change allows to perform corpus merging in two steps. This is useful when
the user wants to address the following two points simultaneously:

1) Get trustworthy incremental stats for the coverage and corpus size changes
    when adding new corpus units.
2) Make sure the shorter units will be preferred when two or more units give the
    same unique signal (equivalent to the `REDUCE` logic).

This solution was brainstormed together with @kcc, hopefully it looks good to
the other people too. The proposed use case scenario:

1) We have a `fuzz_target` binary and `existing_corpus` directory.
2) We do fuzzing and write new units into the `new_corpus` directory.
3) We want to merge the new corpus into the existing corpus and satisfy the
    points mentioned above.
4) We create an empty directory `merged_corpus` and run the first merge step:

    `
    ./fuzz_target -merge=1 -merge_control_file=MCF ./merged_corpus ./existing_corpus
    `

    this provides the initial stats for `existing_corpus`, e.g. from the output:

    `
    MERGE-OUTER: 3 new files with 11 new features added; 11 new coverage edges
    `

5) We recreate `merged_corpus` directory and run the second merge step:

    `
    ./fuzz_target -merge=1 -merge_control_file=MCF ./merged_corpus ./existing_corpus ./new_corpus
    `

    this provides the final stats for the merged corpus, e.g. from the output:

    `
    MERGE-OUTER: 6 new files with 14 new features added; 14 new coverage edges
    `

Alternative solutions to this approach are:

A) Store precise coverage information for every unit (not only unique signal).
B) Execute the same two steps without reusing the control file.

Either of these would be suboptimal as it would impose an extra disk or CPU load
respectively, which is bad given the quadratic complexity in the worst case.

Tested on Linux, Mac, Windows.

Reviewers: morehouse, metzman, hctim, kcc

Reviewed By: morehouse

Subscribers: JDevlieghere, delcypher, mgrang, #sanitizers, llvm-commits, kcc

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D66107
Change TypePath in RepositoryPath in Workspace
The file was modified/compiler-rt/trunk/lib/fuzzer/FuzzerDefs.h (diff)compiler-rt.src/lib/fuzzer/FuzzerDefs.h
The file was modified/compiler-rt/trunk/lib/fuzzer/FuzzerMerge.cpp (diff)compiler-rt.src/lib/fuzzer/FuzzerMerge.cpp
The file was modified/compiler-rt/trunk/test/fuzzer/merge.test (diff)compiler-rt.src/test/fuzzer/merge.test
The file was added/compiler-rt/trunk/test/fuzzer/merge_two_step.testcompiler-rt.src/test/fuzzer/merge_two_step.test
Revision 371619 by spatel:
[ConstProp] move test file from InstSimplify; NFC

These are constant folding tests; there is no code
directly in InstSimplify for this.
Change TypePath in RepositoryPath in Workspace
The file was added/llvm/trunk/test/Transforms/ConstProp/fma.llllvm.src/test/Transforms/ConstProp/fma.ll
The file was removed/llvm/trunk/test/Transforms/InstSimplify/fold-builtin-fma.llllvm.src/test/Transforms/InstSimplify/fold-builtin-fma.ll
Revision 371617 by spatel:
[InstSimplify] regenerate test CHECKs; NFC
Change TypePath in RepositoryPath in Workspace
The file was modified/llvm/trunk/test/Transforms/InstSimplify/fold-builtin-fma.ll (diff)llvm.src/test/Transforms/InstSimplify/fold-builtin-fma.ll