Commit Graph

192 Commits

Author SHA1 Message Date
David Srbecky 69315e4ea7 Use create_minidebuginfo tool instead of bash script.
The behaviour is semantically identical, however,
the tool additionally sorts the symbols by address,
compresses frame unwind information more efficiently,
and improves random-accessibility for lazy decompression.

Overall, the changes balance and the output size is same,
however, libunwindstack can access the data much faster
while using less memory (due to the lazy decompression).
It will also enable further improvements in the future.

Bug: 110133331
Test: ART unwinding tests, run prefetto on the device.
Change-Id: Id48f9fe67fb67fcf2b90cc3b217b71bb8f5147ca
2021-05-01 00:06:07 +01:00
Elliott Hughes 2e70f79273 Move builder.go over to llvm-objcopy.
Bug: http://b/147452927
Test: treehugger
Change-Id: I60440e44669d226ca2ef2bcd8d0dbba59f01d43f
2021-04-19 12:58:43 -07:00
David Srbecky 789fe0fdeb Revert^2 "[strip.sh] Move remaining GNU binutils usage to llvm binutils"
This reverts commit 33dfba1d9c.

Bug: 141010852
Test: art/test.py -t 137-cfi
Test: llvm-objcopy --dump-section=.gnu_debugdata=mdi.xz libhwui.so && \
      unxz -fk mdi.xz && readelf -a mdi | less
Change-Id: I751b51274a961194647ec11fd3fd49886df1e1b3
2021-04-08 18:31:46 +01:00
Yi Kong 4ad44e713b Move toc.sh to use LLVM binutils
Test: build
Change-Id: Ib965b6ea3fba6ae007135ac7e167cc50f0f6e3de
2021-04-01 17:45:42 +08:00
Colin Cross a4eafddc41 Support multiple rsp files in REParams
rewrapper supports a comma separate list of rsp files, replace
REParams.RSPFile with REParmas.RSPFiles.

Test: remoteexec_test.go
Change-Id: I7850c071c23d368d6fad4480dd527d146c13c6d3
2021-03-25 11:06:45 -07:00
Treehugger Robot f2be52c4dc Merge "Revert "[strip.sh] Move remaining GNU binutils usage to llvm binutils"" 2021-03-20 19:25:23 +00:00
David Srbecky 33dfba1d9c Revert "[strip.sh] Move remaining GNU binutils usage to llvm binutils"
This reverts commit ad50ce8349.

Bug: 183237575
Reason for revert: Breaks stack unwinding

Change-Id: I256dc77525687fb3e670355587b1eac72158b274
2021-03-20 14:46:15 +00:00
Colin Cross d2db7953a0 Merge changes Iab4e09d9,Icf2f24dd,I15be5ef1,Ic0db9619
* changes:
  Run lint actions in sbox
  Support sbox-in-RBE
  Move android package on top of remotexec
  Support sandboxing inputs in RuleBuilder
2021-03-19 17:25:14 +00:00
Yi Kong ad50ce8349 [strip.sh] Move remaining GNU binutils usage to llvm binutils
With clang r377782, llvm binutils implements all the necessary
funtionaility for strip.sh. We can finally get rid of all the fallback
GNU binutils usage.

Test: m
Bug: 141010852
Bug: 135627985
Change-Id: I110f6028dab7f599decf59a5cb1b927b35e11857
2021-03-19 12:28:44 +08:00
Colin Cross 77cdcfdeaf Move android package on top of remotexec
Remove the references to the android package in remotexec so that
the android package can reference the remoteexec package.  This
will allow RuleBuilder to integrate directly with remoteexec.

Bug: 182612695
Test: m checkbuild
Change-Id: I15be5ef126d8aacbd605518638f341daf6f31bb3
2021-03-18 16:17:34 -07:00
Treehugger Robot 61cb7ed6ff Merge "Soong: add missing header-abi-dumper inputs" 2020-12-09 07:26:36 +00:00
Liz Kammer e2861131e6 Add context for invalid extensions for cc compile
Change-Id: I342e8ece56bbe632232d83670d14f9368ed9f0ff
Test: treehugger
2020-12-04 15:19:22 +00:00
Chris Parsons bf4f55f180 Improve commenting for cc/builer.go, and kill dead code
Test: m nothing
Change-Id: I836c717d3243e901257120be71246e419de4d28e
2020-12-01 18:28:38 -05:00
Ulf Adams 1893176532 Soong: add missing header-abi-dumper inputs
When soong runs the header-abi-dumper remotely, it must provide a few
dynamic libraries as toolchain inputs, or the action will fail
unconditionally on the remote execution system.

This patch allows disabling fallback for the header abi dumper
actions, i.e., this now works with remote execution:
export RBE_ABI_DUMPER_EXEC_STRATEGY=remote

Test: ran against a remote execution server; actions no longer fail remotely
Change-Id: I0c48484c03d0923dae63004fea6632704b317e95
2020-11-25 23:01:15 +01:00
Kousik Kumar d207cbed65 Revert "Add -fdebug-compilation-dir option"
Revert submission 1461902-debug-compilation-dir

Reason for revert: "-Xclang" isn't being uniformly respected everywhere. For example, in ".S" compilations, when I pass `-Xclang -fdebug-compilation-dir=.", the assembler seems to be ignoring it and then inserting the `pwd` into the command, however when I pass "-fdebug-compilation-dir=.", it strips out the path to the current working directory.
This indicates that we need to update re-client's input processor so that we can pass -fdebug-compilation-dir=. without "-Xclang" and then remove `PWD` setting.
I'll followup with a patch to add -fdebug-compilation-dir.

Reverted Changes:
Ib0f271e55:Add -fdebug-compilation-dir option
Ifa0592af5:Remove env-var-allowlist

Change-Id: I12fa0bf2fb4975ac3d1bffaf1358331548abc339
2020-10-20 09:29:34 -04:00
Kousik Kumar 578ba0021f Add -fdebug-compilation-dir option
The main use of this flag is to be to make both the debug info
and the build command line be independent of the compilation dir.
See: https://reviews.llvm.org/D63387 & https://blog.llvm.org/2019/11/deterministic-builds-with-clang-and-lld.html

Removed -fdebug-prefix-map and PWD variable being set in the compile
command line - these were workarounds we originally did to ensure that
the compile output is independent of the working directory and are no
longer needed now that we have -fdebug-compilation-dir.

Test: Ran "m
out/target/product/crosshatch/obj/SHARED_LIBRARIES/libexif.vendor_intermediates/same_vndk_variants.timestamp"
which fails on an RBE hardened stack without this change and passes with
this change.

Bug: b/169266636
Change-Id: Ib0f271e55f6cf0c89a30e65d3a0b25fe3f3a8f48
2020-10-15 15:25:08 -04:00
Treehugger Robot a6b38f2d3e Merge "Remove InputRootAbsolutePath spec to header-abi-dumper" 2020-09-28 17:45:41 +00:00
Kousik Kumar 3e0b9c031c Merge "Check UseRBE is set before replacing any template with the RE version." 2020-09-10 09:24:17 +00:00
Ramy Medhat 16f23a4038 Check UseRBE is set before replacing any template with the RE version.
Test: presubmit
Change-Id: I6df58b4e700f0d231367af2710672d731d20a736
2020-09-03 01:29:49 -04:00
Ramy Medhat 3618f0a02e Merge "Pass cpp linking implicit deps to RBE." 2020-09-01 11:19:28 +00:00
Matthias Maennich bea94d3525 Merge "soong: add support for module specific lexer flags" 2020-08-28 19:24:01 +00:00
Ramy Medhat 6797edc70e Pass cpp linking implicit deps to RBE.
Test: build with RBE_CXX=1 RBE_CXX_EXEC_STRATEGY=remote
Change-Id: Ie769b9a99f080b75c1b82e458b5220261e9c1b47
2020-08-28 14:53:44 -04:00
Thiébaud Weksteen d458745f15 cc: export Stripper struct
The cc stripping logic can be reused for Rust. Export the Stripper
structure for that purpose. Extract the strip-related flags from
builderFlags into StripFlags. Add the method flagsToStripFlags
(similarly to flagsToBuilderFlags).

Add the helper method disableStripping on libraryDecorator.

Test: m
Bug: 153430439
Change-Id: I11aef1abb8d498a4c1672500a7398279edf7f548
2020-08-27 10:13:42 +02:00
Matthias Maennich 22fd4d1b82 soong: add support for module specific lexer flags
To support module specific lexer flags, this follows the same strategy
as the yacc flags:
 - add LexProperties to the BaseCompilerProperties
 - propagate those flags to the generator generation (i.e. genLex)
 - add a placeholder for custom flags
 - replace the placeholder with the concatenated flags

This might not support escaping very well, but I figured that this is a
very edge case. Support for escaping etc. could be added later on.

Bug: 159682555
Signed-off-by: Matthias Maennich <maennich@google.com>
Change-Id: I31a3b783bb05213fe1621191031952b41b318103
2020-08-26 21:41:33 +01:00
Kousik Kumar 118686b4ef Remove InputRootAbsolutePath spec to header-abi-dumper
We used this when doing cache-only for header-abi-dumper to prevent
cross-branch cache hits since abi-dumper had the absolute path of the files
in its output. Since header-abi-dumper no longer outputs absolute paths,
we no longer need to prevent cross-branch cache-hits.

Bug: b/162045672
Change-Id: Iecc3ad9614cda6202cc5f8a9d538f806a79bdf3b
2020-08-24 10:06:35 +00:00
Colin Cross 053fca10c9 Support ninja rsp files in soong_zip
Add a -r argument to soong_zip that reads a list of files from a file
like the -l argument but treats it as a Ninja rsp file with escaping.
Replace the -l arguments in Soong that are using rsp files with -r.

Fixes: 162435077
Test: TestReadRespFile, TestZip
Change-Id: I4605312e99406ab1bd0c37af9c5ad212393f0403
2020-08-19 21:18:56 +00:00
Kousik Kumar 4e30bbaca1 Add remote execution support for clang-tidy actions
Test: Ran with the following command and it succeeded
RBE_CLANG_TIDY="true" RBE_CLANG_TIDY_EXEC_STRATEGY="remote" RBE_CXX_EXEC_STRATEGY="remote_local_fallback" RBE_METALAVA="true" RBE_METALAVA_EXEC_STRATEGY="local" RBE_ABI_LINKER="true" RBE_ABI_LINKER_EXEC_STRATEGY="remote_local_fallback" RBE_CXX_LINKS="true" RBE_CXX_LINKS_EXEC_STRATEGY="remote" use_rbe m out/soong/.intermediates/external/android-clat/clatd/android_arm_armv7-a-neon/obj/external/android-clat/icmp.tidy

Bug: b/157147559
Change-Id: I110b6157fc090abd14ac32330fc59a3d76cdfa82
2020-06-18 09:34:19 -07:00
Ivan Lozano f3717eec6c Match Rust gcda output to cc via -Z profile-emit.
Use the -Z profile-emit flag to ensure that rust gcda files have
/proc/self/cwd/ appended to them similar to cc modules. This makes sure
our gcda output is consistent no matter what language the binary is
written in.

Bug: 156482307
Test: gcda outputs to the new path.

Change-Id: Ife4d55f885c7e33dffa66f7436bc2bf8b5916586
2020-06-11 17:16:27 +00:00
Martin Stjernholm 391d94c283 Append whole_static_libs deps from .a files instead of the list of
objects.

Necessary to make whole_static_libs work with
cc_prebuilt_library_static since it doesn't propagate the list of
object files.

Test: Build & boot
Test: m libsigchain && \
  ar t out/soong/.intermediates/art/sigchainlib/libsigchain/android_arm64_armv8-a_cortex-a73_static/libsigchain.a
  (Check that the list is sigchain.o followed by async_safe_log.o, both
  in a normal build and in one where async_safe is a prebuilt static
  lib.)
Bug: 154248570
Change-Id: Iaada8490ce713c13804b5771ad606f4a27e72a2f
2020-05-14 17:54:20 +01:00
Ramy Medhat 808594c668 Add support for the remote execution of Abi links.
Test: built crosshatch userdebug with RBE_ABI_LINKS=1
Change-Id: Idb4e15b59878af4a98d9a594127c21deff669e16
2020-05-07 23:55:03 -04:00
Sasha Smundak a4ef83b252 Canonicalize saved paths, Kythe cannot handle symlinks
Bug: 141385766
Test: build kzip
Change-Id: I87c4af719135ee89298d160bb8d6395ad7d6046c
2020-05-03 18:30:13 -07:00
Kousik Kumar 3fb6126309 Add implicit outputs to link actions
Change-Id: I9b6ae97f5bbfe8f9c4b9c089544a768982794f32
Bug: b/154733231
Test: Built the failing action with / without this change
2020-04-27 21:53:43 -07:00
Treehugger Robot 867dfd52f5 Merge "Add support for remotely executing header-abi-dumper actions." 2020-04-27 19:16:41 +00:00
Ramy Medhat 31ec942ff7 Add support for remotely executing header-abi-dumper actions.
Test: built with and without RBE_ABI_DUMPER set.
Change-Id: Ie752c07325453076d191813eaa50da03e3a0c2d8
2020-04-27 08:13:42 -07:00
Hsin-Yi Chen 9105fa385a Merge "Add check_all_apis option for header ABI checker" 2020-04-27 03:34:08 +00:00
Oliver Nguyen 0452678a40 Only package gcno files for gcov coverage builds.
Bug: 154550223
Test: m -j NATIVE_COVERAGE=true droid dist tests
Test: m -j CLANG_COVERAGE=true droid dist tests
Merged-In: I81598bcab8db105de6692156c001fc961409ce63
Change-Id: I81598bcab8db105de6692156c001fc961409ce63
2020-04-22 13:30:07 -07:00
Ramy Medhat 9a90fe5e23 Allow remote execution of link actions.
This CL adds a remoteexec package that allows adding a configurable RBE
prefix to the template.

Test: built aosp crosshatch userdebug with and without RBE_CXX_LINKS.
Change-Id: Ica920c3d7f79f2996210b9cbd448126451c1707c
2020-04-16 21:18:10 -04:00
Logan Chien 2a65dda27d Add check_all_apis option for header ABI checker
This commit adds a header ABI checker option to check all APIs that can
be found.  Without this option, the checker by default only checks the
class/struct/enum that are directly or indirectly referred by one of the
exported symbols.  With `check_all_apis: true,`, the checker will check
all class/struct/enum.

Bug: 141709599
Test: Add `header_abi_checker: { check_all_apis: true, }` to a library
      and see breakage if I change some enum.
Change-Id: I61f90e07b60a6752fc6be4398420c1ad1291102f
2020-02-26 09:25:40 +00:00
Kousik Kumar 2976bfd568 Do not add ccWrapper to ccNoDeps rule
The "ccNoDeps" rule was introduced to separate out ".s" compilations
from other ".S" and ".c*" compilations. The ".s" compilation does not
produce a dependency file and does not support running a preprocessor
through it.
However, it does have ".include" directives, that do NOT take a macro,
but still does the equivalent of a "#include". The compilation of ".s"
assembly files also do NOT produce a dependency file.

Because they don't produce a dependency file AND because RBE's input
processor does not yet support finding dependencies for these files, I'm
making this change to not prepend rewrapper for these commands.

Test: Tested by running a build with this change and ensuring the ".s"
compilations aren't being sent to rewrapper.

Change-Id: I60bb14ff92596e4992e9f675bdc199f1440d4327
2020-02-18 08:00:17 -08:00
Treehugger Robot 1562752fc6 Merge "build: soong: windres: use clang to preprocess" 2020-01-30 02:21:02 +00:00
Nick Desaulniers 18eeffa4ce build: soong: windres: use clang to preprocess
Otherwise binaries under:
prebuilts/gcc/linux-x86/host/x86_64-w64-mingw32-4.8/bin/
are being invoked.

https://sourceware.org/binutils/docs-2.33.1/binutils/windres.html
states:
When windres reads an rc file, it runs it through the C preprocessor first.

Use the `--preprocessor` flag to specify the exact command to run. The
args to the preprocessor get wiped out when setting that flag, so
`-E -xc-header -DRC_INVOKED` must be re-passed, else clang will error as
it tries to invoke ld.

Once AOSP packages llvm-rc, aosp/1206346 can be rebased on top, so this
patch should be a short lived change.

Bug: 147295872
Test: <remove mingw gcc> && lunch aosp_x86-eng && \
  cd external/mdnsresponder && mm
Change-Id: Ic0e6b75e7e964ee9f9757e534cddd16438604c34
Suggested-by: Pirama Arumuga Nainar <pirama@google.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
2020-01-29 16:28:06 -08:00
Ramy Medhat 8ea054a81e Add support for experimentally enabling RBE support on specific rules.
This CL adds RBE support to javac, r8, and d8 rules which is only
enabled if respective environment variables are set.

Test: an aosp_crosshatch build with and without the new variables.
Change-Id: Ic82f3627944f6a5ee7b9f3228170c2709b1bfcb8
2020-01-28 12:42:47 -05:00
Sasha Smundak 6c2d4f91f8 Save compilation units in protobuf format.
Bug: 146224091
Test: manual
Change-Id: I7a856bab13f54c78efa0061421c3fcb9341bc6e7
2020-01-15 12:40:03 -08:00
Colin Cross 1f38237c12 Merge changes from topic "reorder-cflags"
* changes:
  Reorder local and global flags
  Split local and global cflags
2019-11-08 05:07:34 +00:00
Colin Cross 6d88dbad0e Reorder local and global flags
Native compiler flags are currently applied in approximately:
global cflags
local cflags
local include dirs
global include dirs
global conlyflags
local conlyflags
global cppflags
local cppflags

This means that a flag that is enabled in the global cppflags
cannot be disabled in the local cflags, and an Android.bp author
must know to disable it in the local cppflags.

The previous CL split the global and local flags into separate
variables.  Rearrange the order that the variables are applied
to be:
global cflags
global conlyflags
global cppflags
local cflags
local include dirs
local conlyflags
local cppflags
global include dirs

Bug: 143713277
Test: m native

Change-Id: I171524ab40096a636a8e549e1e4bc3347ef9f97a
2019-11-07 15:27:58 -08:00
Colin Cross 4af21ed26f Split local and global cflags
Native compiler flags are currently applied in approximately:
global cflags
local cflags
local include dirs
global include dirs
global conlyflags
local conlyflags
global cppflags
local cppflags

This means that a flag that is enabled in the global cppflags
cannot be disabled in the local cflags, and an Android.bp author
must know to disable it in the local cppflags.  A better order
would be:
global cflags
global conlyflags
global cppflags
local cflags
local conlyflags
local cppflags
local include dirs
global include dirs

We are mixing both the global and local cflags into a single
variable, and similar for conlyflags and cppflags, which
prevents reordering them.  This CL prepares to reorder them
by splitting the global and local cflags into separate variables.

Bug: 143713277
Test: m native
Change-Id: Ic55a8c3516c331dc5f2af9d00e59ceca9d3e6c15
2019-11-07 15:27:58 -08:00
Ramy Medhat dd0418a4d4 Run non-RBE supported actions in the local pool when USE_RBE is set.
Bug: 143938974
Test: ran CTS build at -j500 successfully.
Change-Id: I55074bd67308cd716972e24fb56a20bc393d5d9d
2019-11-05 22:57:35 +00:00
Yi Kong c49c393f73 Repack libgcc.a to only include required objects
Previous solution by using objcopy uses a quirky behaviour of the GNU
objcopy and there is no equivalent option in llvm-objcopy.

Instead of removing symbols, extract and repack libgcc to only include
required objects.

Bug: 142585047
Test: presubmit
Change-Id: I58af74c18838f797e481da38c3265f0624fddf99
2019-10-24 16:34:54 -07:00
Colin Cross 2e2dbc250a Use localPool consistently for UseGoma() == true
Remove the distinction between pctx.StaticRule and
pctx.AndroidStaticRule so that all of the local rules correctly
get assigned to the localPool.  Also put Module and Singleton
rules into the localPool.

Test: compare out/soong/build.ninja
Change-Id: Id2bb38eff3c7209340fe55bc9006f00bd3661d81
2019-10-01 14:11:20 -07:00
Sasha Smundak 651436460a Pass filename mappings to C++ and Java extractors.
Android builds by default put artifacts into out/ subdirectory of
the source tree, causing the extractor to record their names as
relative. The indexer considers such files as sources, which is wrong.
Fortunately, the extractor can be fed a set of filename rewriting
rules (see build/tools/vnames.json).
Also, undo previous unsuccessful attempt use to absolute path for the
output directory to distinguish between source code and artifacts.

Bug: 141385476
Test: run the build, inspect compilation units of the kzip file
Change-Id: I89ec3aed8fd14f43ea6e0b226d54f643346f6125
2019-09-27 10:28:11 -07:00