Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1419252
Test:
1. Applied https://googleplex-android-review.git.corp.google.com/c/platform/build/soong/+/15723183, confirmed that the build fails on rvc-dev (by running `USE_RBE=false m `)
2. Subsequently applied this patch and tested to confirm that the failure is fixed (by running `USE_RBE=false m out/soong/.intermediates/vendor/qcom/sm7250/proprietary/commonsys/qcrilOemHook/qcrilmsgtunnel/android_common/combined/qcrilmsgtunnel.jar` and `USE_RBE=true m out/soong/.intermediates/vendor/qcom/sm7250/proprietary/commonsys/qcrilOemHook/qcrilmsgtunnel/android_common/combined/qcrilmsgtunnel.jar`).
Bug: b/198045383
Merged-In: I6df58b4e700f0d231367af2710672d731d20a736
Change-Id: I52b7a3810fdf3314ae107f4b8330e9d6f62c6302
(cherry picked from commit f15ed4fe61)
Bug: b/166182389
Test: built with and without RBE_ABI_DUMPER set.
Change-Id: Ie752c07325453076d191813eaa50da03e3a0c2d8
Merged-In: Ie752c07325453076d191813eaa50da03e3a0c2d8
This CL adds a remoteexec package that allows adding a configurable RBE
prefix to the template.
Bug: b/166182389
Test: built aosp crosshatch userdebug with and without RBE_CXX_LINKS.
Change-Id: Ica920c3d7f79f2996210b9cbd448126451c1707c
Merged-In: Ica920c3d7f79f2996210b9cbd448126451c1707c
The "ccNoDeps" rule was introduced to separate out ".s" compilations
from other ".S" and ".c*" compilations. The ".s" compilation does not
produce a dependency file and does not support running a preprocessor
through it.
However, it does have ".include" directives, that do NOT take a macro,
but still does the equivalent of a "#include". The compilation of ".s"
assembly files also do NOT produce a dependency file.
Because they don't produce a dependency file AND because RBE's input
processor does not yet support finding dependencies for these files, I'm
making this change to not prepend rewrapper for these commands.
Test: Tested by running a build with this change and ensuring the ".s"
compilations aren't being sent to rewrapper.
Bug: b/166182389
Change-Id: I60bb14ff92596e4992e9f675bdc199f1440d4327
Merged-In: I60bb14ff92596e4992e9f675bdc199f1440d4327
This CL adds RBE support to javac, r8, and d8 rules which is only
enabled if respective environment variables are set.
Test: an aosp_crosshatch build with and without the new variables.
Bug: b/166182389
Change-Id: Ic82f3627944f6a5ee7b9f3228170c2709b1bfcb8
Merged-In: Ic82f3627944f6a5ee7b9f3228170c2709b1bfcb8
Remove the distinction between pctx.StaticRule and
pctx.AndroidStaticRule so that all of the local rules correctly
get assigned to the localPool. Also put Module and Singleton
rules into the localPool.
Bug: b/166182389
Test: compare out/soong/build.ninja
Change-Id: Id2bb38eff3c7209340fe55bc9006f00bd3661d81
Merged-In: Id2bb38eff3c7209340fe55bc9006f00bd3661d81
Bug: b/156613606
Test: built with and without RBE_ABI_DUMPER set.
Change-Id: Ie752c07325453076d191813eaa50da03e3a0c2d8
Merged-In: Ie752c07325453076d191813eaa50da03e3a0c2d8
Bug: b/154733231
Change-Id: I9b6ae97f5bbfe8f9c4b9c089544a768982794f32
Merged-In: I9b6ae97f5bbfe8f9c4b9c089544a768982794f32
Test: Built the failing action with / without this change
This CL adds a remoteexec package that allows adding a configurable RBE
prefix to the template.
Change-Id: I6b4ea773f6bae5706415a1830f822db7decca92f
Test: built aosp crosshatch userdebug with and without RBE_CXX_LINKS.
Merged-In: Ica920c3d7f79f2996210b9cbd448126451c1707c
Bug: b/154820732
The "ccNoDeps" rule was introduced to separate out ".s" compilations
from other ".S" and ".c*" compilations. The ".s" compilation does not
produce a dependency file and does not support running a preprocessor
through it.
However, it does have ".include" directives, that do NOT take a macro,
but still does the equivalent of a "#include". The compilation of ".s"
assembly files also do NOT produce a dependency file.
Because they don't produce a dependency file AND because RBE's input
processor does not yet support finding dependencies for these files, I'm
making this change to not prepend rewrapper for these commands.
Test: Tested by running a build with this change and ensuring the ".s"
compilations aren't being sent to rewrapper.
Change-Id: I60bb14ff92596e4992e9f675bdc199f1440d4327
Otherwise binaries under:
prebuilts/gcc/linux-x86/host/x86_64-w64-mingw32-4.8/bin/
are being invoked.
https://sourceware.org/binutils/docs-2.33.1/binutils/windres.html
states:
When windres reads an rc file, it runs it through the C preprocessor first.
Use the `--preprocessor` flag to specify the exact command to run. The
args to the preprocessor get wiped out when setting that flag, so
`-E -xc-header -DRC_INVOKED` must be re-passed, else clang will error as
it tries to invoke ld.
Once AOSP packages llvm-rc, aosp/1206346 can be rebased on top, so this
patch should be a short lived change.
Bug: 147295872
Test: <remove mingw gcc> && lunch aosp_x86-eng && \
cd external/mdnsresponder && mm
Change-Id: Ic0e6b75e7e964ee9f9757e534cddd16438604c34
Suggested-by: Pirama Arumuga Nainar <pirama@google.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
This CL adds RBE support to javac, r8, and d8 rules which is only
enabled if respective environment variables are set.
Test: an aosp_crosshatch build with and without the new variables.
Change-Id: Ic82f3627944f6a5ee7b9f3228170c2709b1bfcb8
Native compiler flags are currently applied in approximately:
global cflags
local cflags
local include dirs
global include dirs
global conlyflags
local conlyflags
global cppflags
local cppflags
This means that a flag that is enabled in the global cppflags
cannot be disabled in the local cflags, and an Android.bp author
must know to disable it in the local cppflags.
The previous CL split the global and local flags into separate
variables. Rearrange the order that the variables are applied
to be:
global cflags
global conlyflags
global cppflags
local cflags
local include dirs
local conlyflags
local cppflags
global include dirs
Bug: 143713277
Test: m native
Change-Id: I171524ab40096a636a8e549e1e4bc3347ef9f97a
Native compiler flags are currently applied in approximately:
global cflags
local cflags
local include dirs
global include dirs
global conlyflags
local conlyflags
global cppflags
local cppflags
This means that a flag that is enabled in the global cppflags
cannot be disabled in the local cflags, and an Android.bp author
must know to disable it in the local cppflags. A better order
would be:
global cflags
global conlyflags
global cppflags
local cflags
local conlyflags
local cppflags
local include dirs
global include dirs
We are mixing both the global and local cflags into a single
variable, and similar for conlyflags and cppflags, which
prevents reordering them. This CL prepares to reorder them
by splitting the global and local cflags into separate variables.
Bug: 143713277
Test: m native
Change-Id: Ic55a8c3516c331dc5f2af9d00e59ceca9d3e6c15
Previous solution by using objcopy uses a quirky behaviour of the GNU
objcopy and there is no equivalent option in llvm-objcopy.
Instead of removing symbols, extract and repack libgcc to only include
required objects.
Bug: 142585047
Test: presubmit
Change-Id: I58af74c18838f797e481da38c3265f0624fddf99
Remove the distinction between pctx.StaticRule and
pctx.AndroidStaticRule so that all of the local rules correctly
get assigned to the localPool. Also put Module and Singleton
rules into the localPool.
Test: compare out/soong/build.ninja
Change-Id: Id2bb38eff3c7209340fe55bc9006f00bd3661d81
Android builds by default put artifacts into out/ subdirectory of
the source tree, causing the extractor to record their names as
relative. The indexer considers such files as sources, which is wrong.
Fortunately, the extractor can be fed a set of filename rewriting
rules (see build/tools/vnames.json).
Also, undo previous unsuccessful attempt use to absolute path for the
output directory to distinguish between source code and artifacts.
Bug: 141385476
Test: run the build, inspect compilation units of the kzip file
Change-Id: I89ec3aed8fd14f43ea6e0b226d54f643346f6125
The P flag is supported as of llvm r354044, add back the flag to reduce
the size of intermediate archieve files. This does not affect the final
binaries.
Test: built
Bug: 71618641
Change-Id: I017780e4dcaa31c7fbe10b5e7482db1bba83e716
Usually, ".S" files are processes with the c preprocessor, and ".s"
files are not, so they don't have any dependency information, since it
is generated by the preprocessor.
But with the -xassembler-with-cpp flag, ".s" files are processed with
the preprocessor, so we should ask for dependency information from them.
Test: NINJA_ARGS="-t deps out/soong/.intermediates/external/sonivox/arm-wt-22k/libsonivox/android_arm_armv7-a-neon_core_static/obj/external/sonivox/arm-wt-22k/lib_src/ARM-E_filter_gnu.o" m
Test: treehugger
Change-Id: Iee7baeebc2b205b5a2f33e7c1705ea4a5b4fc95a
* changes:
Introduce inject_bssl_hash library property.
BoringSSL FIPS build - introduce extraLibFlags and use for STL libs.
Allow linker scripts when building objects.
Allow .o files as srcs.
Rationale: On non-bionic, stl.go currently adds system libraries to
ldFlags, this causes problems for partialLd rules. However adding the
same libraries to libFlags breaks some existing modules due to symbol
conflicts as the system libraries are linked before some module code.
Introduced a general mechanism for adding libraries to be linked
last rather than making this STL-specific.
Bug: 134581881
Bug: 137267623
Test: TH
Change-Id: I779f28c6586b3fea85cc6299b686e4fde95262d3
We've been getting these dependencies transitively through the
dependency on the object itself (which is a workaround for the lack of
dep file support in these tools). But for remote builds to work, we need
to know about these dependencies like any other object compilation.
For regular builds, this increases the size of the ninja file by a few
tens of megabytes (~1-2%).
WITH_TIDY builds were already larger (~40-50%), but are now about 90%
larger than a normal build.
Test: treehugger
Change-Id: Icdb4ca3d4d08d5706593d96d5c627149fa14fed8
This commit adds tags, such as NDK, VNDK-core, and PLATFORM, to
LSDUMP_PATHS. The script updating the reference ABI dumps uses the tags
to determine the directories where the dumps should be created.
Test: make findlsdumps
Bug: 133176785
Change-Id: I8540286238cf0ec55c65e1c4f60cb9c12e5e57a1
Use Kythe (https://kythe.io) to build cross reference for the Android
source code. ~generate the input for it during the build. This is done
on demand: if XREF_CORPUS environment variable is set, build emits a
Ninja rule to generate Kythe input for each compilation rule. It
also emits two consolidation rules (`xref_cxx` and `xref_java`),
that depend on all Kythe input generation rules for C++ and Java.
The value of the XREF_CORPUS environment variable is recorded in the
generated files and thus passed to Kythe. For the AOSP master branch it is
`android.googlesource.com/platform/superproject`, so the command to build
all input for Kythe on that branch is:
```
XREF_CORPUS=android.googlesource.com/platform/superproject m xref_cxx xref_java
```
Each Kythe input generation rule generates a single file with .kzip
extension. Individual .kzip files have a lot of common information, so
there will be a post-build consolidation step run to combine them.
The consolidated .kzip file is then passed to Kythe backend.
The tools to generate .kzip files are provided by Kythe (it calls them
'extractors'). We are going to build them in toolbuilding branches
(clang-tools and build-tools) and check them in as binaries into master
and other PDK branches:
For C++, `prebuilts/clang-tools/linux-x86/bin/cxx_extractor`
for Java, `prebuilts/build-tools/common/framework/javac_extractor.jar`
Bug: 121267023
Test: 1) When XREF_CORPUS is set, build generates Ninja rules to create
.kzip files; 2) When XREF_CORPUS is set, building
`xref_cxx`/`xref_java` creates .kzip files; 3) Unless XREF_CORPUS is
set, build generates the same Ninja rules as before
Change-Id: If957b35d7abc82dbfbb3665980e7c34afe7c789e
Clang will soon start using relocations that are currently unsupported by
the version of ld.bfd that Android uses when HWASAN is enabled. This will
cause partialLd to fail since the clang driver defaults to ld.bfd.
Switch to using LLD, since it will support those relocations.
Change-Id: I0964f53dc63773f3b52bb377f863df9c39961a07
This reverts commit 6d8c0a50bc.
The switchover to lld appears to be making adb crash on exit for unclear
reasons. Revert it as a quick fix pending further investigation.
Bug: http://b/134613180
Test: adb.exe under windbg
Change-Id: Ibdf5981c3c6828b3684974b7c30a7c96449c64c5
Failures in strip.sh are still occurring with darwinStripPool set to
10, try 5.
Bug: 132822437
Test: none
Change-Id: I2df247c89b6d8f6e741d4e17057b06ff9ec72b40
Merged-In: I2df247c89b6d8f6e741d4e17057b06ff9ec72b40
(cherry picked from commit badf8d691f)
Failures in strip.sh are still occurring with darwinStripPool set to
10, try 5.
Bug: 132822437
Test: none
Change-Id: I2df247c89b6d8f6e741d4e17057b06ff9ec72b40
strip.sh can use a file descriptor per .o file when run on .a files,
which can hit the system file descriptor limit on darwin. This
causes failures when manay variants of libgcc_stripped are built
simultaneously. Put all strip rules on darwin into a pool that
limits them to 10 concurrent processes, which will limit the file
descriptor usage to ~7500.
Fixes: 132822437
Test: no mention of darwinStripPool in out/soong/build.ninja on linux
Test: m libgcc_stripped on darwin
Change-Id: I3d4fbbd8d44d2e9059a79df113ab95336ec2c658
Merged-In: I3d4fbbd8d44d2e9059a79df113ab95336ec2c658
(cherry picked from commit ee3ea31a24)
New strip option named keep_symbols_and_debug_frame, that will keep the
symbols and the .debug_frame. This is meant for use by libc.so only on
arm32. Other libraries might want to use it to keep better unwinding
information on device.
Bug: 132992102
Test: Built libc.so with this option and verified that it contains
Test: the .debug_frame section.
Change-Id: I823a28199dec8316e8b26fe31ff9f17e6b11d406
strip.sh can use a file descriptor per .o file when run on .a files,
which can hit the system file descriptor limit on darwin. This
causes failures when manay variants of libgcc_stripped are built
simultaneously. Put all strip rules on darwin into a pool that
limits them to 10 concurrent processes, which will limit the file
descriptor usage to ~7500.
Fixes: 132822437
Test: no mention of darwinStripPool in out/soong/build.ninja on linux
Test: m libgcc_stripped on darwin
Change-Id: I3d4fbbd8d44d2e9059a79df113ab95336ec2c658
Kousik Kumar