It looks like there is a bug in rewrapper/reproxy that makes it upload the
output file as an input if it still exists in the output tree. This causes
unnecessary cache misses on incremental builds, and can also cause the
action to fail remotely (depending on the service). By removing the output
file at the beginning of the action, we ensure that it can't be uploaded.
Test: ran against a remote execution server; actions no longer fail remotely
Change-Id: Iaac2b23b35d2c876c0244edbfd4a8f159df0563c
It looks like there is a bug in rewrapper/reproxy that makes it upload the
output file as an input if it still exists in the output tree. This causes
unnecessary cache misses on incremental builds, and can also cause the
action to fail remotely (depending on the service). By removing the output
file at the beginning of the action, we ensure that it can't be uploaded.
Test: ran against a remote execution server; actions no longer fail remotely
Change-Id: I98fcc0cf105553a9920b0e96965a438e052d7c05
When soong runs the header-abi-dumper remotely, it must provide a few
dynamic libraries as toolchain inputs, or the action will fail
unconditionally on the remote execution system.
This patch allows disabling fallback for the header abi dumper
actions, i.e., this now works with remote execution:
export RBE_ABI_DUMPER_EXEC_STRATEGY=remote
Test: ran against a remote execution server; actions no longer fail remotely
Change-Id: I0c48484c03d0923dae63004fea6632704b317e95
By default, nanopb outputs a timestamp into the .c and .h files it generates.
This is usually considered bening. However, when using remote execution or
remote caching, this poisons all actions that have these files as inputs, and
also appears to affect transitive actions (actions depending on actions that
have these files as inputs).
This change adds the -T flag to suppress the timestamp, which in turn makes
the output files deterministic, which in turn allows the remote cache to
return previous results rather than requiring a re-execution of the
corresponding actions.
Test: ran the build a few times to check that the output is deterministic
Change-Id: Iecfa143dbf12a9bb7703dd0bc3f1b3e3fd56317c
The hardlink optimization attempts to save I/O by hardlinking input
files into the sandbox instead of copying them, but it never works on
a Linux build due to nsjail always making out and the source tree into
separate bind-mounted filesystems. If the optimization did work it
would actually cause build failures when it hardlinked the relative
bionic/libc/fs_config_generator.py symlink into the sandbox directory
without the target. Just remove the optimization for now since it
never works.
Test: m checkbuild
Change-Id: I96e8c0c145e7c99958639594edf8a93b69ae90eb
Previously, TestGetDistForGoals tested multiple test cases within a
single test so when it failed it was difficult to determine which test
case was the cause. This change runs each test case as its own
nested test.
It also corrects the order of expectedLine and line format parameters
to match the order in the message.
Test: m nothing
Bug: 174226317
Change-Id: I1408ec4125afc5c0b392cd7643dd3f630fe468e5
Source-generated library modules will have a dependency on the source
variant. This creates a cycle in the dependency graph which triggers a
warning from rust-analyzer. Ignore this type of dependency.
Test: m nothing
Change-Id: I13365093ebb88b00f6a72734b01114ec3e9a320e
This patch reworks the approach introduced in
https://r.android.com/1450819. That patch based the decision which
libraries should be added to the manifest <uses-library> tags by the
manifest_fixer on the "shared" status of the library.
That approach is incorrect for two reasons:
- It doesn't make sense to have a non-shared library in class loader
context ("shared" libraries are those specified in
frameworks/base/data/etc/platform.xml, and they are the only ones
that PackageManager knows about).
- It doesn't make sense to add anything but the top-level of the
class loader context tree to the manifest, because this part of the
tree is flattened to a sequence, and PackageManager cannot restore
it to the previous tree shape (there is an information loss).
This patch removes the "shared" bit of information from class loader
context elements and assumes that all libraries that end up in class
loader context are shared. Consequently, only the top-level libraries
should be passed to manifest_fixer.
Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Bug: 168686456
Change-Id: I902690f0f38f1047fa79cf6ccbe956077eceaab0
No current use cases for soong.config and user-editable local
configuration options for Soong, so let's remove it to reduce the API
surface.
The MegaDevice configuration is used for building every module, for
every cpu variant, for every architecture, but it apparently isn't
used for a while now.
Test: m nothing
Test: TH
Fixes: 174188200
Change-Id: I2550e70ff6f9c8b57e9a7cc517d6a119a032a27a
Mostly documentation changes, but includes a few refactorings like
changing the variable names, reordering functions, reordering statements
in logical order, etc.
Bug: 173472337
Test: m
Change-Id: Iefdc8e463bcda60187b98e8c90661e220e8cdf40
gensrcs modules run the same command once for each input file to
produce each output file. Each command needs its own depfile instead
of reusing one per shard. The depfiles will be merged together
by RuleBuilder into one depfile per shard.
Test: TestGenSrcs
Change-Id: Iaf4f2cf9f5592c20e32944ddf34e0a61aff17ba8
This is to facilitate consolidation of compatibility suite logic.
Test: m nothing and compare soong Android mk files -- no diffs
Change-Id: Ibc60d14fdde93835124e18a5c484bbda496bdb9e
This is to enable consolidating test compatibility suite handling in
Soong's AndroidMk files.
Test: go test soong tests
Test: m nothing and compare soong/Android mk files -- only diffs is that
lines matching "^LOCAL_SHARED_LIBRARIES := $" are absent
Change-Id: I4922a1c8b0231cd98ebd2bc7257956eb9c0d240c
This is to facilitate consolidation of compatibility suite logic.
Test: go soong tests
Test: m nothing & compare soong Android mk files -- only diffs are
elimination of duplicate LOCAL_UNINSTALLABLE_MODULE := true lines
Change-Id: I76211505b2b6afde5d0431586b77d799cf70c6b9
Relands Ic22603a5c0718b5a21686672a7471f952b4d1017 with a minor
change to track libc++ dependencies for python hosts and after
a fix to an internal genrule that depended on transitively
installed java libraries (ag/13068670).
Soong currently assumes that installed files should depend on
installed files of all transitive dependencies, which results
in extra installed file dependencies through genrules, static
libs, etc.
Annotate dependency tags for dependencies for which the
installed files are necessary such as shared libraries
and JNI libraries.
This avoids extra installed files, and is also a first step
towards genrules using their own copy of tools instead of
the installed copy.
Bug: 124313442
Test: m checkbuild
Test: java.TestBinary
Test: cc.TestInstallSharedLibs
Test: deptag_test.go
Change-Id: I725871249d561428e6f67bba6a7c65b580012b72
This CL improves the clarity of the non-hermetic nature of using
HostSystemTools, and make the build fail if a caller tries to use a host
system tool that isn't allowlisted in config.go.
The only caller seems to be for xcrun, which is a special case that is
pretty hard to remove.
Test: TH presubmit
Change-Id: Icd3e7330406a08c00ac9682e37e2d42669f2e6b7
Mostly documentation changes, but includes a few refactorings like
changing the variable names, reording functions, reordering statements
in logical order, etc.
Bug: 173472337
Test: m
Change-Id: Ie1799c0972d63da823ad375f008018de782529d1
* changes:
Add java_data parameter to python modules
Use local variations for python version splits
Support SourceFileProducer in android.OutputFilesForModule
hilt_android requires seven separate annotation processors, which
is only feasible to support using exported_plugins to avoid having
to list all seven in every module that uses it. Unfortunately they
all set generates_api: true. Turbine is already disabled for modules
that directly use a plugin that sets generates_api: true, because
turbine doesn't run annotation processors. Also add support for
disabling turbine if a module transitively uses a plugin that
generates APIs via exported_plugins.
Bug: 173397767
Test: TestExportedPlugins
Change-Id: If70354a3dd67efb4ce88bc9c934d41ccb6241b28
csuite has a python module that wants to embed the outputs of java
modules. This has caused issues with mismatched variants bewteen
the arch-specific python module and the common java modules. Add
a java_data property that is similar to the data property but
uses the common arch variant.
Bug: 173977903
Test: m checkbuild
Change-Id: I0f2f0e4159650cd5d42b510d5177678e7ee91b4d
Use a local variation for python version splits. This causes
dependencies from python modules not to look for variations
that match the version split, which will simplify a python module
depending on a java module (for example for test data) because
Soong won't look for a python version variant of the java module.
Bug: 173977903
Test: go test ./python
Change-Id: Ib034140c478ffbc7467ab830d3cfa5683c31d05c
Add support to android.OutputFilesForModule to get paths from a
SourceFileProducer as well as an OutputFileProducer.
Bug: 173977903
Test: m checkbuild
Change-Id: I4b2ca2837342ddbb4210bee8f549a636d8b8b049
Thiébaud Weksteen