This change adds some additional tests to cover the different
combinations of setting OutputFile and DistFiles properties in the
AndroidMkEntry.
These tests will prevent follow up changes from inadvertently changing
the behavior that some modules rely on.
Test: m nothing
Bug: 174226317
Change-Id: I6ab6f2711a5fa416ea2b84a1e0b8754d525d7a0c
Uses the dists property to copy the removed API specification file for
each API scope alongside the API specification file in the dist.
Bug: 158544979
Test: m nothing
m dist sdk - before and after this change, compare result to
make sure that there are no significant differences other than
the after now contains -removed.txt files for each scope of each
java_sdk_library
Change-Id: If23c594d29a12e49d2c5a8945ad488874bb1590c
A previous change handles dist properties automatically for all module
types and as a result has made the apexBundle setting of DistFiles
redundant so this change removes that.
Test: m nothing
m dist sdk - before and after this change, compare result to
make sure that there are no significant differences.
Test the effect on the apex by following instructions in
http://b/172951145
Bug: 174226317
Change-Id: If6f741b5b73b3335f4c478a6528f5f311cd633e4
A previous change handles dist properties automatically for all module
types and as a result has made the java.Library setting of DistFiles
redundant so this change removes that and the tests that duplicate
tests of the general mechanism.
Test: m nothing
m dist sdk - before and after this change, compare result to
make sure that there are no significant differences.
Bug: 174226317
Change-Id: Ib79a3bdd46897efd84a9c456c37c374bd6036303
Previously in order to make use of dist entries with tag properties it
was necessary for each module type to be modified to store the results
of GenerateTaggedDistFiles(module) in the AndroidMkEntry.DistFiles
property. This change generalizes that mechanism to work across all
module types.
This change does improve the behavior of a couple of tests where
previously a dist with tag property was ignored because tagged dist
files were not available now it is used correctly.
Some module types do not implement OutputFileProducer interface and so
they cannot handle dist with tag properties. This change makes it an
error to attempt to do that. That necessitated adding OutputFiles(tag)
method to customModule in queryview_test.go as some of the tests in
that file used dist entries with tag properties.
Follow up changes will remove the code that was made redundant by this.
Test: m nothing
m dist sdk - before and after this change, compare result to
make sure that there are no significant differences.
Bug: 174226317
Change-Id: Ifc54d67db10ce0d0fe8179c05b97a2be2113be4e
Change https://r.android.com/1335521 added tag property to the Dist
struct so that it could be used to select one of a number of different
output files to copy to the dist instead of the single file that the
module type made available for dist. The output files were selected
by passing the tag to OutputFiles(tag).
Module types that wanted to support this new approach had to explicitly
set AndroidMkEntries.DistFiles = GenerateTaggedDistFiles(module).
Unfortunately, doing that had a side effect of changing the behavior of
dist entries without a tag.
That was because the change treated a tag that was not specified, as
being the same as "". So, prior to the change no tag meant use the
default dist file but after it meant use the paths returned by
OutputFiles(""). That changed the behavior of the java.Library type
which affected the behavior of the android_app module type.
Prior to the change the java_library would make the
Library.outputFile available for dist when no tag was specified. After
that change it would make Library.outputFile plus
Library.extraOutputFiles. The latter is usually empty except for
android_app which adds some extra files into there which will now be
copied to the dist. That change may have been intentional but there
was no mention of it in the change or the bug. Even if it wasn't
intentional it may still be beneficial.
Any module type that wants to add support for tags in dist runs the
risk of introducing similar changes in behavior. This change
differentiates between the tag not being set and the tag being set to
"" to avoid that possibility and to make the default behavior
explicit for those module types that have switched.
It does so as follows:
* Adds a DefaultDistTag constant that is used when the tag is not set.
It is a string that is unlikely to be used as an actual tag as it
does not start with a . and uses some special characters.
* The DefaultDistTag is used in MakeDefaultDistFiles(paths) to indicate
that the supplied paths are the default ones and and also in
GenerateTaggedDistFiles() for Dist structures that have no tag
property set.
* The DefaultDistTag is passed to OutputFiles(tag) just in case the
module type has explicitly defined the paths to associate with that
tag in there. If it has then it overrides the legacy behavior. If it
has not then it is just ignored and falls back to using the previous
behavior.
* The java.Library.OutputFiles(tag) method explicitly handles the
DefaultDistTag and returns Library.outputFile for it which restores
the behavior from before the change that added dist.tag support.
* Similar change was made to apexBundle.OutputFiles(tag) in order to
preserve its previous behaviour.
* The customModule used by TestGetDistContributions has been modified
to also preserve its previous behavior after this change.
Test: m nothing
m dist sdk - before and after this change, compare result to
make sure that there are no significant differences.
Test the effect on the apex by following instructions in
http://b/172951145
Bug: 174226317
Change-Id: Ib8f0d9307751cc2ed34e3d9a5538d3c144666f6d
Previously, only the dist property's nested properties were checked for
correctness. This change also checks the dists property's nested dist
structures and adds some tests to verify that the checks are run and
correctly report the location of the incorrect property even when it is
within a slice of dist structs.
Test: m nothing
Bug: 174226317
Change-Id: If5a19360e1e4c98ee3b5afc813e35349d1fc6f6f
A previous change duplicated the test cases from TestGetDistForGoals()
to test the getDistContributions() method. This change removes the
duplicate tests and leaves a single test to verify that
GetDistForGoals() uses getDistContributions() and
generateDistContributionsForMake() correctly.
Test: m nothing
Bug: 174226317
Change-Id: I545501016b932f9af0d5fda2f452a3a09932d3fb
Previously, the GetDistForGoals(Module) func combined the processing
of the dist properties with generating the make specific rules for
generating that dist. That has a couple of problems:
1. It combines two pieces of functionality into one method which is
bad practice.
2. It makes it hard to test because the make specific output ends up
containing absolute paths to temporary directories created by the
test.
3. It makes switching to a non-make output difficult and fragile as
changing the output will also require changing the tests.
This change adds an intermediate data structure to contain the result
of the dist processing. That processing is done by the new method
getDistContributions(Module) which returns the new intermediate
structure. It also adds generateDistContributionsForMake(..) to
generate the make output. The GetDistForGoals(Module) func uses them to
implement the previous behavior.
It adds identical tests to those in TestGetDistForGoals() but leaves
those tests alone to show that this refactoring does not change the
behavior. Follow up changes will clean up TestGetDistForGoals(). It
also adds a test for generateDistContributionsForMake(..).
Bug: 174226317
Test: m nothing
m dist sdk - before and after this change, compare result to
make sure that there are no significant differences.
Change-Id: I458b7c8e4485bf66d3498f50df85a8d65fc2ee00
module_names.txt and module_paths.txt did not end with the newline
character in each line. Instead it had literal '\n'.
The character must use '\n' without escape character.
Test: m vndk
Change-Id: I1de1d0993994c8a95b02426777f62bfb2bed7b6f
The sbox tool is used to wrap genrule actions, and run them in a
temporary directory. Afterwards, the outputs are moved into their
proper location in the output tree. However, some tools embed the file
name of the output file (as passed to the tool) into the output file.
For example, the perfetto code generator script uses the output file
name to generate a C header guard when it generates
perfetto_src_base_version_gen_h/gen/perfetto_version.gen.h.
When using remote execution / remote caching, these genrule actions are
run locally whenever the output file doesn't exist, effectively making
them unique (across time and between users).
They then cause cache misses on all actions depending on these output
files as well as on transitive actions. In the above example, this
causes libperfetto.so to differ, which then causes all actions
depending on libperfetto.so to be rerun unnecessarily.
As a fix, this commit changes the sbox tool to use the sha1 hash of the
manifest path. The tool already seems to assume that there are no
concurrent runs of the same tool with the same manifest (otherwise
it would fail flakily depending on exact timing). It seems therefore
safe to use a temporary path deterministically derived from the
manifest path.
Test: ran multiple builds w/ remote execution; observe proper caching
Change-Id: I5b73ffd3b7f85cbb0336dfa1675de7ac0e2fd1a8
It looks like there is a bug in rewrapper/reproxy that makes it upload the
output file as an input if it still exists in the output tree. This causes
unnecessary cache misses on incremental builds, and can also cause the
action to fail remotely (depending on the service). By removing the output
file at the beginning of the action, we ensure that it can't be uploaded.
Test: ran against a remote execution server; actions no longer fail remotely
Change-Id: Iaac2b23b35d2c876c0244edbfd4a8f159df0563c
It looks like there is a bug in rewrapper/reproxy that makes it upload the
output file as an input if it still exists in the output tree. This causes
unnecessary cache misses on incremental builds, and can also cause the
action to fail remotely (depending on the service). By removing the output
file at the beginning of the action, we ensure that it can't be uploaded.
Test: ran against a remote execution server; actions no longer fail remotely
Change-Id: I98fcc0cf105553a9920b0e96965a438e052d7c05
When soong runs the header-abi-dumper remotely, it must provide a few
dynamic libraries as toolchain inputs, or the action will fail
unconditionally on the remote execution system.
This patch allows disabling fallback for the header abi dumper
actions, i.e., this now works with remote execution:
export RBE_ABI_DUMPER_EXEC_STRATEGY=remote
Test: ran against a remote execution server; actions no longer fail remotely
Change-Id: I0c48484c03d0923dae63004fea6632704b317e95
By default, nanopb outputs a timestamp into the .c and .h files it generates.
This is usually considered bening. However, when using remote execution or
remote caching, this poisons all actions that have these files as inputs, and
also appears to affect transitive actions (actions depending on actions that
have these files as inputs).
This change adds the -T flag to suppress the timestamp, which in turn makes
the output files deterministic, which in turn allows the remote cache to
return previous results rather than requiring a re-execution of the
corresponding actions.
Test: ran the build a few times to check that the output is deterministic
Change-Id: Iecfa143dbf12a9bb7703dd0bc3f1b3e3fd56317c
The hardlink optimization attempts to save I/O by hardlinking input
files into the sandbox instead of copying them, but it never works on
a Linux build due to nsjail always making out and the source tree into
separate bind-mounted filesystems. If the optimization did work it
would actually cause build failures when it hardlinked the relative
bionic/libc/fs_config_generator.py symlink into the sandbox directory
without the target. Just remove the optimization for now since it
never works.
Test: m checkbuild
Change-Id: I96e8c0c145e7c99958639594edf8a93b69ae90eb
Previously, TestGetDistForGoals tested multiple test cases within a
single test so when it failed it was difficult to determine which test
case was the cause. This change runs each test case as its own
nested test.
It also corrects the order of expectedLine and line format parameters
to match the order in the message.
Test: m nothing
Bug: 174226317
Change-Id: I1408ec4125afc5c0b392cd7643dd3f630fe468e5
Source-generated library modules will have a dependency on the source
variant. This creates a cycle in the dependency graph which triggers a
warning from rust-analyzer. Ignore this type of dependency.
Test: m nothing
Change-Id: I13365093ebb88b00f6a72734b01114ec3e9a320e
This patch reworks the approach introduced in
https://r.android.com/1450819. That patch based the decision which
libraries should be added to the manifest <uses-library> tags by the
manifest_fixer on the "shared" status of the library.
That approach is incorrect for two reasons:
- It doesn't make sense to have a non-shared library in class loader
context ("shared" libraries are those specified in
frameworks/base/data/etc/platform.xml, and they are the only ones
that PackageManager knows about).
- It doesn't make sense to add anything but the top-level of the
class loader context tree to the manifest, because this part of the
tree is flattened to a sequence, and PackageManager cannot restore
it to the previous tree shape (there is an information loss).
This patch removes the "shared" bit of information from class loader
context elements and assumes that all libraries that end up in class
loader context are shared. Consequently, only the top-level libraries
should be passed to manifest_fixer.
Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Bug: 168686456
Change-Id: I902690f0f38f1047fa79cf6ccbe956077eceaab0
No current use cases for soong.config and user-editable local
configuration options for Soong, so let's remove it to reduce the API
surface.
The MegaDevice configuration is used for building every module, for
every cpu variant, for every architecture, but it apparently isn't
used for a while now.
Test: m nothing
Test: TH
Fixes: 174188200
Change-Id: I2550e70ff6f9c8b57e9a7cc517d6a119a032a27a
Ulya Trafimovich