Commit Graph

27880 Commits

Author SHA1 Message Date
Colin Cross f1a035e6be Pass pctx and ctx to NewRuleBuilder
Enable the RuleBuilder and RuleBuilderCommand methods to access
the BuilderContext by passing it to NewRuleBuilder instead of
RuleBuilder.Build.

Test: genrule_test.go
Test: rule_builder_test.go
Test: m checkbuild
Change-Id: I63e6597e19167393876dc2259d6f521363b7dabc
2020-12-01 16:22:16 -08:00
Jiyong Park 33e1763094 Merge "android_filesystem is installable" 2020-12-01 23:36:28 +00:00
Treehugger Robot 464e6c71df Merge "Add test suite handling to central androidmk code" 2020-12-01 22:19:26 +00:00
Liz Kammer 57f5b33ad3 Add test suite handling to central androidmk code
MTS is introducing partial MTS test suites that are per-module, with
names of the format: mts-${MODULE}. By centralizing the code for test
suites, we can automatically add "mts" test suite when an
"mts-${MODULE}" test suite is specified, reducing duplication.

Test: m mts
Bug: 170318013
Change-Id: I8ce9d3c252fcc0a937bb5f2826d21cb6c6932d82
2020-12-01 12:23:56 -08:00
Jiyong Park 65c49f5979 android_filesystem is installable
Also document about the android_filesystem module type.

Bug: 172414391
Test: m
Change-Id: Iae07af9dd22f34c8e0e4c833333bafdf82dcab2a
2020-12-01 23:47:39 +09:00
Treehugger Robot 882c042575 Merge "Document more functions, minor cleanup" 2020-12-01 14:07:35 +00:00
Ivan Lozano 9c3f653207 Merge "rust: Add support for multiple protos per module." 2020-12-01 13:11:57 +00:00
Ulyana Trafimovich 0d18ef42ec Merge changes from topic "soong-fixit-skavdrik"
* changes:
  Documenting java/dexpreopt_bootjars.go
  Documenting dexpreopt/class_loader_context.go.
2020-12-01 10:01:43 +00:00
Jiyong Park cc1157cd49 Reformat comments in packaging.go
set textwidth=100

Bug: N/A
Test: N/A

Change-Id: I2fb315c9488010d189b7757a4d93be181672b4a6
2020-12-01 05:27:48 +00:00
Jiyong Park 493e7ec3ee Merge "Clients of PackagingBase can customize dependency tag to use" 2020-12-01 03:44:38 +00:00
Jiyong Park 65b6224471 Clients of PackagingBase can customize dependency tag to use
Previously, the dep tag used by PackagingBase was fixed, which prevented
some of its clients (e.g. cvd-host-package) from opting in to
android.InstallAlwaysNeededDependencyTag. Now, PackagingBase.AddDeps
accepts the dependency tag to use.

Also, dependencies toward rust dylib, rlib, and proc_macro are
configured to return true on InstallDepNeeded. This is required to
install shared_lib dependencies of the rust modules when they are
depended on by a rust binary.

Exempt-From-Owner-Approval: a trivial change after +2 from the owner.
This has to land ASAP as many users are affected by acloud being
unavailable.

Bug: N/A
Test: m
Test: acloud create --local-instance --local-image
Change-Id: If22aee7c6f314fcb03b9d4fe6901a2557f1e899c
2020-12-01 03:43:05 +00:00
Treehugger Robot 93bfe6bf76 Merge "Allow rust code in device/google/cuttlefish" 2020-12-01 00:16:41 +00:00
Treehugger Robot 6e25bb9e5f Merge "Soong: use deterministic temp dir names in sbox" 2020-12-01 00:00:30 +00:00
Treehugger Robot bd8b0d26b2 Merge "Add and update comments in ui/build/upload.go" 2020-11-30 22:01:38 +00:00
Alex Humesky ff5c9fcf48 Merge "Update comments in genrule.go." 2020-11-30 21:16:29 +00:00
Treehugger Robot 2f53f61d59 Merge "droidstubs: Remove unnecessary setting of DistFiles" 2020-11-30 20:49:17 +00:00
Treehugger Robot fd9fdec18e Merge "Add 'lo-prio' tag to temp blocklist." 2020-11-30 20:37:47 +00:00
Alex Humesky 29e3bbe8b8 Update comments in genrule.go.
Test: Noop
Change-Id: Ib341899ea0f04e059d5471ee6e5bff8ebba82583
2020-11-30 12:30:20 -05:00
Paul Duffin d8aed4b275 droidstubs: Remove unnecessary setting of DistFiles
Instead of setting DistFiles to use the api stub (if present) as the
default dist file this simply makes OutputFiles(tag) return the api
stub file for the DefaultDistTag.

Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences.
Bug: 174226317
Change-Id: I39d615c5bf8d20d59db7eeb98d942210ec0aa8ba
2020-11-30 17:20:09 +00:00
Sasha Smundak 1e53392425 Document more functions, minor cleanup
Fixes: 173745248
Test: treehugger
Change-Id: I075cc5197fff79179ea976927701290078e4d9ac
2020-11-30 08:53:58 -08:00
Jiyong Park fdea25781f Merge "rust modules can be included in apex" 2020-11-30 15:41:28 +00:00
Jiyong Park 99644e92c8 rust modules can be included in apex
We will have some APEXes having rust binaries and libraries. So, adding
the support for the types of modules.

rust.Module now inherits from android.ApexModuleBase and implements
the android.ApexModule interface.

Bug: 172414324
Test: m

Exempt-From-Owner-Approval: rebased after +2 from the owner
Change-Id: I356ef4c45f782a6460f001e83af96d1710642d80
2020-11-30 15:40:48 +00:00
Ivan Lozano 57f434e858 rust: Add support for multiple protos per module.
This further emulates the rust-protobuf gen_mod_rs flag by providing
support for generating a single module containing multiple protobuf
definitions.

Bug: 171361369
Test: New Soong tests.
Test: Example module containing multiple protos works.
Change-Id: I815f9628a8289ae512758073dac49bc4535abf01
2020-11-30 14:55:00 +00:00
Paul Duffin 8abcbdc0b3 Merge "java_sdk_library: Copy removed API specification file to dist" 2020-11-30 13:24:19 +00:00
Paul Duffin 11402caa47 Merge "Remove redundant setting of DistFiles by apexBundle" 2020-11-30 13:24:00 +00:00
Paul Duffin a67ff601d9 Merge "Remove redundant setting of DistFiles by java.Library" 2020-11-30 13:23:50 +00:00
Paul Duffin 5e95589fe9 Merge "Generalize the handling of dist tags" 2020-11-30 12:42:55 +00:00
Paul Duffin 4326e0e094 Merge "Differentiate between no dist tag and an empty dist tag" 2020-11-30 12:31:53 +00:00
Mathew Inwood 17a8cd9634 Add 'lo-prio' tag to temp blocklist.
This is a temporary measure to allow these APIs to be identified by
downstream tooling. Before S is finished, these APIs will be moved
to the max-target-R list anyway.

Adding the lo-prio flag should have no semantic impact on these APIs
but does allow them to be distinguished from the regular blocklist
APIs.

Test: m
Bug: 174455522
Change-Id: I031dcecbdef34a4f93a0fec1a92d95f769918486
2020-11-30 11:47:30 +00:00
Thiébaud Weksteen c9b71a446a Merge "Use 1.48.0 prebuilts" 2020-11-30 09:29:23 +00:00
Ulya Trafimovich 3bfabf2626 Documenting java/dexpreopt_bootjars.go
Test: m nothing
Bug: 173092919
Change-Id: I20018e1df6ad5e21172bb91435ae2c9cad7024e6
2020-11-27 18:10:22 +00:00
Ulya Trafimovich 480d174a16 Documenting dexpreopt/class_loader_context.go.
Test: m nothing
Bug: 173092919
Change-Id: Iec64d7449a66b45be547e0d42bb3808215750cd1
2020-11-27 18:10:22 +00:00
Paul Duffin 040e906fcf java_sdk_library: Copy removed API specification file to dist
Uses the dists property to copy the removed API specification file for
each API scope alongside the API specification file in the dist.

Bug: 158544979
Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences other than
      the after now contains -removed.txt files for each scope of each
      java_sdk_library
Change-Id: If23c594d29a12e49d2c5a8945ad488874bb1590c
2020-11-27 15:17:44 +00:00
Paul Duffin 281bc00bbf Remove redundant setting of DistFiles by apexBundle
A previous change handles dist properties automatically for all module
types and as a result has made the apexBundle setting of DistFiles
redundant so this change removes that.

Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences.
      Test the effect on the apex by following instructions in
      http://b/172951145
Bug: 174226317
Change-Id: If6f741b5b73b3335f4c478a6528f5f311cd633e4
2020-11-27 15:17:44 +00:00
Paul Duffin 620680516d Remove redundant setting of DistFiles by java.Library
A previous change handles dist properties automatically for all module
types and as a result has made the java.Library setting of DistFiles
redundant so this change removes that and the tests that duplicate
tests of the general mechanism.

Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences.
Bug: 174226317
Change-Id: Ib79a3bdd46897efd84a9c456c37c374bd6036303
2020-11-27 15:17:44 +00:00
Paul Duffin af970a2e7a Generalize the handling of dist tags
Previously in order to make use of dist entries with tag properties it
was necessary for each module type to be modified to store the results
of GenerateTaggedDistFiles(module) in the AndroidMkEntry.DistFiles
property. This change generalizes that mechanism to work across all
module types.

This change does improve the behavior of a couple of tests where
previously a dist with tag property was ignored because tagged dist
files were not available now it is used correctly.

Some module types do not implement OutputFileProducer interface and so
they cannot handle dist with tag properties. This change makes it an
error to attempt to do that. That necessitated adding OutputFiles(tag)
method to customModule in queryview_test.go as some of the tests in
that file used dist entries with tag properties.

Follow up changes will remove the code that was made redundant by this.

Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences.
Bug: 174226317
Change-Id: Ifc54d67db10ce0d0fe8179c05b97a2be2113be4e
2020-11-27 15:17:44 +00:00
Paul Duffin 74f05598eb Differentiate between no dist tag and an empty dist tag
Change https://r.android.com/1335521 added tag property to the Dist
struct so that it could be used to select one of a number of different
output files to copy to the dist instead of the single file that the
module type made available for dist. The output files were selected
by passing the tag to OutputFiles(tag).

Module types that wanted to support this new approach had to explicitly
set AndroidMkEntries.DistFiles = GenerateTaggedDistFiles(module).
Unfortunately, doing that had a side effect of changing the behavior of
dist entries without a tag.

That was because the change treated a tag that was not specified, as
being the same as "". So, prior to the change no tag meant use the
default dist file but after it meant use the paths returned by
OutputFiles(""). That changed the behavior of the java.Library type
which affected the behavior of the android_app module type.

Prior to the change the java_library would make the
Library.outputFile available for dist when no tag was specified. After
that change it would make Library.outputFile plus
Library.extraOutputFiles. The latter is usually empty except for
android_app which adds some extra files into there which will now be
copied to the dist. That change may have been intentional but there
was no mention of it in the change or the bug. Even if it wasn't
intentional it may still be beneficial.

Any module type that wants to add support for tags in dist runs the
risk of introducing similar changes in behavior. This change
differentiates between the tag not being set and the tag being set to
"" to avoid that possibility and to make the default behavior
explicit for those module types that have switched.

It does so as follows:
* Adds a DefaultDistTag constant that is used when the tag is not set.
  It is a string that is unlikely to be used as an actual tag as it
  does not start with a . and uses some special characters.
* The DefaultDistTag is used in MakeDefaultDistFiles(paths) to indicate
  that the supplied paths are the default ones and and also in
  GenerateTaggedDistFiles() for Dist structures that have no tag
  property set.
* The DefaultDistTag is passed to OutputFiles(tag) just in case the
  module type has explicitly defined the paths to associate with that
  tag in there. If it has then it overrides the legacy behavior. If it
  has not then it is just ignored and falls back to using the previous
  behavior.
* The java.Library.OutputFiles(tag) method explicitly handles the
  DefaultDistTag and returns Library.outputFile for it which restores
  the behavior from before the change that added dist.tag support.
* Similar change was made to apexBundle.OutputFiles(tag) in order to
  preserve its previous behaviour.
* The customModule used by TestGetDistContributions has been modified
  to also preserve its previous behavior after this change.

Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences.
      Test the effect on the apex by following instructions in
      http://b/172951145
Bug: 174226317
Change-Id: Ib8f0d9307751cc2ed34e3d9a5538d3c144666f6d
2020-11-27 15:17:44 +00:00
Paul Duffin 62269498be Improve test coverage of dist processing code
This change adds some additional tests to cover the different
combinations of setting OutputFile and DistFiles properties in the
AndroidMkEntry.

These tests will prevent follow up changes from inadvertently changing
the behavior that some modules rely on.

Test: m nothing
Bug: 174226317
Change-Id: I6ab6f2711a5fa416ea2b84a1e0b8754d525d7a0c
2020-11-27 15:17:44 +00:00
Paul Duffin 89968e3dec Expand dist property checks to cover dists
Previously, only the dist property's nested properties were checked for
correctness. This change also checks the dists property's nested dist
structures and adds some tests to verify that the checks are run and
correctly report the location of the incorrect property even when it is
within a slice of dist structs.

Test: m nothing
Bug: 174226317
Change-Id: If5a19360e1e4c98ee3b5afc813e35349d1fc6f6f
2020-11-27 15:17:44 +00:00
Paul Duffin d83988dbad Remove duplicate tests from TestGetDistForGoals
A previous change duplicated the test cases from TestGetDistForGoals()
to test the getDistContributions() method. This change removes the
duplicate tests and leaves a single test to verify that
GetDistForGoals() uses getDistContributions() and
generateDistContributionsForMake() correctly.

Test: m nothing
Bug: 174226317
Change-Id: I545501016b932f9af0d5fda2f452a3a09932d3fb
2020-11-27 15:17:44 +00:00
Paul Duffin 8b0349c652 Separate dist processing from make specific output
Previously, the GetDistForGoals(Module) func combined the processing
of the dist properties with generating the make specific rules for
generating that dist. That has a couple of problems:
1. It combines two pieces of functionality into one method which is
   bad practice.
2. It makes it hard to test because the make specific output ends up
   containing absolute paths to temporary directories created by the
   test.
3. It makes switching to a non-make output difficult and fragile as
   changing the output will also require changing the tests.

This change adds an intermediate data structure to contain the result
of the dist processing. That processing is done by the new method
getDistContributions(Module) which returns the new intermediate
structure. It also adds generateDistContributionsForMake(..) to
generate the make output. The GetDistForGoals(Module) func uses them to
implement the previous behavior.

It adds identical tests to those in TestGetDistForGoals() but leaves
those tests alone to show that this refactoring does not change the
behavior. Follow up changes will clean up TestGetDistForGoals(). It
also adds a test for generateDistContributionsForMake(..).

Bug: 174226317
Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences.
Change-Id: I458b7c8e4485bf66d3498f50df85a8d65fc2ee00
2020-11-27 15:17:38 +00:00
Thiébaud Weksteen c6fe19d637 Use 1.48.0 prebuilts
Bug: 173721343
Test: build external/rust/ and crosvm
Change-Id: Ifde24640fc262c5eebdcd623556e925e7d78ae85
2020-11-27 14:42:57 +01:00
Paul Duffin 3e9614198a Merge "Extract method to process bp file for androidmk_test.go" 2020-11-27 11:53:46 +00:00
Ulyana Trafimovich af07f732e6 Merge "Assume any <uses-library> is shared, add only toplevel ones to manifest." 2020-11-27 10:44:54 +00:00
Paul Duffin 103aaae715 Extract method to process bp file for androidmk_test.go
Test: m nothing
Bug: 174226317
Change-Id: Ifc9ee5088fe766d971718fc112d1e95719a2b825
2020-11-27 01:24:02 +00:00
Treehugger Robot 9c4c534d25 Merge "Documenting apex/builer.go" 2020-11-27 00:55:40 +00:00
Ulf Adams b73daa588d Soong: use deterministic temp dir names in sbox
The sbox tool is used to wrap genrule actions, and run them in a
temporary directory. Afterwards, the outputs are moved into their
proper location in the output tree. However, some tools embed the file
name of the output file (as passed to the tool) into the output file.

For example, the perfetto code generator script uses the output file
name to generate a C header guard when it generates
perfetto_src_base_version_gen_h/gen/perfetto_version.gen.h.

When using remote execution / remote caching, these genrule actions are
run locally whenever the output file doesn't exist, effectively making
them unique (across time and between users).

They then cause cache misses on all actions depending on these output
files as well as on transitive actions. In the above example, this
causes libperfetto.so to differ, which then causes all actions
depending on libperfetto.so to be rerun unnecessarily.

As a fix, this commit changes the sbox tool to use the sha1 hash of the
manifest path. The tool already seems to assume that there are no
concurrent runs of the same tool with the same manifest (otherwise
it would fail flakily depending on exact timing). It seems therefore
safe to use a temporary path deterministically derived from the
manifest path.

Test: ran multiple builds w/ remote execution; observe proper caching
Change-Id: I5b73ffd3b7f85cbb0336dfa1675de7ac0e2fd1a8
2020-11-26 22:11:05 +01:00
Paul Duffin d35d92a7b5 Merge "Improve TestGetDistForGoals debuggability" 2020-11-26 17:16:53 +00:00
Thiébaud Weksteen 463ab7a27e Merge "Ignore variant dependencies in rust-project.json" 2020-11-26 09:15:09 +00:00
A. Cody Schuffelen 4e5fc3b422 Allow rust code in device/google/cuttlefish
Test: TreeHugger
Change-Id: If7bce48e101bd49389d952556bfdf95d7899f01b
2020-11-25 18:29:39 -08:00