Commit Graph

31843 Commits

Author SHA1 Message Date
Paul Duffin 982093faaf Merge "Differentiate between no dist tag and an empty dist tag" am: 4326e0e094
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1511911

Change-Id: I9044ee88fbf8bd7b0fc800ea11f4369816bb23b9
2020-11-30 13:05:01 +00:00
Paul Duffin 4326e0e094 Merge "Differentiate between no dist tag and an empty dist tag" 2020-11-30 12:31:53 +00:00
Thiébaud Weksteen db1f2483da Merge "Use 1.48.0 prebuilts" am: c9b71a446a
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1512614

Change-Id: I03d2c3fdc6753a840557c94540bd610928e55bbb
2020-11-30 09:51:20 +00:00
Thiébaud Weksteen c9b71a446a Merge "Use 1.48.0 prebuilts" 2020-11-30 09:29:23 +00:00
Paul Duffin 61a4f20964 Improve test coverage of dist processing code am: 62269498be
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1512612

Change-Id: Id74f0ee3d7b4077ca3ff60292530ec12c0696b3c
2020-11-27 19:13:06 +00:00
Paul Duffin 879488716b Expand dist property checks to cover dists am: 89968e3dec
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1511912

Change-Id: Ie88a749d00c712cc92ff9f7d2376ab0cc30d03c4
2020-11-27 19:13:02 +00:00
Paul Duffin f3f916c2a6 Remove duplicate tests from TestGetDistForGoals am: d83988dbad
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1512611

Change-Id: Ic06bb2b734c237bd51aec0ad961358d8a5260880
2020-11-27 19:12:58 +00:00
Paul Duffin 04159e7961 Separate dist processing from make specific output am: 8b0349c652
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1512610

Change-Id: Ied0373d2789934815175f7b761e331cc5106553c
2020-11-27 19:12:28 +00:00
Paul Duffin 74f05598eb Differentiate between no dist tag and an empty dist tag
Change https://r.android.com/1335521 added tag property to the Dist
struct so that it could be used to select one of a number of different
output files to copy to the dist instead of the single file that the
module type made available for dist. The output files were selected
by passing the tag to OutputFiles(tag).

Module types that wanted to support this new approach had to explicitly
set AndroidMkEntries.DistFiles = GenerateTaggedDistFiles(module).
Unfortunately, doing that had a side effect of changing the behavior of
dist entries without a tag.

That was because the change treated a tag that was not specified, as
being the same as "". So, prior to the change no tag meant use the
default dist file but after it meant use the paths returned by
OutputFiles(""). That changed the behavior of the java.Library type
which affected the behavior of the android_app module type.

Prior to the change the java_library would make the
Library.outputFile available for dist when no tag was specified. After
that change it would make Library.outputFile plus
Library.extraOutputFiles. The latter is usually empty except for
android_app which adds some extra files into there which will now be
copied to the dist. That change may have been intentional but there
was no mention of it in the change or the bug. Even if it wasn't
intentional it may still be beneficial.

Any module type that wants to add support for tags in dist runs the
risk of introducing similar changes in behavior. This change
differentiates between the tag not being set and the tag being set to
"" to avoid that possibility and to make the default behavior
explicit for those module types that have switched.

It does so as follows:
* Adds a DefaultDistTag constant that is used when the tag is not set.
  It is a string that is unlikely to be used as an actual tag as it
  does not start with a . and uses some special characters.
* The DefaultDistTag is used in MakeDefaultDistFiles(paths) to indicate
  that the supplied paths are the default ones and and also in
  GenerateTaggedDistFiles() for Dist structures that have no tag
  property set.
* The DefaultDistTag is passed to OutputFiles(tag) just in case the
  module type has explicitly defined the paths to associate with that
  tag in there. If it has then it overrides the legacy behavior. If it
  has not then it is just ignored and falls back to using the previous
  behavior.
* The java.Library.OutputFiles(tag) method explicitly handles the
  DefaultDistTag and returns Library.outputFile for it which restores
  the behavior from before the change that added dist.tag support.
* Similar change was made to apexBundle.OutputFiles(tag) in order to
  preserve its previous behaviour.
* The customModule used by TestGetDistContributions has been modified
  to also preserve its previous behavior after this change.

Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences.
      Test the effect on the apex by following instructions in
      http://b/172951145
Bug: 174226317
Change-Id: Ib8f0d9307751cc2ed34e3d9a5538d3c144666f6d
2020-11-27 15:17:44 +00:00
Paul Duffin 62269498be Improve test coverage of dist processing code
This change adds some additional tests to cover the different
combinations of setting OutputFile and DistFiles properties in the
AndroidMkEntry.

These tests will prevent follow up changes from inadvertently changing
the behavior that some modules rely on.

Test: m nothing
Bug: 174226317
Change-Id: I6ab6f2711a5fa416ea2b84a1e0b8754d525d7a0c
2020-11-27 15:17:44 +00:00
Paul Duffin 89968e3dec Expand dist property checks to cover dists
Previously, only the dist property's nested properties were checked for
correctness. This change also checks the dists property's nested dist
structures and adds some tests to verify that the checks are run and
correctly report the location of the incorrect property even when it is
within a slice of dist structs.

Test: m nothing
Bug: 174226317
Change-Id: If5a19360e1e4c98ee3b5afc813e35349d1fc6f6f
2020-11-27 15:17:44 +00:00
Paul Duffin d83988dbad Remove duplicate tests from TestGetDistForGoals
A previous change duplicated the test cases from TestGetDistForGoals()
to test the getDistContributions() method. This change removes the
duplicate tests and leaves a single test to verify that
GetDistForGoals() uses getDistContributions() and
generateDistContributionsForMake() correctly.

Test: m nothing
Bug: 174226317
Change-Id: I545501016b932f9af0d5fda2f452a3a09932d3fb
2020-11-27 15:17:44 +00:00
Paul Duffin 8b0349c652 Separate dist processing from make specific output
Previously, the GetDistForGoals(Module) func combined the processing
of the dist properties with generating the make specific rules for
generating that dist. That has a couple of problems:
1. It combines two pieces of functionality into one method which is
   bad practice.
2. It makes it hard to test because the make specific output ends up
   containing absolute paths to temporary directories created by the
   test.
3. It makes switching to a non-make output difficult and fragile as
   changing the output will also require changing the tests.

This change adds an intermediate data structure to contain the result
of the dist processing. That processing is done by the new method
getDistContributions(Module) which returns the new intermediate
structure. It also adds generateDistContributionsForMake(..) to
generate the make output. The GetDistForGoals(Module) func uses them to
implement the previous behavior.

It adds identical tests to those in TestGetDistForGoals() but leaves
those tests alone to show that this refactoring does not change the
behavior. Follow up changes will clean up TestGetDistForGoals(). It
also adds a test for generateDistContributionsForMake(..).

Bug: 174226317
Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences.
Change-Id: I458b7c8e4485bf66d3498f50df85a8d65fc2ee00
2020-11-27 15:17:38 +00:00
Thiébaud Weksteen c6fe19d637 Use 1.48.0 prebuilts
Bug: 173721343
Test: build external/rust/ and crosvm
Change-Id: Ifde24640fc262c5eebdcd623556e925e7d78ae85
2020-11-27 14:42:57 +01:00
Paul Duffin bc71c946d8 Merge "Extract method to process bp file for androidmk_test.go" am: 3e9614198a
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1512609

Change-Id: I21e3286f6967560d17bb71b489a848bb569a2972
2020-11-27 12:28:55 +00:00
Paul Duffin 3e9614198a Merge "Extract method to process bp file for androidmk_test.go" 2020-11-27 11:53:46 +00:00
Ulyana Trafimovich 38352b8f69 Merge "Assume any <uses-library> is shared, add only toplevel ones to manifest." am: af07f732e6
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1511271

Change-Id: Ia0dd1ab9cd3f89dcaed1b8a54f336103f4732a7a
2020-11-27 11:16:28 +00:00
Ulyana Trafimovich af07f732e6 Merge "Assume any <uses-library> is shared, add only toplevel ones to manifest." 2020-11-27 10:44:54 +00:00
Paul Duffin 103aaae715 Extract method to process bp file for androidmk_test.go
Test: m nothing
Bug: 174226317
Change-Id: Ifc9ee5088fe766d971718fc112d1e95719a2b825
2020-11-27 01:24:02 +00:00
Treehugger Robot 024e2442bc Merge "Documenting apex/builer.go" am: 9c4c534d25
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1509688

Change-Id: I5c65d5e0bbc591de5591f6d1c9228042f036675f
2020-11-27 01:08:31 +00:00
Treehugger Robot 9c4c534d25 Merge "Documenting apex/builer.go" 2020-11-27 00:55:40 +00:00
Paul Duffin 80b40d415a Merge "Improve TestGetDistForGoals debuggability" am: d35d92a7b5
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1511910

Change-Id: I32162832b2fe085ded36f671cd4abd2982f25105
2020-11-26 17:38:59 +00:00
Paul Duffin d35d92a7b5 Merge "Improve TestGetDistForGoals debuggability" 2020-11-26 17:16:53 +00:00
Thiébaud Weksteen e006241e6f Merge "Ignore variant dependencies in rust-project.json" am: 463ab7a27e
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1511272

Change-Id: Iddf43a1dcd27f625656bfe75ac824f980e034833
2020-11-26 10:12:17 +00:00
Thiébaud Weksteen 463ab7a27e Merge "Ignore variant dependencies in rust-project.json" 2020-11-26 09:15:09 +00:00
Treehugger Robot c8e3ceb4b5 Merge "Remove hardlink optmization in sbox" am: 5bbec1a371
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1511711

Change-Id: I85d7c64d249c7ec813c2636f9a767ec909968054
2020-11-26 00:20:34 +00:00
Liz Kammer 0a1221c89a Merge "Refactor rust to use AndroidMkEntries" am: 0fd0e95e1b
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1510112

Change-Id: I993479a405c6eea4b448ef039911fcb35c39af4e
2020-11-26 00:20:24 +00:00
Liz Kammer b1084e3f6c Merge "Refactor python to use AndroidMkEntries." am: c55dcfcc2b
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1510111

Change-Id: I61e4f606a50161ea82a0d18f3b21ab44118effa7
2020-11-26 00:20:16 +00:00
Liz Kammer adc711564e Merge "Convert csuite config to use AndroidMkEntries" am: 6953adeb95
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1510569

Change-Id: If091d2cfadc2c41a7b15fe2ef24d89e63c630e85
2020-11-26 00:20:08 +00:00
Treehugger Robot 5bbec1a371 Merge "Remove hardlink optmization in sbox" 2020-11-26 00:20:01 +00:00
Treehugger Robot 1434382a5d Merge "Soong: add -T to nanopb call to suppress timestamp" am: aa8ac24222
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1511276

Change-Id: I0b0c3986b738a5a9c27097ca794f6b6e8246ed1f
2020-11-26 00:19:26 +00:00
Liz Kammer 0fd0e95e1b Merge "Refactor rust to use AndroidMkEntries" 2020-11-26 00:06:03 +00:00
Liz Kammer c55dcfcc2b Merge "Refactor python to use AndroidMkEntries." 2020-11-26 00:05:52 +00:00
Liz Kammer 6953adeb95 Merge "Convert csuite config to use AndroidMkEntries" 2020-11-26 00:05:30 +00:00
Treehugger Robot aa8ac24222 Merge "Soong: add -T to nanopb call to suppress timestamp" 2020-11-25 23:47:20 +00:00
Ulf Adams 82fd89b8ab Soong: add -T to nanopb call to suppress timestamp
By default, nanopb outputs a timestamp into the .c and .h files it generates.
This is usually considered bening. However, when using remote execution or
remote caching, this poisons all actions that have these files as inputs, and
also appears to affect transitive actions (actions depending on actions that
have these files as inputs).

This change adds the -T flag to suppress the timestamp, which in turn makes
the output files deterministic, which in turn allows the remote cache to
return previous results rather than requiring a re-execution of the
corresponding actions.

Test: ran the build a few times to check that the output is deterministic

Change-Id: Iecfa143dbf12a9bb7703dd0bc3f1b3e3fd56317c
2020-11-25 22:52:12 +01:00
Colin Cross 2282a24858 Merge "Use a unique depfile for each gensrcs command" am: b0ce5757b9
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1509811

Change-Id: I3ebce42606360e3d78bc4a8e49021cf89d440faf
2020-11-25 21:45:57 +00:00
Colin Cross b0ce5757b9 Merge "Use a unique depfile for each gensrcs command" 2020-11-25 21:18:13 +00:00
Colin Cross d03797e48a Remove hardlink optmization in sbox
The hardlink optimization attempts to save I/O by hardlinking input
files into the sandbox instead of copying them, but it never works on
a Linux build due to nsjail always making out and the source tree into
separate bind-mounted filesystems.  If the optimization did work it
would actually cause build failures when it hardlinked the relative
bionic/libc/fs_config_generator.py symlink into the sandbox directory
without the target.  Just remove the optimization for now since it
never works.

Test: m checkbuild
Change-Id: I96e8c0c145e7c99958639594edf8a93b69ae90eb
2020-11-25 18:28:58 +00:00
Paul Duffin 0cc047ad5d Improve TestGetDistForGoals debuggability
Previously, TestGetDistForGoals tested multiple test cases within a
single test so when it failed it was difficult to determine which test
case was the cause. This change runs each test case as its own
nested test.

It also corrects the order of expectedLine and line format parameters
to match the order in the message.

Test: m nothing
Bug: 174226317
Change-Id: I1408ec4125afc5c0b392cd7643dd3f630fe468e5
2020-11-25 17:02:32 +00:00
Thiébaud Weksteen 3c5905b0cb Ignore variant dependencies in rust-project.json
Source-generated library modules will have a dependency on the source
variant. This creates a cycle in the dependency graph which triggers a
warning from rust-analyzer. Ignore this type of dependency.

Test: m nothing
Change-Id: I13365093ebb88b00f6a72734b01114ec3e9a320e
2020-11-25 16:11:49 +01:00
Ulya Trafimovich 78a7155c17 Assume any <uses-library> is shared, add only toplevel ones to manifest.
This patch reworks the approach introduced in
https://r.android.com/1450819. That patch based the decision which
libraries should be added to the manifest <uses-library> tags by the
manifest_fixer on the "shared" status of the library.

That approach is incorrect for two reasons:

  - It doesn't make sense to have a non-shared library in class loader
    context ("shared" libraries are those specified in
    frameworks/base/data/etc/platform.xml, and they are the only ones
    that PackageManager knows about).

  - It doesn't make sense to add anything but the top-level of the
    class loader context tree to the manifest, because this part of the
    tree is flattened to a sequence, and PackageManager cannot restore
    it to the previous tree shape (there is an information loss).

This patch removes the "shared" bit of information from class loader
context elements and assumes that all libraries that end up in class
loader context are shared. Consequently, only the top-level libraries
should be passed to manifest_fixer.

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Bug: 168686456
Change-Id: I902690f0f38f1047fa79cf6ccbe956077eceaab0
2020-11-25 14:47:05 +00:00
Jingwen Chen 29c6597927 Merge "Comment android/config.go" am: 65d5b7f0e9
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1506854

Change-Id: I17c1ce874ed6d05ac918cbf17459456ea0e76ef7
2020-11-25 05:41:53 +00:00
Treehugger Robot 7ad5f7b0d3 Merge "Minor cleanup of environment.go." am: b21906cd70
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1509808

Change-Id: I853086ba409399d47ab4c7b0d884a0272e475f2c
2020-11-25 05:41:46 +00:00
Treehugger Robot 8c4748afa4 Merge "Minor cleanup of cleanbuild.go (and partially build.go)." am: bf6d82f929
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1509809

Change-Id: I691d3619a7efb2efcf67647dff45496c72fae240
2020-11-25 05:40:48 +00:00
Jingwen Chen 65d5b7f0e9 Merge "Comment android/config.go" 2020-11-25 04:18:56 +00:00
Treehugger Robot b21906cd70 Merge "Minor cleanup of environment.go." 2020-11-25 03:28:33 +00:00
Treehugger Robot bf6d82f929 Merge "Minor cleanup of cleanbuild.go (and partially build.go)." 2020-11-25 03:27:25 +00:00
Treehugger Robot 0acad3aa48 Merge "Add comments about sysprop_library" am: c2acb8cbf0
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1508076

Change-Id: Ib7d42cac8cec7a68569022d892800a9e42ee3056
2020-11-25 02:24:37 +00:00
Treehugger Robot c2acb8cbf0 Merge "Add comments about sysprop_library" 2020-11-25 00:54:14 +00:00