Commit Graph

27880 Commits

Author SHA1 Message Date
Treehugger Robot 5bbec1a371 Merge "Remove hardlink optmization in sbox" 2020-11-26 00:20:01 +00:00
Liz Kammer 0fd0e95e1b Merge "Refactor rust to use AndroidMkEntries" 2020-11-26 00:06:03 +00:00
Liz Kammer c55dcfcc2b Merge "Refactor python to use AndroidMkEntries." 2020-11-26 00:05:52 +00:00
Liz Kammer 6953adeb95 Merge "Convert csuite config to use AndroidMkEntries" 2020-11-26 00:05:30 +00:00
Treehugger Robot aa8ac24222 Merge "Soong: add -T to nanopb call to suppress timestamp" 2020-11-25 23:47:20 +00:00
Ulf Adams 82fd89b8ab Soong: add -T to nanopb call to suppress timestamp
By default, nanopb outputs a timestamp into the .c and .h files it generates.
This is usually considered bening. However, when using remote execution or
remote caching, this poisons all actions that have these files as inputs, and
also appears to affect transitive actions (actions depending on actions that
have these files as inputs).

This change adds the -T flag to suppress the timestamp, which in turn makes
the output files deterministic, which in turn allows the remote cache to
return previous results rather than requiring a re-execution of the
corresponding actions.

Test: ran the build a few times to check that the output is deterministic

Change-Id: Iecfa143dbf12a9bb7703dd0bc3f1b3e3fd56317c
2020-11-25 22:52:12 +01:00
Colin Cross b0ce5757b9 Merge "Use a unique depfile for each gensrcs command" 2020-11-25 21:18:13 +00:00
Colin Cross d03797e48a Remove hardlink optmization in sbox
The hardlink optimization attempts to save I/O by hardlinking input
files into the sandbox instead of copying them, but it never works on
a Linux build due to nsjail always making out and the source tree into
separate bind-mounted filesystems.  If the optimization did work it
would actually cause build failures when it hardlinked the relative
bionic/libc/fs_config_generator.py symlink into the sandbox directory
without the target.  Just remove the optimization for now since it
never works.

Test: m checkbuild
Change-Id: I96e8c0c145e7c99958639594edf8a93b69ae90eb
2020-11-25 18:28:58 +00:00
Paul Duffin 0cc047ad5d Improve TestGetDistForGoals debuggability
Previously, TestGetDistForGoals tested multiple test cases within a
single test so when it failed it was difficult to determine which test
case was the cause. This change runs each test case as its own
nested test.

It also corrects the order of expectedLine and line format parameters
to match the order in the message.

Test: m nothing
Bug: 174226317
Change-Id: I1408ec4125afc5c0b392cd7643dd3f630fe468e5
2020-11-25 17:02:32 +00:00
Thiébaud Weksteen 3c5905b0cb Ignore variant dependencies in rust-project.json
Source-generated library modules will have a dependency on the source
variant. This creates a cycle in the dependency graph which triggers a
warning from rust-analyzer. Ignore this type of dependency.

Test: m nothing
Change-Id: I13365093ebb88b00f6a72734b01114ec3e9a320e
2020-11-25 16:11:49 +01:00
Ulya Trafimovich 78a7155c17 Assume any <uses-library> is shared, add only toplevel ones to manifest.
This patch reworks the approach introduced in
https://r.android.com/1450819. That patch based the decision which
libraries should be added to the manifest <uses-library> tags by the
manifest_fixer on the "shared" status of the library.

That approach is incorrect for two reasons:

  - It doesn't make sense to have a non-shared library in class loader
    context ("shared" libraries are those specified in
    frameworks/base/data/etc/platform.xml, and they are the only ones
    that PackageManager knows about).

  - It doesn't make sense to add anything but the top-level of the
    class loader context tree to the manifest, because this part of the
    tree is flattened to a sequence, and PackageManager cannot restore
    it to the previous tree shape (there is an information loss).

This patch removes the "shared" bit of information from class loader
context elements and assumes that all libraries that end up in class
loader context are shared. Consequently, only the top-level libraries
should be passed to manifest_fixer.

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Bug: 168686456
Change-Id: I902690f0f38f1047fa79cf6ccbe956077eceaab0
2020-11-25 14:47:05 +00:00
Jingwen Chen 65d5b7f0e9 Merge "Comment android/config.go" 2020-11-25 04:18:56 +00:00
Treehugger Robot b21906cd70 Merge "Minor cleanup of environment.go." 2020-11-25 03:28:33 +00:00
Treehugger Robot bf6d82f929 Merge "Minor cleanup of cleanbuild.go (and partially build.go)." 2020-11-25 03:27:25 +00:00
Treehugger Robot c2acb8cbf0 Merge "Add comments about sysprop_library" 2020-11-25 00:54:14 +00:00
Jiyong Park b81b99032d Documenting apex/builer.go
Mostly documentation changes, but includes a few refactorings like
changing the variable names, reordering functions, reordering statements
in logical order, etc.

Bug: 173472337
Test: m
Change-Id: Iefdc8e463bcda60187b98e8c90661e220e8cdf40
2020-11-25 09:53:46 +09:00
Colin Cross 3ea4eb8d35 Use a unique depfile for each gensrcs command
gensrcs modules run the same command once for each input file to
produce each output file.  Each command needs its own depfile instead
of reusing one per shard.  The depfiles will be merged together
by RuleBuilder into one depfile per shard.

Test: TestGenSrcs
Change-Id: Iaf4f2cf9f5592c20e32944ddf34e0a61aff17ba8
2020-11-24 15:05:28 -08:00
Liz Kammer e2a17a5a40 Convert csuite config to use AndroidMkEntries
This is to facilitate consolidation of compatibility suite logic.

Test: m nothing and compare soong Android mk files -- no diffs
Change-Id: Ibc60d14fdde93835124e18a5c484bbda496bdb9e
2020-11-24 13:41:28 -08:00
Christopher Parsons dc71a6e28a Merge "Improve commenting of cc/binary.go." 2020-11-24 20:34:39 +00:00
Colin Cross 38214f5ac5 Merge changes I8af00c8c,Ia526ae89
* changes:
  Reland: Rewrite sbox to use a textproto manifest
  Fix gensrcs subdirs
2020-11-24 19:27:35 +00:00
Liz Kammer d8dceb0f2d Refactor python to use AndroidMkEntries.
This is to enable consolidating test compatibility suite handling in
Soong's AndroidMk files.

Test: go test soong tests
Test: m nothing and compare soong/Android mk files -- only diffs is that
      lines matching "^LOCAL_SHARED_LIBRARIES := $" are absent
Change-Id: I4922a1c8b0231cd98ebd2bc7257956eb9c0d240c
2020-11-24 10:10:03 -08:00
Liz Kammer 4018a8d4d7 Refactor rust to use AndroidMkEntries
This is to facilitate consolidation of compatibility suite logic.

Test: go soong tests
Test: m nothing & compare soong Android mk files -- only diffs are
      elimination of duplicate LOCAL_UNINSTALLABLE_MODULE := true lines
Change-Id: I76211505b2b6afde5d0431586b77d799cf70c6b9
2020-11-24 09:44:51 -08:00
Colin Cross e5218b6be6 Merge "Annotate dependency tags for dependencies of installed files" 2020-11-24 16:01:35 +00:00
Rupert Shuttleworth 1f304e674d Minor cleanup of cleanbuild.go (and partially build.go).
Test: TH presubmit.
Change-Id: I439c16687b1877d367ff3d0f27e30c9bac97ef58
2020-11-24 14:50:25 +00:00
Rupert Shuttleworth 80115fa6b0 Minor cleanup of environment.go.
Test: TH presubmit.
Change-Id: I89415d6b1219d2204b4fcce4b3a7456250cd0967
2020-11-24 13:16:30 +00:00
Thiébaud Weksteen f503dc3ba1 Merge "Add comments to {cc,rust}/strip.go" 2020-11-24 08:39:28 +00:00
Treehugger Robot fbb138cd7e Merge "Refactor queryview templates into a different file." 2020-11-24 07:54:51 +00:00
Treehugger Robot 5bca812a4e Merge "Fail if non-allowlisted host system tool is used." 2020-11-24 07:02:30 +00:00
Inseob Kim 07def12d50 Add comments about sysprop_library
Bug: 173473767
Test: m nothing
Change-Id: I96395742ae369baf9d54be8f92b01860ee0351ab
2020-11-24 14:13:31 +09:00
Treehugger Robot 547471cffc Merge "Remove restriction on exported plugins that generate APIs" 2020-11-24 05:06:40 +00:00
Treehugger Robot 333d235475 Merge "Documenting apex/apex.go" 2020-11-24 05:05:30 +00:00
Jingwen Chen f87a720101 Merge "Refactor 'in_make' to mean Kati is not skipped." 2020-11-24 02:09:21 +00:00
Colin Cross e9fe2949b8 Annotate dependency tags for dependencies of installed files
Relands Ic22603a5c0718b5a21686672a7471f952b4d1017 with a minor
change to track libc++ dependencies for python hosts and after
a fix to an internal genrule that depended on transitively
installed java libraries (ag/13068670).

Soong currently assumes that installed files should depend on
installed files of all transitive dependencies, which results
in extra installed file dependencies through genrules, static
libs, etc.

Annotate dependency tags for dependencies for which the
installed files are necessary such as shared libraries
and JNI libraries.

This avoids extra installed files, and is also a first step
towards genrules using their own copy of tools instead of
the installed copy.

Bug: 124313442
Test: m checkbuild
Test: java.TestBinary
Test: cc.TestInstallSharedLibs
Test: deptag_test.go
Change-Id: I725871249d561428e6f67bba6a7c65b580012b72
2020-11-23 18:06:08 -08:00
Jingwen Chen 1b461e41bd Merge "Move bazel_module property to a common file, and add it to filegroup." 2020-11-24 01:54:43 +00:00
Jingwen Chen 2838c818da Fail if non-allowlisted host system tool is used.
This CL improves the clarity of the non-hermetic nature of using
HostSystemTools, and make the build fail if a caller tries to use a host
system tool that isn't allowlisted in config.go.

The only caller seems to be for xcrun, which is a special case that is
pretty hard to remove.

Test: TH presubmit
Change-Id: Icd3e7330406a08c00ac9682e37e2d42669f2e6b7
2020-11-23 20:43:41 -05:00
Jingwen Chen c711fecfc1 Comment android/config.go
Fixes: 173973687
Test: TH presubmit
Change-Id: I4bab950e92a69b12cb7738515a6c516e5b273745
2020-11-23 20:27:45 -05:00
Jingwen Chen 1ad9dc00a0 Merge "Delete check for EXPERIMENTAL_JAVA_LANGUAGE_LEVEL_9." 2020-11-24 01:12:02 +00:00
Jiyong Park c0ec6f99d7 Documenting apex/apex.go
Mostly documentation changes, but includes a few refactorings like
changing the variable names, reording functions, reordering statements
in logical order, etc.

Bug: 173472337
Test: m

Change-Id: Ie1799c0972d63da823ad375f008018de782529d1
2020-11-24 09:28:50 +09:00
Colin Cross d2d8fd6f51 Merge changes I0f2f0e41,Ib034140c,I4b2ca283
* changes:
  Add java_data parameter to python modules
  Use local variations for python version splits
  Support SourceFileProducer in android.OutputFilesForModule
2020-11-23 23:20:06 +00:00
Colin Cross c9fe10f5b8 Remove restriction on exported plugins that generate APIs
hilt_android requires seven separate annotation processors, which
is only feasible to support using exported_plugins to avoid having
to list all seven in every module that uses it.  Unfortunately they
all set generates_api: true.  Turbine is already disabled for modules
that directly use a plugin that sets generates_api: true, because
turbine doesn't run annotation processors.  Also add support for
disabling turbine if a module transitively uses a plugin that
generates APIs via exported_plugins.

Bug: 173397767
Test: TestExportedPlugins
Change-Id: If70354a3dd67efb4ce88bc9c934d41ccb6241b28
2020-11-23 11:42:26 -08:00
Colin Cross 1bc63938f0 Add java_data parameter to python modules
csuite has a python module that wants to embed the outputs of java
modules.  This has caused issues with mismatched variants bewteen
the arch-specific python module and the common java modules.  Add
a java_data property that is similar to the data property but
uses the common arch variant.

Bug: 173977903
Test: m checkbuild
Change-Id: I0f2f0e4159650cd5d42b510d5177678e7ee91b4d
2020-11-23 18:35:55 +00:00
Colin Cross e20113d8ab Use local variations for python version splits
Use a local variation for python version splits.  This causes
dependencies from python modules not to look for variations
that match the version split, which will simplify a python module
depending on a java module (for example for test data) because
Soong won't look for a python version variant of the java module.

Bug: 173977903
Test: go test ./python
Change-Id: Ib034140c478ffbc7467ab830d3cfa5683c31d05c
2020-11-23 18:35:49 +00:00
Colin Cross 74b1e2b880 Support SourceFileProducer in android.OutputFilesForModule
Add support to android.OutputFilesForModule to get paths from a
SourceFileProducer as well as an OutputFileProducer.

Bug: 173977903
Test: m checkbuild
Change-Id: I4b2ca2837342ddbb4210bee8f549a636d8b8b049
2020-11-23 18:35:17 +00:00
Paul Duffin 0dabd31057 Merge "java_sdk_library: Allow dist artifacts to be named" 2020-11-23 12:17:45 +00:00
Jingwen Chen 17d690c8b0 Delete check for EXPERIMENTAL_JAVA_LANGUAGE_LEVEL_9.
This has been a no-op for a year now. a1c9e9da55

Test: TH presubmit
Change-Id: Id9cc56045cbca4ebae64170e5443501b9ccc8260
2020-11-23 05:00:42 -05:00
Mathew Inwood 6ba603c8a1 Merge "Explicitly tag max-target-o APIs as "lo-prio"." 2020-11-23 09:45:48 +00:00
Yo Chiang bb7895c85a Merge "prebuilt_etc: Refactoring" 2020-11-23 09:28:55 +00:00
Jingwen Chen cda22c9bb9 Refactor 'in_make' to mean Kati is not skipped.
In Nougat and before, Make wrote a marker file to indicate that
soong_build was invoked from Make to change certain behaviors of Soong
at build time.
https://cs.android.com/android/platform/superproject/+/android-7.1.2_r36:build/core/soong.mk;l=70-73;drc=ae18638b0406ad107b0882a02a13cdd8b92f2a4e

Things have changed, and now soong_build is invoked from soong_ui, which
supports a --skip-make configuration flag:
https://cs.android.com/android/platform/superproject/+/master:build/soong/ui/build/build.go;l=31-33;drc=680387bf1d3ce7cbc77f535be7c42cec411b1687

Thus, the various remnants of 'EmbeddedInMake' and 'inMake'
configuration are misleading, since soong_build is no longer invoked
from Make. This CL refactors all instances to actually mean that
Kati is enabled (not skipped with --skip-make), and will run after
soong_build finishes, so Kati-specific behavior like the AndroidMk
singleton should run.

Test: TH presubmit

Change-Id: I576ab8e54f99f5c8ddf9feaf9a828019b279e266
2020-11-23 00:29:18 -05:00
Jingwen Chen 30f5aaaa77 Move bazel_module property to a common file, and add it to filegroup.
This enables prototyping against aosp/1441774 to demonstrate mixed
builds with converted BUILD files.

Test: TH Presubmit
Test: USE_BAZEL_ANALYSIS=1 m libc && prebuilts/build-tools/linux-x86/bin/ninja -f out/combined-aosp_cf_x86_auto.ninja -t commands libc | grep bazel-out | wc -l # 2 build actions
Bug: 171263886

Change-Id: I7b5cd0449d043ba26a339a0ef98b562fc62e13c8
2020-11-22 22:01:44 -05:00
Treehugger Robot b26070efef Merge "Documenting android/apex.go" 2020-11-23 00:44:35 +00:00