Commit Graph

208 Commits

Author SHA1 Message Date
Treehugger Robot 4c2e4f381a Merge "Support ninja rsp files in soong_zip" 2020-08-24 18:40:56 +00:00
Martin Stjernholm fbb486fc7a Propagate sanitize.never flag in SDK snapshots.
This is necessary to not get dependencies on libclang_rt sanitizer libs
for the CRT objects which lead to cyclic dependencies in sanitizer
builds.

Test: `lunch aosp_blueline_hwasan-userdebug && m nothing`
  in a tree that has the prebuilts created and unzipped from
  `build/soong/scripts/build-aml-prebuilts.sh runtime-module-{sdk,host-exports}`
Bug: 151303681
Change-Id: I3f848a084280bdc3ade4b74df03e981d8cc61222
2020-08-21 18:43:51 +01:00
Colin Cross 053fca10c9 Support ninja rsp files in soong_zip
Add a -r argument to soong_zip that reads a list of files from a file
like the -l argument but treats it as a Ninja rsp file with escaping.
Replace the -l arguments in Soong that are using rsp files with -r.

Fixes: 162435077
Test: TestReadRespFile, TestZip
Change-Id: I4605312e99406ab1bd0c37af9c5ad212393f0403
2020-08-19 21:18:56 +00:00
Colin Cross aede88c1c7 Reland: Deduplicate APEX variants that would build identically
APEX variants that share the same SDK version and updatability
almost always use identical command line arguments to build but
with different intermediates directories.  This causes unnecessary
build time and disk space for duplicated work.

Deduplicate APEX variants that would build identically.  Create
aliases from the per-APEX variations to the new shared variations
so that the APEX modules can continue to depend on them via the
APEX name as the variation.

This has one significant change in behavior.  Before this change,
if an APEX had two libraries in its direct dependencies and one
of those libraries depended on the other, and the second library
had stubs, then the first library would depend on the implementation
of the second library and not the stubs.  After this change, if
the first library is also present in a second APEX but the second
library is not, then the common variant shared between the two
APEXes would use the stubs, not the implementation.

In a correctly configured set of build rules this change will
be irrelevant, because if the compilation worked for the second
APEX using stubs then it will work for the common variant using
stubs.  However, if an incorrect change to the build rules is
made this could lead to confusing errors, as a previously-working
common variant could suddenly stop building when a module is added
to a new APEX without its dependencies that require implementation
APIs to compile.

This change reduces the number of modules in an AOSP arm64-userdebug
build by 3% (52242 to 50586), reduces the number of variants of the
libcutils module from 74 to 53, and reduces the number of variants
of the massive libart[d] modules from 44 to 32.

This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix
to always mark permissions XML files of java_sdk_library modules as
unique per apex since they contain the APEX filename, and a fix
to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes
instead of DepIsInSameApex to check if two modules are in the same
apex to account for a module that depends on another in a way that
doesn't normally include the dependency in the APEX (e.g. a libs
property), but the dependency is directly included in the APEX.

Bug: 164216768
Test: go test ./build/soong/apex/...
Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-19 10:21:17 -07:00
Pete Bentley 0c7b26e0e2 Revert "Deduplicate APEX variants that would build identically"
This reverts commit d6b2525b00.

Reason for revert: <Breaks tests - confirmed by Forrest: b/165188843>

Change-Id: I7cb68cb87522415004390c0672dc774e0067b122
2020-08-18 13:44:59 +00:00
Colin Cross d6b2525b00 Deduplicate APEX variants that would build identically
APEX variants that share the same SDK version and updatability
almost always use identical command line arguments to build but
with different intermediates directories.  This causes unnecessary
build time and disk space for duplicated work.

Deduplicate APEX variants that would build identically.  Create
aliases from the per-APEX variations to the new shared variations
so that the APEX modules can continue to depend on them via the
APEX name as the variation.

This has one significant change in behavior.  Before this change,
if an APEX had two libraries in its direct dependencies and one
of those libraries depended on the other, and the second library
had stubs, then the first library would depend on the implementation
of the second library and not the stubs.  After this change, if
the first library is also present in a second APEX but the second
library is not, then the common variant shared between the two
APEXes would use the stubs, not the implementation.

In a correctly configured set of build rules this change will
be irrelevant, because if the compilation worked for the second
APEX using stubs then it will work for the common variant using
stubs.  However, if an incorrect change to the build rules is
made this could lead to confusing errors, as a previously-working
common variant could suddenly stop building when a module is added
to a new APEX without its dependencies that require implementation
APIs to compile.

This change reduces the number of modules in an AOSP arm64-userdebug
build by 3% (52242 to 50586), reduces the number of variants of the
libcutils module from 74 to 53, and reduces the number of variants
of the massive libart[d] modules from 44 to 32.

Bug: 164216768
Test: go test ./build/soong/apex/...
Change-Id: I0529837476a253c32b3dfb98dcccf107427c742c
2020-08-17 15:18:31 -07:00
Martin Stjernholm caa47d7eed Disable all host OS variants except those explicitly included.
Necessary to avoid problems with implicitly enabled host OS'es, e.g.
linux_glibc getting enabled when we only supply a linux_bionic linker
in runtime-module-host-exports. That will then cause a non-functional
prebuilt to take precedence over source if the prebuilt is preferred.

We don't do this for device since we so far only support a single
device OS (android).

This introduces the notion that SDK member types can be host OS
dependent or not. That way java members with host prebuilts don't get
restricted to a specific host OS.

Test: m nothing
Test: build/soong/scripts/build-aml-prebuilts.sh runtime-module-host-exports
  Check that the generated Android.bp correctly disables the bionic
  linker prebuilt for linux_glibc.
Test: art/build/apex/runtests.sh
  on master-art with an updated runtime SDK snapshot
Test: art/tools/buildbot-build.sh {--host,--target}
  on master-art with an updated runtime SDK snapshot
Bug: 160349757
Change-Id: Idad7ef138cdbcbd209d390bf6c10ca8365d4619f
2020-08-03 21:06:50 +01:00
Martin Stjernholm b4bfd148eb Merge "Set compile_multilib for SDK snapshots for device." 2020-08-03 17:55:25 +00:00
Paul Duffin 021f4e525f Fix prebuilt mutator ordering in tests
Previously, the prebuilt mutators were added by the
cc.RegisterRequiredBuildComponentsForTest() function as a convenience
but unfortunately it lead to some of the mutators being in a different
order in the tests than in the normal build.

This change:
* Extracts the RegisterPrebuiltMutators() call from
  cc.RegisterRequiredBuildComponentsForTest()
* Makes sure that the prebuilt mutators are registered before the
  visibility gatherer and enforcer mutators.

Bug: 162505935
Test: m nothing
Change-Id: I7d959b558200b502f0a5e4653c41ea01414e142a
2020-07-31 16:12:01 +01:00
Paul Duffin a37eca27c8 Avoid creating APEX variant for sdk member
Previously, an APEX variant was created for a module that was a member
of an SDK just in case it had to be replaced with an APEX requested
snapshotted version of that member. That was necessary because that was
the only way to have APEX specific replacements.

Since then a new method called ReplaceDependenciesIf() has been added
which provides fine grained control over which dependencies are
replaced. This change uses that new method to handle the replacements
which allows the APEX variants to be removed.

Bug: 161928524
Test: m nothing
Change-Id: If3869dd6753dc182b099af566b20fbc9c9c6eff7
2020-07-24 11:10:10 +01:00
Pete Gillin 84c3807b99 Enforce the legacy core/platform API restriction.
This change silently decides whether modules which depend on either
sdkPrivate or sdkCorePlatform get the legacy or the stable version of
the core/platform API, based on whether the module's name is on a
hard-coded list or not.

Test: m java
Test: make a target from the list when its entry is commented out, which correctly fails
Bug: 157640067
Change-Id: I15e5a6c2f07e73718803501d705de0d7ab9bec90
Merged-In: I15e5a6c2f07e73718803501d705de0d7ab9bec90
Merged-In: Iaa97ddaa015e8079fcb3426585c5101c7ec9e22a
(cherry picked from commit c0f4373106)
2020-07-23 09:06:52 +01:00
Treehugger Robot 0bcda81dd5 Merge "Generate module lib API and scope together" 2020-07-21 04:21:07 +00:00
Paul Duffin 15f34ef811 Generate module lib API and scope together
Previously, due to limitations in metalava the module lib API and stubs
source were generated separately. That limitation has since been
removed so this change optimizes the generating code.

Changes:
* Add an annotation field to apiScope to contain the annotation that
  identifies the scope.
* Rename droidstubArgs to extraArgs to better reflect that they are not
  passed to droidstubs.
* Rename droidstubsArgsForGeneratingStubsSource to droidstubsArgs.
* Remove droidstubsArgsForGeneratingApi and
  createStubsSourceAndApiTogether fields along with code that supports
  generating stubs source and the API separately.
* Fix the module lib test.

Test: m checkapi
Bug: 161364853
Change-Id: I821886bb68645addf5ae9c96ed2f4f3649151d1a
2020-07-20 20:51:25 +01:00
Martin Stjernholm 4cfa2c692b Set compile_multilib for SDK snapshots for device.
Previously compile_multilib was ignored for device and defaulted to
"both", which could lead to an internal inconsistency in the snapshot
if a member was 32- or 64-only.

This also implements common property optimisation for compile_multilib,
by adding a struct that can be extended to other properties in the
future.

Test: m nothing
Bug: 151303681
Change-Id: I3e96222858e5e755173aca03508a23c431977e2a
2020-07-17 14:32:15 +01:00
Martin Stjernholm 89238f4703 Add compile_multilib properties to all native prebuilts (reland).
There was a special case in cc_prebuilt_binary only, which resulted in
prebuilt libraries getting both 32 and 64 bit variants even when their
sources only had one of them, and the other variant would be defunct
since it wouldn't get any prebuilt artefact.

This moves the handling of compile_multilib completely to the common
update code, so that SDK members don't need to deal with it. It doesn't
take SDK member defaults into account, which means a bit more
boilerplate in the snapshots, but it's simpler and less error prone
(different SDK member types have different defaults).

This relands https://r.android.com/1359962 after disabling the SDK
tests on mac once and for all.

Bug: 151303681
Test: `go test -v ./sdk` in build/soong on mac and linux
Change-Id: I05f6603b2ac9b8676b25c3e297165ca23284e9cc
2020-07-17 14:32:15 +01:00
Martin Stjernholm 7feceb2354 Add support for SDK tests with LinuxBionic OS type.
Break apart test helpers a bit to make it possible to enable
LinuxBionic in a single test, and add LinuxBionic support to
cc.GatherRequiredDepsForTest.

Test: m nothing
Bug: 160349757
Change-Id: Iace1024c964cee2308c130c945daef9e46c18c66
2020-07-16 21:55:15 +01:00
Martin Stjernholm 3ff2e661bd Disable all SDK snapshot generation tests on mac.
We frequently miss putting in the per-test skip condition, and since
mac isn't tested in presubmit this leads to build breakages and time
consuming reverts. To avoid that this blanket disables all the SDK
tests on mac. It's not used there and we got test coverage in many
linux-based builds.

Bug: 145598135
Bug: 161315642
Test: `go test -v ./sdk` in build/soong on mac and linux
Change-Id: I2aea92fef2c0f8c2742396fe36610501dc5a6f0f
2020-07-15 16:58:05 +01:00
Jakub Pawlowski ca7b2c3eb7 Merge "Revert "Add compile_multilib properties to all native prebuilts."" 2020-07-15 10:33:41 +00:00
Charles Chen 29936ae5a7 Revert "Add compile_multilib properties to all native prebuilts."
This reverts commit 1e9c2677fd.

Reason for revert: Break build

Bug: 161315642
Change-Id: Ibf3d5e88412327e8f08d08272b75cd3df34a71e5
2020-07-15 04:01:45 +00:00
Treehugger Robot 6da69732e4 Merge "Add compile_multilib properties to all native prebuilts." 2020-07-15 02:51:21 +00:00
Treehugger Robot 9d98ac15bb Merge "Fix string concatenation error" 2020-07-12 23:08:40 +00:00
Martin Stjernholm 1e9c2677fd Add compile_multilib properties to all native prebuilts.
There was a special case in cc_prebuilt_binary only, which resulted in
prebuilt libraries getting both 32 and 64 bit variants even when their
sources only had one of them, and the other variant would be defunct
since it wouldn't get any prebuilt artefact.

This moves the handling of compile_multilib completely to the common
update code, so that SDK members don't need to deal with it. It doesn't
take SDK member defaults into account, which means a bit more
boilerplate in the snapshots, but it's simpler and less error prone
(different SDK member types have different defaults).

Bug: 151303681
Test: m nothing
Test: build/soong/scripts/build-aml-prebuilts.sh art-module-host-exports
  Check that the generated Android.bp passes Soong.
Change-Id: Ib73444c6788ee1c78480bdb103aa2b8ae8f2c63c
2020-07-11 00:38:24 +01:00
Colin Cross afa6a77c64 Fix string concatenation error
Fix an error caught by golint that prevents using go test.  string(int)
produces a rune, not a string representation of the int.  Use fmt.Errorf
instead to format the string.

Test: go test
Change-Id: I4c2e1e30a0f1b3dc24afd123c38c4e41c2abb47a
2020-07-10 16:32:49 -07:00
Paul Duffin ed62b9cc2a Differentiate sdk snapshot include dir by os type
Test: m nothing
Change-Id: Ide032aa64bb1fee69969c5e007bee6643d475309
2020-07-10 12:18:03 +01:00
Paul Duffin cee7e66b07 Ensure that sdk/module_exports depends on source members
Previously, preferring a prebuilt of an sdk/module_exports's member
would cause the sdk/module_exports to depend on the prebuilt instead
of the source and cause problems with the build.

This chance prevents the dependency from an sdk/module_exports to its
members from being replaced with prebuilts.

Bug: 160785918
Test: m nothing
Change-Id: Iee4bcd438c11929e30fb5766701b05a0e89956d9
2020-07-09 17:51:06 +01:00
Treehugger Robot 2c79c871d9 Merge changes Iefcf4fbe,I450d4769,I1ffc7e7b
* changes:
  Prepend default to allow overriding in the bp file.
  Do not follow SDK member dependencies for APEX payloads.
  Do not follow prebuiltDependencyTags in APEX payload walks.
2020-07-09 12:32:31 +00:00
Paul Duffin 44f1d8404b Stop java_sdk_library_import from depending on source modules
Previously, java_sdk_library_import added the dependencies on its child
components in the deps mutator after prebuilts without a matching
source module are renamed to the source module name. That meant that
the java_sdk_library_import has to use the source module name and ended
up depending on the source module unless it was preferred.

This change adds a new component deps mutator that runs before the
PrebuiltRenameMutator so that the java_sdk_library_import can add
dependencies onto the prebuilt modules. It also updates an affected
test.

Bug: 159902351
Test: m nothing
Change-Id: I3576c4873302743e51aff547ea1497bef6d748ac
2020-07-08 16:46:16 +01:00
Martin Stjernholm 26ab8e80e4 Prepend default to allow overriding in the bp file.
Bug: 143948100
Bug: 151303681
Test: m nothing
Change-Id: Iefcf4fbe9d2971ce267242185165f0c38f68db30
2020-07-07 18:14:26 +01:00
Martin Stjernholm cc77601d10 Do not follow SDK member dependencies for APEX payloads.
Test: m nothing
Test: `m` with prebuilts/runtime in the manifest (along with other
  fixes)
Bug: 151303681
Change-Id: I450d476975c7ab4434228b8c4baf3af192142211
2020-07-07 18:14:26 +01:00
Jingwen Chen 40fd90ae52 Support multiple dists per Android.bp module, and dist output selection.
This CL adds "dists" to the base property struct to support multiple
dist file configurations, and generic tag support to dist tagged outputs
of modules.

Fixes: b/152834186
Test: soong tests and `m sdk dist`

Change-Id: I80c86bc9b7b09e671f640a4480c45d438bdd9a2a
Signed-off-by: Jingwen Chen <jingwen@google.com>
2020-06-25 12:42:07 +00:00
Paul Duffin 4e77284ab4 Include resources in sdk snapshot of java library impl
Bug: 159607838
Test: m nothing
Change-Id: Ie7ddbffc3957f832abfadb27292c436e8e5dcbf3
2020-06-24 13:04:15 +01:00
Martin Stjernholm 47ed352227 Propagate unique_host_soname to cc_library prebuilts.
Test: m nothing
Test: Check on go/Android.bp that cc_prebuilt_library supports
  unique_host_soname
Bug: 158743135
Change-Id: Ie02a5cae057fb9092c226d8c5b7f63e1f66ad066
2020-06-17 23:06:43 +01:00
Paul Duffin 0c5bae5e83 java_sdk_library: Add system-server scope
Bug: 155164730
Test: m nothing
Merged-In: I49a2dab5c064b05f16691a3fae65f2b4ffc53bfd
Change-Id: I49a2dab5c064b05f16691a3fae65f2b4ffc53bfd
(cherry picked from commit 5a757b1ebb)
2020-06-15 15:02:19 +01:00
Colin Cross 440e0d0542 Use inclusive language in build/soong
Test: m checkbuild
Change-Id: Id07890b7cbc2397291a658ca00e86b43c743aafc
2020-06-11 15:33:16 -07:00
Pete Gillin 1f41dbff64 Split the core/platform API into stable and legacy versions.
For now, everything outside libcore still uses the legacy version.

Test: treehugger
Bug: 157640067
Change-Id: If5234e9ee533ff537926801a0af045d36b1caf01
2020-06-08 10:53:51 +01:00
Jaewoong Jung 264700345d Merge "Soong package structure refactoring" 2020-06-02 16:30:50 +00:00
Paul Duffin 3dbf9fd892 Copy removed.txt file to the snapshot correctly
Previously, the code copied the current.txt file to both the
current_api and removed_api properties. This change copies the
removed.txt file to the removed_api property instead.

Bug: 157980685
Test: m nothing
Change-Id: Iad34e91051da43222d22c240c16f50887c43d73e
2020-06-02 14:40:28 +01:00
Jaewoong Jung 4b79e98a6e Soong package structure refactoring
Give prebuilt_etc and sh_binary their own packages and split the
gigantic main Android.bp up to small, per-package ones.

Test: m nothing, TreeHugger
Bug: 156980228
Change-Id: I7b00cd344b9f16861f1ff39edf0029f016b853d0
2020-06-01 13:44:48 -07:00
Martin Stjernholm a6a224586b Merge "Propagate flags necessary for a prebuilt of the Bionic linker." 2020-06-01 09:55:10 +00:00
Martin Stjernholm 7130fabd20 Propagate flags necessary for a prebuilt of the Bionic linker.
The flags are necessary to avoid implicit dependencies on crt libs,
libc, and the linker (recursively). The reason cc_prebuilt_binary react
to these flags is that they can affect the install rules.

Test: m nothing
Test: art/test/testrunner/run_build_test_target.py art-linux-bionic-x64
  on a master-art branch using snapshot built from runtime-module-sdk
  and runtime-module-host-exports
Bug: 152255951
Change-Id: I6b2fe92d105d1f446fffd00bd3267a416f75efb7
2020-05-28 22:58:01 +01:00
Paul Duffin d7eb1c2a76 java_sdk_library: Propagate shared_library to snapshot
Passes the shared_library property through to the snapshot. It does not
optimize away the default value in order to make it easier to invert
the default value in future. The current default value was only chosen
for convenience because most existing usages were already treated as
shared libraries. It would be safer if modules had to opt in to be used
as shared libraries.

Bug: 155164730
Test: m nothing
Change-Id: I33c7323f2389b44ed49cebe517ae2cce349117f1
2020-05-28 17:39:25 +01:00
Paul Duffin 6b3ca917b0 Merge "Syntax check generated Android.bp snapshot" 2020-05-19 07:53:27 +00:00
Treehugger Robot e3fa145692 Merge "Fix special case when appending a visibility rule to //visibility:private." 2020-05-15 14:08:24 +00:00
Martin Stjernholm 08f72a9709 Merge changes I404f6e5c,I0d2d86f0
* changes:
  Add explicit visibility on the self package in EffectiveVisibilityRules.
  Don't print :__pkg__ for package rules, since that's the default.
2020-05-14 21:49:05 +00:00
Martin Stjernholm 64aeaaddef Fix special case when appending a visibility rule to //visibility:private.
Test: m nothing
Bug: 151303681
Change-Id: Ibe7e953219e4660c653063e87cb08407a2cb513d
2020-05-14 16:52:24 +01:00
Martin Stjernholm 0641d189d0 Add explicit visibility on the self package in EffectiveVisibilityRules.
Modules in a package is implicitly visible within the same package.
When making effective visibility rules which work from another location
that visibility needs to be mentioned explicitly. In practice this
allows prebuilts to remain visible to other modules in the source
package when the prebuilts are preferred.

Test: m nothing
Test: m conscrypt-module-sdk, check that the snapshot Soong modules give
  visibility to //external/conscrypt
Bug: 151303681
Change-Id: I404f6e5ca7021974a8c7be5e4d6b4982c050b8a5
2020-05-14 16:50:59 +01:00
Martin Stjernholm 01407c525a Don't print :__pkg__ for package rules, since that's the default.
To reduce clutter a little.

Test: m nothing
Change-Id: I0d2d86f00c3d38e603d6f58c4b1dd9a4170bf819
2020-05-14 16:50:59 +01:00
Paul Duffin 6b836ba566 Correct module_lib scope name
The module_lib scope should be called module-lib in order to pick up
the latest filegroup. Without it the API lint does not use a baseline
and so reports issues with released and unchangeable APIs.

It is also needed for the correct dist path.

Test: m update-api
Bug: 155164730
Change-Id: I7dbafd7164d5da600ca45c39a7f93a5a40027a1f
2020-05-14 00:53:50 +01:00
Paul Duffin f7a6433bab java_sdk_library: Propagate naming_scheme to snapshot
Bug: 155164730
Test: m nothing
Change-Id: Iec7c52d2102b6294540090f3c1e32bd9d8cc7da8
2020-05-13 20:40:53 +01:00
Paul Duffin 87a05a389d java_sdk_library: Add sdk_version for API scopes
Allow the sdk_version against which the stubs for a scope are compiled
to be specified in the module on a per scope basis.

Bug: 155164730
Test: m nothing
Change-Id: I5881e5ee7c2169c30f544882344a60a602dae917
2020-05-13 13:09:23 +01:00