This change ensures that each prebuilt in a snapshot explicitly
specifies its visibility even when that visibility is the current
default. This is done for two reasons:
1. It simplifies a follow up change that adds visibility rules to an
existing set of rules.
2. It ensures that the snapshots are independent of the current Soong
default visibility.
The latter is important because we intend to switch from modules being
visible to everyone by default (i.e. //visibility:public) to only being
visible to modules in the same package (i.e. //visibility:private). By
making the snapshots of modules that do not specify any visibility
explicitly specify that they are "//visibility:public" it ensures that
the snapshots will not need to be changed when the default changes.
Bug: 168301990
Test: m nothing
Change-Id: Ia034f4a1e5124c17f46d73b0e9a6c5f2a251038e
This is a quick fix to avoid build failures due to prebuilts lacking
stubs versions that the sources have, which causes dependencies on
older version stubs fail on master-art-host where the sources of e.g.
Bionic are no longer available.
More work is required to make the version handling of stubs DTRT - a
TODO comment outlines it.
Test: m nothing
Test: `m runtime-module-sdk`, check that the generated Android.bp
has the full stubs.versions list like the sources.
Bug: 169373910
Change-Id: I0ebfaf94f9d45a0e58d9785c40e7cea626f0ab83
When generating Javadoc the processor needs to be given information
about the doctags that are present in the source. This change allows
that information to be managed with the java_sdk_library that generates
the stubs source from which the Javadoc is generated.
Bug: 168301990
Test: Built offline-sdk-docs with and without the change and
diffed them. The only difference was the timestamp.js
file.
Change-Id: I4adbeb0781bc2191461fec856ffa90ea185e7434
liblog snapshot needs to sanitizer.address=false to avoid cycle in asan
builds. Adding that separately in library_sdk_member.go would start to
feel like whack-a-mole, so the snapshot generation is instead extended
to handle nested property structs.
This uses the BpPropertySet.AddProperty extension in
https://r.android.com/1423510, and common value optimisation now
recurses into non-anonymous structs, instead of comparing them as a
whole.
Test: m nothing
Test: `m SANITIZE_TARGET=address nothing` with prebuilts/runtime
present in the manifest and a fresh snapshot made with this
Bug: 151303681
Change-Id: I472554117a488e6c800045cb2ed59377778571a4
Both will create a nested property set, that may be merged with an
existing one.
Test: m nothing
Bug: 151303681
Change-Id: I30696ba3eb8960ca6fa54c9ee2cf6229ab9f5da9
A host target is considered as being cross-compiled when the target
can't run natively on the build machine. For example, linux_glibc/x86_64
is a non-cross target on a standard x86/Linux machine, but is a cross
host on Mac. Previously, whether cross or not was a static attribute of
an OsType. For example, Windows was always considered as cross host,
while linux_bionic was not. This becomes a problem when we support more
host targets like linux_bionic/arm64 which should be cross-host on
standard x86/Linux machines.
This change removes HostCross from the OsClass type and instead adds a
property HostCross to the Target type. When a target is being added, it
is initialized to true when the target can't run natively on the current
build machine.
Bug: 168086242
Test: m
Change-Id: Ic37c8db918873ddf324c86b12b5412952b0f2be2
Needed for jni_headers which is marked as recovery_available because libchrome
is also marked and that library is used by others that are the recovery
partition. There may be other dependency paths that require jni_headers to be
recovery_available that was just the first one found.
Bug: 168301990
Test: m nothing
Change-Id: I916ecfe7700b9c153b29115e849da148fc7e7a76
The naming_scheme property was not removed as it may be useful for
future when migrating to java_sdk_library.
Bug: 168301990
Test: m nothing
Change-Id: Ie97dd60355a207f1312a2dd910f1fb25b46fd737
This is necessary to not get dependencies on libclang_rt sanitizer libs
for the CRT objects which lead to cyclic dependencies in sanitizer
builds.
Test: `lunch aosp_blueline_hwasan-userdebug && m nothing`
in a tree that has the prebuilts created and unzipped from
`build/soong/scripts/build-aml-prebuilts.sh runtime-module-{sdk,host-exports}`
Bug: 151303681
Change-Id: I3f848a084280bdc3ade4b74df03e981d8cc61222
Add a -r argument to soong_zip that reads a list of files from a file
like the -l argument but treats it as a Ninja rsp file with escaping.
Replace the -l arguments in Soong that are using rsp files with -r.
Fixes: 162435077
Test: TestReadRespFile, TestZip
Change-Id: I4605312e99406ab1bd0c37af9c5ad212393f0403
APEX variants that share the same SDK version and updatability
almost always use identical command line arguments to build but
with different intermediates directories. This causes unnecessary
build time and disk space for duplicated work.
Deduplicate APEX variants that would build identically. Create
aliases from the per-APEX variations to the new shared variations
so that the APEX modules can continue to depend on them via the
APEX name as the variation.
This has one significant change in behavior. Before this change,
if an APEX had two libraries in its direct dependencies and one
of those libraries depended on the other, and the second library
had stubs, then the first library would depend on the implementation
of the second library and not the stubs. After this change, if
the first library is also present in a second APEX but the second
library is not, then the common variant shared between the two
APEXes would use the stubs, not the implementation.
In a correctly configured set of build rules this change will
be irrelevant, because if the compilation worked for the second
APEX using stubs then it will work for the common variant using
stubs. However, if an incorrect change to the build rules is
made this could lead to confusing errors, as a previously-working
common variant could suddenly stop building when a module is added
to a new APEX without its dependencies that require implementation
APIs to compile.
This change reduces the number of modules in an AOSP arm64-userdebug
build by 3% (52242 to 50586), reduces the number of variants of the
libcutils module from 74 to 53, and reduces the number of variants
of the massive libart[d] modules from 44 to 32.
This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix
to always mark permissions XML files of java_sdk_library modules as
unique per apex since they contain the APEX filename, and a fix
to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes
instead of DepIsInSameApex to check if two modules are in the same
apex to account for a module that depends on another in a way that
doesn't normally include the dependency in the APEX (e.g. a libs
property), but the dependency is directly included in the APEX.
Bug: 164216768
Test: go test ./build/soong/apex/...
Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
APEX variants that share the same SDK version and updatability
almost always use identical command line arguments to build but
with different intermediates directories. This causes unnecessary
build time and disk space for duplicated work.
Deduplicate APEX variants that would build identically. Create
aliases from the per-APEX variations to the new shared variations
so that the APEX modules can continue to depend on them via the
APEX name as the variation.
This has one significant change in behavior. Before this change,
if an APEX had two libraries in its direct dependencies and one
of those libraries depended on the other, and the second library
had stubs, then the first library would depend on the implementation
of the second library and not the stubs. After this change, if
the first library is also present in a second APEX but the second
library is not, then the common variant shared between the two
APEXes would use the stubs, not the implementation.
In a correctly configured set of build rules this change will
be irrelevant, because if the compilation worked for the second
APEX using stubs then it will work for the common variant using
stubs. However, if an incorrect change to the build rules is
made this could lead to confusing errors, as a previously-working
common variant could suddenly stop building when a module is added
to a new APEX without its dependencies that require implementation
APIs to compile.
This change reduces the number of modules in an AOSP arm64-userdebug
build by 3% (52242 to 50586), reduces the number of variants of the
libcutils module from 74 to 53, and reduces the number of variants
of the massive libart[d] modules from 44 to 32.
Bug: 164216768
Test: go test ./build/soong/apex/...
Change-Id: I0529837476a253c32b3dfb98dcccf107427c742c
Necessary to avoid problems with implicitly enabled host OS'es, e.g.
linux_glibc getting enabled when we only supply a linux_bionic linker
in runtime-module-host-exports. That will then cause a non-functional
prebuilt to take precedence over source if the prebuilt is preferred.
We don't do this for device since we so far only support a single
device OS (android).
This introduces the notion that SDK member types can be host OS
dependent or not. That way java members with host prebuilts don't get
restricted to a specific host OS.
Test: m nothing
Test: build/soong/scripts/build-aml-prebuilts.sh runtime-module-host-exports
Check that the generated Android.bp correctly disables the bionic
linker prebuilt for linux_glibc.
Test: art/build/apex/runtests.sh
on master-art with an updated runtime SDK snapshot
Test: art/tools/buildbot-build.sh {--host,--target}
on master-art with an updated runtime SDK snapshot
Bug: 160349757
Change-Id: Idad7ef138cdbcbd209d390bf6c10ca8365d4619f
Previously, the prebuilt mutators were added by the
cc.RegisterRequiredBuildComponentsForTest() function as a convenience
but unfortunately it lead to some of the mutators being in a different
order in the tests than in the normal build.
This change:
* Extracts the RegisterPrebuiltMutators() call from
cc.RegisterRequiredBuildComponentsForTest()
* Makes sure that the prebuilt mutators are registered before the
visibility gatherer and enforcer mutators.
Bug: 162505935
Test: m nothing
Change-Id: I7d959b558200b502f0a5e4653c41ea01414e142a
Previously, an APEX variant was created for a module that was a member
of an SDK just in case it had to be replaced with an APEX requested
snapshotted version of that member. That was necessary because that was
the only way to have APEX specific replacements.
Since then a new method called ReplaceDependenciesIf() has been added
which provides fine grained control over which dependencies are
replaced. This change uses that new method to handle the replacements
which allows the APEX variants to be removed.
Bug: 161928524
Test: m nothing
Change-Id: If3869dd6753dc182b099af566b20fbc9c9c6eff7
This change silently decides whether modules which depend on either
sdkPrivate or sdkCorePlatform get the legacy or the stable version of
the core/platform API, based on whether the module's name is on a
hard-coded list or not.
Test: m java
Test: make a target from the list when its entry is commented out, which correctly fails
Bug: 157640067
Change-Id: I15e5a6c2f07e73718803501d705de0d7ab9bec90
Merged-In: I15e5a6c2f07e73718803501d705de0d7ab9bec90
Merged-In: Iaa97ddaa015e8079fcb3426585c5101c7ec9e22a
(cherry picked from commit c0f4373106)
Previously, due to limitations in metalava the module lib API and stubs
source were generated separately. That limitation has since been
removed so this change optimizes the generating code.
Changes:
* Add an annotation field to apiScope to contain the annotation that
identifies the scope.
* Rename droidstubArgs to extraArgs to better reflect that they are not
passed to droidstubs.
* Rename droidstubsArgsForGeneratingStubsSource to droidstubsArgs.
* Remove droidstubsArgsForGeneratingApi and
createStubsSourceAndApiTogether fields along with code that supports
generating stubs source and the API separately.
* Fix the module lib test.
Test: m checkapi
Bug: 161364853
Change-Id: I821886bb68645addf5ae9c96ed2f4f3649151d1a
Previously compile_multilib was ignored for device and defaulted to
"both", which could lead to an internal inconsistency in the snapshot
if a member was 32- or 64-only.
This also implements common property optimisation for compile_multilib,
by adding a struct that can be extended to other properties in the
future.
Test: m nothing
Bug: 151303681
Change-Id: I3e96222858e5e755173aca03508a23c431977e2a
There was a special case in cc_prebuilt_binary only, which resulted in
prebuilt libraries getting both 32 and 64 bit variants even when their
sources only had one of them, and the other variant would be defunct
since it wouldn't get any prebuilt artefact.
This moves the handling of compile_multilib completely to the common
update code, so that SDK members don't need to deal with it. It doesn't
take SDK member defaults into account, which means a bit more
boilerplate in the snapshots, but it's simpler and less error prone
(different SDK member types have different defaults).
This relands https://r.android.com/1359962 after disabling the SDK
tests on mac once and for all.
Bug: 151303681
Test: `go test -v ./sdk` in build/soong on mac and linux
Change-Id: I05f6603b2ac9b8676b25c3e297165ca23284e9cc
Break apart test helpers a bit to make it possible to enable
LinuxBionic in a single test, and add LinuxBionic support to
cc.GatherRequiredDepsForTest.
Test: m nothing
Bug: 160349757
Change-Id: Iace1024c964cee2308c130c945daef9e46c18c66
We frequently miss putting in the per-test skip condition, and since
mac isn't tested in presubmit this leads to build breakages and time
consuming reverts. To avoid that this blanket disables all the SDK
tests on mac. It's not used there and we got test coverage in many
linux-based builds.
Bug: 145598135
Bug: 161315642
Test: `go test -v ./sdk` in build/soong on mac and linux
Change-Id: I2aea92fef2c0f8c2742396fe36610501dc5a6f0f
There was a special case in cc_prebuilt_binary only, which resulted in
prebuilt libraries getting both 32 and 64 bit variants even when their
sources only had one of them, and the other variant would be defunct
since it wouldn't get any prebuilt artefact.
This moves the handling of compile_multilib completely to the common
update code, so that SDK members don't need to deal with it. It doesn't
take SDK member defaults into account, which means a bit more
boilerplate in the snapshots, but it's simpler and less error prone
(different SDK member types have different defaults).
Bug: 151303681
Test: m nothing
Test: build/soong/scripts/build-aml-prebuilts.sh art-module-host-exports
Check that the generated Android.bp passes Soong.
Change-Id: Ib73444c6788ee1c78480bdb103aa2b8ae8f2c63c
Fix an error caught by golint that prevents using go test. string(int)
produces a rune, not a string representation of the int. Use fmt.Errorf
instead to format the string.
Test: go test
Change-Id: I4c2e1e30a0f1b3dc24afd123c38c4e41c2abb47a
Previously, preferring a prebuilt of an sdk/module_exports's member
would cause the sdk/module_exports to depend on the prebuilt instead
of the source and cause problems with the build.
This chance prevents the dependency from an sdk/module_exports to its
members from being replaced with prebuilts.
Bug: 160785918
Test: m nothing
Change-Id: Iee4bcd438c11929e30fb5766701b05a0e89956d9
* changes:
Prepend default to allow overriding in the bp file.
Do not follow SDK member dependencies for APEX payloads.
Do not follow prebuiltDependencyTags in APEX payload walks.
Previously, java_sdk_library_import added the dependencies on its child
components in the deps mutator after prebuilts without a matching
source module are renamed to the source module name. That meant that
the java_sdk_library_import has to use the source module name and ended
up depending on the source module unless it was preferred.
This change adds a new component deps mutator that runs before the
PrebuiltRenameMutator so that the java_sdk_library_import can add
dependencies onto the prebuilt modules. It also updates an affected
test.
Bug: 159902351
Test: m nothing
Change-Id: I3576c4873302743e51aff547ea1497bef6d748ac
Test: m nothing
Test: `m` with prebuilts/runtime in the manifest (along with other
fixes)
Bug: 151303681
Change-Id: I450d476975c7ab4434228b8c4baf3af192142211
This CL adds "dists" to the base property struct to support multiple
dist file configurations, and generic tag support to dist tagged outputs
of modules.
Fixes: b/152834186
Test: soong tests and `m sdk dist`
Change-Id: I80c86bc9b7b09e671f640a4480c45d438bdd9a2a
Signed-off-by: Jingwen Chen <jingwen@google.com>
Test: m nothing
Test: Check on go/Android.bp that cc_prebuilt_library supports
unique_host_soname
Bug: 158743135
Change-Id: Ie02a5cae057fb9092c226d8c5b7f63e1f66ad066
Previously, the code copied the current.txt file to both the
current_api and removed_api properties. This change copies the
removed.txt file to the removed_api property instead.
Bug: 157980685
Test: m nothing
Change-Id: Iad34e91051da43222d22c240c16f50887c43d73e
Give prebuilt_etc and sh_binary their own packages and split the
gigantic main Android.bp up to small, per-package ones.
Test: m nothing, TreeHugger
Bug: 156980228
Change-Id: I7b00cd344b9f16861f1ff39edf0029f016b853d0
The flags are necessary to avoid implicit dependencies on crt libs,
libc, and the linker (recursively). The reason cc_prebuilt_binary react
to these flags is that they can affect the install rules.
Test: m nothing
Test: art/test/testrunner/run_build_test_target.py art-linux-bionic-x64
on a master-art branch using snapshot built from runtime-module-sdk
and runtime-module-host-exports
Bug: 152255951
Change-Id: I6b2fe92d105d1f446fffd00bd3267a416f75efb7
Passes the shared_library property through to the snapshot. It does not
optimize away the default value in order to make it easier to invert
the default value in future. The current default value was only chosen
for convenience because most existing usages were already treated as
shared libraries. It would be safer if modules had to opt in to be used
as shared libraries.
Bug: 155164730
Test: m nothing
Change-Id: I33c7323f2389b44ed49cebe517ae2cce349117f1
Modules in a package is implicitly visible within the same package.
When making effective visibility rules which work from another location
that visibility needs to be mentioned explicitly. In practice this
allows prebuilts to remain visible to other modules in the source
package when the prebuilts are preferred.
Test: m nothing
Test: m conscrypt-module-sdk, check that the snapshot Soong modules give
visibility to //external/conscrypt
Bug: 151303681
Change-Id: I404f6e5ca7021974a8c7be5e4d6b4982c050b8a5
The module_lib scope should be called module-lib in order to pick up
the latest filegroup. Without it the API lint does not use a baseline
and so reports issues with released and unchangeable APIs.
It is also needed for the correct dist path.
Test: m update-api
Bug: 155164730
Change-Id: I7dbafd7164d5da600ca45c39a7f93a5a40027a1f
Allow the sdk_version against which the stubs for a scope are compiled
to be specified in the module on a per scope basis.
Bug: 155164730
Test: m nothing
Change-Id: I5881e5ee7c2169c30f544882344a60a602dae917
The API file for a scope represents the differences between the API
provided by that scope and that provided by the scope that it extends.
On the other hand the stubs source for a scope represents the union of
the API provided by the scope and the scope it extends (all the way
back to public).
Unfortunately, while metalava supports this behavior for scopes that
extend public (e.g. system and test) it does not support this behavior
for scopes that extend others, e.g. module_lib which extends system.
This is because it always assumes that the baseline for the API file
is 'public' and so has no way to defined other baselines.
This change works around that by having separate droidstubs modules to
generate the API and stubs sources for scopes that require different
arguments to generate the API and stubs sources.
Test: m checkapi
Bug: 155164730
Change-Id: Iea7d59852d7aeb503120acf3c44e08eb0d9d07b9
Checks the syntax of the Android.bp file generated for the sdk
snapshot. While this will not detect all possible problems with the
generated Android.bp file it will detect some and it is far better to
detect those issues during generation than when it is unpacked and
used.
Bug: 155628860
Test: m nothing
Change-Id: Ieec86a7a49fa2e3bd8b9f83aca540114232a3476
Previously, when writing to generatedContents \n characters were
quoted (replaced with \\n) so as to allow them to be preserved through
ninja/rsp/bash and were unquoted (replaced \\n with \n) just before
redirecting to the output file. That meant that any code which wanted
to access the contents for other purposes, e.g. testing had to unquote
\\n.
This change moves the quoting to be part of the code that generates the
ninja rule which simplifies any other code that has to access the
contents.
Without quoting the generated Android.bp files are not formatted
properly, are all on one line and completely unreadable.
Bug: 156286550
Test: m art-module-sdk and check generated Android.bp file to make sure
it is properly formatted.
Change-Id: I768c3b96ed08a3daf251730e2a10d9d72338c49a
Change since last attempt: Disable test that breaks on darwin.
Adds a test that fails with unknown property android.stubs.versions
and then fixes that by marking the field from which that property is
created with 'ignored-on-host' and implemented the isHostVariant on
*osTypeSpecificInfo.
Bug: 155628860
Test: m nothing
Change-Id: Ia4e744c9e799d5adaf8a2f761516f568ec363ad4
Adds a filter mechanism that can exclude property values from being
included in the common value extraction. That is needed to prevent the
snapshot mechanism from generating invalid output for properties that
are ignored on host (and have their values cleared) and which are not
tagged with `android:"arch_variant"`.
Changes:
* Updates the documentation of SdkMemberType to explain what effect
the 'ignored-on-host' tag has.
* Adds some tests for this new mechanism.
Bug: 155628860
Test: m nothing
Change-Id: Ibafdb6e921ba5abe505bd8a91ca5a1d9c9b5d0cb
Previously, the snapshot code did not know whether a specific property
could be arch specific or not and assumed that they all were which
meant that it could generate snapshots containing arch specific values
for properties that are not arch specific and so would fail when
unpacked.
This change requires arch specific fields in SdkMemberProperties to be
tagged as such using `android:"arch_variant"` (just as in module input
property structures). Any property without that must have properties
that are common across all variants.
Bug: 155628860
Test: m nothing
Change-Id: I3df60f0b53ba02ec2c55a80c7da058eac5909d26
* changes:
Revert "Detect invalid arch specific properties in snapshot"
Revert "Adds support for 'ignored-on-host'"
Revert "Fix snapshot of a host/device cc_library with stubs"
Revert submission 1302576
Bug: 156054601
Reason for revert: Presumed root cause of build break.
Reverted Changes:
Ifc8116e11:Detect invalid arch specific properties in snapsho...
I7ebd33307:Adds support for 'ignored-on-host'
I167b47a13:Fix snapshot of a host/device cc_library with stub...
Change-Id: Id7eba0bdde5c579e10e9b42d94a7cfab5f34995f
Revert submission 1302576
Bug: 156054601
Reason for revert: Presumed root cause of build break.
Reverted Changes:
Ifc8116e11:Detect invalid arch specific properties in snapsho...
I7ebd33307:Adds support for 'ignored-on-host'
I167b47a13:Fix snapshot of a host/device cc_library with stub...
Change-Id: I2a7ac0ef0232177eefc26542c11dc675d6f4cab2
Revert submission 1302576
Bug: 156054601
Reason for revert: Presumed root cause of build break.
Reverted Changes:
Ifc8116e11:Detect invalid arch specific properties in snapsho...
I7ebd33307:Adds support for 'ignored-on-host'
I167b47a13:Fix snapshot of a host/device cc_library with stub...
Change-Id: Ibccce5286605bb71c6be3b3550ba86d8b7e24fa7
Adds two new properties to specify the visibility of the stubs modules
(source and library) created by the java_sdk_library. Excludes
visibility property from being inherited when creating the module so it
can be properly specified by copying across the relevant property.
Test: m checkapi
Bug: 155164730
Change-Id: Iffdd9f191ff0d74646356ac577560cc38efdd790
Adds api scope for the module_lib and adds that to the list of all
all available scopes. The scope is generated if and only if the
api_surfaces property contains "module_lib".
No other changes are needed as the generation of the APIs is completely
driven by the allApiScopes array and the information in its contained
apiScope structures.
Test: m checkapi
Bug: 155164730
Change-Id: I7769af6823badca8715a270f86cf53b4e954b7df
Adds a per scope set of properties that allows explicit control over
the API surfaces generated.
Previously, the term active was used to determine whether it was
generated but that was a little abstract and unclear so has been
replaced by generated.
Test: m nothing
Bug: 155164730
Change-Id: I7539d89618b61f6b9d1a4b60cc3f9614b157f0d9
Adds a test that fails with unknown property android.stubs.versions
and then fixes that by marking the field from which that property is
created with 'ignored-on-host' and implemented the isHostVariant on
*osTypeSpecificInfo.
Bug: 155628860
Test: m nothing
Change-Id: I167b47a1374f541aa09d7e045972d740f1d9009c
Adds a filter mechanism that can exclude property values from being
included in the common value extraction. That is needed to prevent the
snapshot mechanism from generating invalid output for properties that
are ignored on host (and have their values cleared) and which are not
tagged with `android:"arch_variant"`.
Changes:
* Updates the documentation of SdkMemberType to explain what effect
the 'ignored-on-host' tag has.
* Adds some tests for this new mechanism.
Bug: 155628860
Test: m nothing
Change-Id: I7ebd333079619dba546bc8c4911d567e0287b676
Previously, the snapshot code did not know whether a specific property
could be arch specific or not and assumed that they all were which
meant that it could generate snapshots containing arch specific values
for properties that are not arch specific and so would fail when
unpacked.
This change requires arch specific fields in SdkMemberProperties to be
tagged as such using `android:"arch_variant"` (just as in module input
property structures). Any property without that must have properties
that are common across all variants.
Bug: 155628860
Test: m nothing
Change-Id: Ifc8116e11d987cfe7aec2eeaa964f3bbf36b5dc2
Refactoring in preparation for follow up changes.
Also:
* Adds a new AssertErrorMessageEquals() helper method.
* Improved error reporting in the accessor and added name to
extractorProperty to ensure meaningful errors are reported.
* Added String() string method to propertiesContainer.
* Reports errors using the field name as the errors are not really
fixable by developers and it is more meaningful to the build team.
Bug: 155628860
Test: m nothing
Change-Id: I5c5b8436bcbc39e4e7cd35df2577b2dac53e702a
Refactors the existing code to wrap a getter with a structure in order
to simplify adding additional per-property information. Adds an
emptyValue field to contain the empty value appropriate for the field
type.
Bug: 155628860
Test: m nothing
Change-Id: I7e169bd9b6e51b88e35bdf8673a5056337f778c1
This allows additional metadata to be associated with a set of
properties that will be used in a follow up change to filter
properties that are ignored in some variants (e.g. host variants).
Bug: 155628860
Test: m nothing
Change-Id: Ie83c1ffbb71fd5d7a08d350571dad3e3209b1431
This change fixes an issue with the definition of the
android_stubs_current (and similar) modules in the sdk tests. They were
incorrectly defined as java_sdk_library_import and not java_import.
Test: m nothing
Bug: 155164730
Change-Id: I4e9bfebdd2ff0a374e12284ccdf5ee5518968969
Test: m nothing
Test: Build an SDK snapshot including libc, check that the generated
cc_prebuilt_library_shared for it is a stub that doesn't depend on
ld-android.
Bug: 152255951
Bug: 154310674
Change-Id: Ie5ec02aebdf00b75756a6eda40db821488d30065
Minor refactoring of checkMergeZip(string) -> checkMergeZips(...string)
to allow testing of multiple merge zips.
Bug: 153443117
Test: m nothing
Change-Id: I8db00f611ced15f8476ba16f2834a72e8c913596
Previously, adding java_library to an sdk required that the names of
any APEXes that transitively compiled against it were added to its
apex_available property. This change removes that requirement.
Also corrects the dependency path in the TestApexAvailable_IndirectDep
error which previously passed through "shared from static" static
dependency tags even though those are explicitly NOT followed when
checking apex_available settings.
Bug: 152878661
Test: m droid
Change-Id: I995ed38956c1bc210b09494812de012fed9f9232
The DepIsInSameApex() and RequiredSdks() methods were defined in a few
places to avoid having to depend on the whole ApexModule/SdkAware
interfaces directly. However, that has a couple of issues:
1) It duplicates functionality making it difficult to change, changes
to the definitions outside the main interfaces do not cause compile
time failures, instead they result in a runtime change in behavior
which can be difficult to debug.
2) IDE navigation (specifically in Intellij) does not detect that the
duplicate definitions can resolve to the definitions in the main
interface.
This change extracts the methods into their own interfaces and reuses
those interfaces instead of duplicating the methods to fix both of
these issues.
Bug: 152878661
Test: m nothing
Change-Id: I0cfdf342a14eb0bfb82b1bd17e0633d81c7facfb
This partially reverts https://r.android.com/1278193. The propagated
symbol files contain all versions, even though only one is applicable
in a given SDK snapshot.
It's uncertain what repercussions this might have, but one is that if
we were to update a snapshot for a fixed version then it might change
because the symbol file contains new versions that aren't applicable.
Since the symbol file isn't actually needed at this point it's better
to wait with this step until the use cases for it are more clear.
Test: m nothing
Test: Create an SDK snapshot with Bionic libs, drop it into a
master-art tree without bionic/ in it, build ART APEXes, and check
that the Soong phase completes (specifically that the stubs are
detected even without symbol files).
Bug: 152481980
Change-Id: Ic79f89bc6d11d0b6552fa20791f5680ff9a40c0d
Necessary to make the APEX build logic treat the libraries as API
boundaries rather than dependencies to bundle.
The .so files in the snapshots are the compiled stub libraries in this
case. They are strictly speaking redundant since they can be generated
from the .map.txt files in the snapshots, but doing that would require
extending the cc_prebuilt_library(_shared) module types with a full
compiler pass etc, and that would break a lot of assumptions in the cc
package.
Test: m nothing
Test: Create an SDK snapshot with Bionic libs, drop it into a
master-art tree without bionic/ in it, build ART APEXes, and check
that the Soong phase completes (specifically no errors about various
APEX libs requiring libc that is not available to them).
Bug: 152481980
Change-Id: I31b928e6261198b6dd6f6b17196e714f07b64172
Necessary to get correct prebuilts for many Bionic libs.
Cleaned up numerious "system_shared_libs: []" from test fixtures, since
they otherwise would need correction in the expected results, and it is
better to have a single test focused on testing system_shared_libs
propagation.
Test: m nothing
Bug: 152255951
Change-Id: If2e8a5296223e6281d833312660e8e9e4cd184c0
Previously, the common value extraction code used an empty structure
to get the value to use to clear a field whose value is common. This
change removed the structure and used reflect.Zero(..) to get the
value instead.
Bug: 142935992
Test: m nothing
Change-Id: Ibd5103dacb86e7754a786356c0d15ffbde7f98bf
Previously, when an sdk snapshot only supported a single os type the
compile_multilib was set based on the multilib usages by the members
of that variant. After the change to support multiple os types per
snapshot the multilib setting was based on the multilib usages across
all the members of all sdk variants. That meant that if one os type
used only "64" and the other used "both" then they would both be
treated as "both" leading to missing variants when the snapshot was
unpacked.
This change tracks the multilib usages per os type and adds a property
for each one.
It intentionally changes a couple of tests:
1) Either by adding compile_multilib that is missing.
2) By targeting it at a specific os type instead of host.
The latter change is important to prevent the snapshot from being
used on a host platform (which will match the host target section)
but which is a different os type to the ones supported by the
snapshot.
Bug: 142935992
Test: m nothing
Change-Id: I883919b644292c3d019db223bb4fd5c11b39591f
Migrates system modules and droid stubs over to use the new API for
creating the snapshot modules and removes the old API.
Test: m nothing
Change-Id: Ia825767f1f7ee77f68cfe00f53e09e6f6bfa027f
Previously, java snapshot properties (java_library and java_test)
relied on the properties not being optimized when there was a single os
type and instead being added directly to the common os type properties.
However, that means that the behavior is inconsistent for other member
types depending on whether there was one os type or not.
This change updates the java sdk member handling to support
optimization. This involved:
1) Adding AidlIncludeDirs field to librarySdkMemberProperties to
specify the aidl include dirs instead of extracting that from the
library field.
2) Renaming jarToExport to JarToExport (in both
library/testSdkMemberProperties)to allow it to be optimized.
3) Adding MemberType() and Name() methods to SdkMemberPropertiesContext
to avoid having to store the former in the properties struct and
retrieve the latter from the library/test fields.
4) Removing the now unused library/test fields from the properties
structures.
5) Separating the processing of the jar/test config in
AddToPropertySet(...) as they may be optimized separately.
6) Ditto for the jar/aidl include dirs.
7) While doing this work I noticed that although the contents of the
aidl include dirs are copied into the snapshot the java_import does
not make use of them. Raised bug 151933053 and added TODO to track
that work.
Bug: 142935992
Test: m nothing
Change-Id: Iba9799e111ca5672b2133568163d8c49837ba9cd
Previously passing additional information to the implementations of
AddPrebuiltModule() or the SdkMemberProperties interface would have
required making changes to the API. This change added an
SdkMemberContext object into which additional information can easily
be added without requiring changes to existing implementations.
The BuildSnapshot() method was not modified because it is deprecated
and will be removed in a follow up change.
It also switches the API from passing variants as android.SdkAware to
android.Module. That is for a couple of reasons:
1) SdkAware is designed for managing the relationship between the
module and the SDK, not for generating the output snapshot. As such
there is nothing in SdkAware that is needed for generating the
output snapshot.
2) Accepting android.Module instead makes it easier to use the
underlying code for generating the snapshot module as well as the
individual member modules.
This is in preparation for a number of improvements and bug fixes in
both the snapshot creation code and implementations to address found
while trying to built the platform against ART prebuilts.
Bug: 151937654
Test: m nothing
Change-Id: Iac10f1200c0f283aa35402167eec8f9aeb65a38e
Added link type support in module creation code as cc_prebuilt_library
has to provide both shared and static libraries.
Had to add some calls to FirstUniquePaths() around the include dirs
settings in library_sdk_member as otherwise the shared variant contains
duplicate include dirs (presumably because it gets one set from its
dependency on the static variant and one set of its own). That
difference in include dirs causes problems in the generated
cc_prebuilt_library.
Test: m nothing
Bug: 142918168
Change-Id: Ie7f23fc2341c83c7814cc98e3970df4f5d4c8423
Test: m nothing
Test: Add
sdk {
name: "runtime-module-sdk",
native_shared_libs: [
"libc",
"libdl",
"libm",
"ld-android",
],
native_objects: [
"crtbegin_dynamic",
"crtbegin_static",
"crtend_android",
],
}
to bionic/apex/Android.bp. Then:
build/soong/scripts/build-aml-prebuilts.sh runtime-module-sdk
Take the generated runtime-module-sdk-current.zip and unzip into a
master-art tree without bionic/, edit the generated Android.bp to
extend cc_prebuilt_* modules with:
nocrt: true,
stl: "none",
system_shared_libs: [],
apex_available: ["//apex_available:anyapex"],
recovery_available: true,
vendor_available: true,
ramdisk_available: true,
Then "m com.android.art.debug". This passes Soong but fails in the
build step because more members are required.
Bug: 148934017
Change-Id: I2ab8f6aadb1440b325697cae4a8ed761c62d15d2
Extract the functionality to create an osTypeSpecificInfo struct,
to optimize the properties, and add its properties to a property set
into methods of the *osTypeSpecificInfo struct.
This change is in preparation for adding support for link type which
is another dimension within arch type which itself sits within os type.
Test: m nothing
Bug: 142918168
Change-Id: I025ee90e1461f7389bf4a9d056b281453068cf87
Extract the functionality to create an archTypeSpecificInfo struct and
to add its properties to a property set into methods of the
*archTypeSpecificInfo struct.
Test: m nothing
Bug: 142918168
Change-Id: I2a9e0327b61bce7ad7699cd75de17aa0e5f1ebbb
This change reorganizes the code within the main module creation loop
in preparation for delegating the work to separate types. It has been
split out into its own change to make it easier to review by keeping
the functional changes localized.
Renames the osTypeSpecificInfo archTypes field to archInfos as the
latter is more accurate.
Cleans up the arch variants handling:
1) Groups them by arch type to make them easier to process.
2) Fails fast when there is not exactly one variant per arch type as
otherwise it results in a confusing failure later on.
3) Removes the commonArch flags and instead uses the fact that
osInfo.archInfos is empty when the common architecture variant is
available.
Cleans up the arch type specific property set handling.
1) Adds new archPropertySet variable to allow the choice of where the
arch specific properties are added to be made alongside the choice
of where the os specific properties are to be added.
2) Removes unnecessary check for commonArch around the loop to add
properties from archInfos as the archInfos will be empty when the
common architecture is present.
A number of other changes to make it easier to extract the code into
their own methods.
Test: m nothing
Bug: 142918168
Change-Id: I16a5fa79efff0d08c22916449eb46a5bd910723a
This ensures a consistent output irrespective of whether property sets
are created before or after the properties are added. This provides a
little more flexibility in the creation code which allows that to be
simplfied.
Also switches from using reflection to a type switch.
Bug: 142918168
Test: m nothing
Change-Id: Ia025bfc751f1217d1658de6fb8e15091ea0ea9ff
This was only being used to set the "stl" property for cc library sdk
member type and so that functionality was moved to AddPrebuiltModule()
and FinalizeModule was removed.
Required a few test changes to move the property to the correct
position in the generated module.
Bug: 142918168
Test: m nothing
Change-Id: If6400189833d4ff3285e7a7adf63a9b509e2a03b
This change ensures that the runtime dependencies between a
binary/shared library are correctly specified in the snapshot so that
the build can ensure that shared libraries are built before the targets
that use them.
It adds support for differentiating between references that are
required to refer to another sdk member (required) and those that may
refer to either an sdk member or a non-sdk member (optional). The
latter is used for shared library references as the libraries used by
an sdk member may be provided from outside the sdk. e.g. liblog is not
part of the ART module but is used by some members of the ART sdk.
Bug: 142935992
Test: m nothing
Change-Id: Ia8509ffe79b208c23beba1880fe9c8a92b732685
This change also added support for excluding properties from common
value extraction by using a struct tag of `sdk:"keep"` That was needed
to prevent the fields in SdkMemberPropertiesBase from having their
values cleared.
The purpose of this change is to make it easier to share functionality
across sdk member types.
Bug: 142935992
Test: m nothing
Change-Id: Ie5160a8f854056920e411801ca20721eab7c8578
This change pulled the common value extraction functionality out into
its own structure that can be used to extract common values from
multiple sets of properties.
Precursor to follow up changes that will allow properties to contain
embedded structures making it easier to share functionality across
sdk member types.
Bug: 142935992
Test: m nothing
Change-Id: Ic902ed61e40ced7a2c2fa4b5f793f532c6fc0034
Makes sure that the module snapshots do not rely on the white list
of apex available settings so that when those lists are removed it is
not necessary to update any snapshots.
Bug: 142935992
Test: m nothing
Change-Id: Iedcff7dfc2646a4da77258d16e06657dd2f411f9
The sdk snapshot creates two prebuilts for each member one that is
versioned and one that is not. If they are both installed then they
lead to duplicate rules in make for creating the same installed file.
This change adds an installable property to cc modules that will
prevent the installation of the file and then adds installable: false
on the versioned prebuilt for cc modules.
Bug: 142935992
Test: m nothing
Change-Id: I4cb294c2b0c8a3f411eea569775835d9e41726d6
Checking apex_available was missing some corner cases.
For example, the deps of share deps of cc_library modules are missed
while those from cc_library_shared are correctly tracked.
This was due to..
* calling DepIsInSameApex in WalkDeps: both work fine separately, but
when they are used together, it fails to work. It's due to how WalkDeps
works. (We might fix this bug too risky since it is used very widely)
* incorrect receiver for DepIsInSameApex in apex_deps mutator: receiver
is supposed to be parent, but child was used before. Interestingly lots
of deps are within the same group of module types(cc to cc, java to
java), it has worked. (note that receiver's DepIsInSameApex
implementation can be different).
This change fixes them by..
* walkPayloadDeps is now relying on ApexVariation, which is calculated
correctly by TopDown apex_deps mutator.
* use correct receiver for DepIsInSameApex in apex_deps mutator, which
requires for java.SdkLibrary to override the method and for
java.Library/Import to use passed dep instead of receiver to check its
membership of sdk.
Bug: 151071238
Test: build/boot
Change-Id: I0569ef4bb8e79635e4d97a89f421a8d8b7d26456
Adds support for specifying separate members to an sdk/module_exports
for different os types, e.g. separate ones for android and host.
Adds 'android:"arch_variant"' tag to the dynamically generated fields
for the sdk member types.
Merges the exported members from all variants together.
Determines the device/host_supported flags of the member snapshots by
whether the OsClasses used by their variants rather than the sdk's
host/device supported properties as they may be different.
Bug: 150451422
Test: m nothing
Change-Id: I41fbbcd8723aafd54826aad9b78eced9f66ef51c
Updates the member snapshot creation code to support multiple os types.
It basically sorts the variants by os type, then applies the code to
optimize the arch properties and then it optimizes the properties that
are common across architectures and extracts any properties that are
common across os types.
The java and cc member types needed to be modified to make the location
of the generated files within the snapshot os type dependent when there
is more than one os type. That was done by adding an OsPrefix() method
to the SdkMemberPropertiesBase which returns the os prefix to use when
there is > 1 os type and otherwise returns an empty string.
Added three tests, one for cc shared libraries, one for cc binary and
one for java header libraries.
Bug: 150451422
Test: m nothing
Change-Id: I08f5fbdd7852b06c9a9a2f1cfdc364338a3d5bac
Previously, while sdk tests would pass the sdk code would often fail
in androidmk processing. This change makes the tests more realistic
and will catch the errors earlier.
Bug: 142935992
Test: m nothing
Change-Id: Ifd0b2d7cf24e941c919f6b6e0beb2403a67d4308
Adds support for handling the common arch to the sdk module type
snapshot generation code and then refactors the java_library
and java_test snapshot processing to take advantage of that.
Bug: 150451422
Test: m nothing
Change-Id: If746f375f1c4288ab6b67c7d216593b70258b043
Generalize the processing of arch specific properties to reduce
duplication in snapshot module creation and simplify addition of
support for handling multiple os types.
Supporting multiple os types with the current method for building
snapshot modules would require every affected module type to add
support for it. Rather than duplicate multiple os type handling code
across those module types this work generalizes the process cc modules
use for handling arch types as it can be used as a basis for handling
multiple os types. Migrating module types over to this new process
will insulate them from having to handle multiple os types.
OB
SdkMemberType changes:
* BuildSnapshot is deprecated in favour of the new AddPrebuiltModule()
method.
* Additional methods, CreateVariantPropertiesStruct() and
FinalizeModule() are added.
* A new interface SdkMemberProperties, is defined that handles
extracting information from the variant (prior to common value
optimization) and adding properties to a property set.
The sdk module type uses these new methods and types to delegate the
member type specific processing to the relevant member types while
handling the behavior that is common across all members types, e.g.
extracting common values across multiple architectures. A future change
will leverage this processing to add support for multiple os types.
This change also refactors the cc module processing to use the new
process.
Bug: 150451422
Test: m nothing
Change-Id: If6ab2498407b17f50391d062cd9afc01b5e01af4
Following how NDK also creates its own install path.
Bug: 142935992
Test: cc_sdk_test.go, java_sdk_test.go
Change-Id: I98a3656903f37f6d7c90e6cf609431b2461a6161
Adds a CommonOS variant for sdk that depends on the os specific
variants and is used to generate a single sdk for multiple OsTypes,
e.g. host linux and android.
At the minute the member types only support a single OsType but the
basic mechanism for managing the CommonOS variant and collating the
variants across all of them is there.
The only visible effect of this change is that the location of the
generated snapshot is changed, it is no longer os specific and instead
is in the same location irrespective of which os it is built for.
A lot of tests needed to be changed to specify "common_os" as the
variant type instead of the specific os type. As that is the same across
all tests it is hard coded in CheckSnapshot method.
Test: m nothing
Bug: 150451422
Change-Id: If36be39b06d6910453649f7c288c2d34f688b2f4
Simplifies debugging sdk membership code when it is only called if an
appropriate member is present.
Test: m nothing
Change-Id: I32bc93e0484c829bdbd6b050822249feb6404f89
Makes the tests more realistic as they cover what happens when an sdk
OS specific variant is disabled (Windows is disabled by default). This
will allow issues around disabled variants to be detected early during
Soong bootstrapping.
Test: m nothing
Change-Id: I7ec47dbca52e38750166d755daf706aadea12d15
Ensures that the generated prebuilt modules do not contain empty
property sets.
Bug: 148933848
Test: m nothing
Change-Id: I99d5da1c4e3a72836e81e021ef043df53888e66b
This change adds support for transforming a property set after its
contents. This allows a transform to recursively prune empty property
sets that were created for a module.
The transformPropertySet method was renamed to
transformPropertySetBeforeContents and a new
transformPropertySetAfterContents method was added.
Bug: 148933848
Test: m nothing
Change-Id: Ia198d47e042b98c69406db4bc12859869816a387
Returning nil from transformPropertySet in order to remove it did
not work because it ends up comparing as (*bpPropertySet, nil) and
not an untyped nil which causes the test against nil to fail.
This change adds tests to check that returning nil will delete a
property/property set from the containing property set and fixes the
code so that it passes the tests. It extracts common code to transform
a property set and its contents as well as code for creating new
property sets.
Bug: 148933848
Test: m nothing
Change-Id: I35dc3c39c76e701821891622615c09b094cf697f
This is needed to allow the art-host-module-exports to restrict itself
to just managing the linux 64 bit version of the host tools as that is
the only variant that is currently supported by all host tools. This
greatly simplifies that process and allows us to make progress on the
unbundling.
Bug: 142935992
Test: m nothing
Change-Id: I62d016d97c2df73e5feecf912638f477fedd97c9
If an sdk variant is disabled (e.g. say windows) then it should not
add dependencies on corresponding sdk member variants as if those
variants have not been created then the build breaks unnecessarily.
This is needed to make progress on unbundling art which provides at
least one host tool (dex2oat) that is not supported on all platforms
and some (hiddenapi) that only produce 64 bit versions.
Initially, the art prebuilts will only support building 64 bit
versions of linux host tools.
Bug: 142935992
Test: m art-module-sdk art-module-host-exports art-module-test-exports
Change-Id: Idfbb40bb3cabc6eb731a2b38f045ed14f0a713cc
While it is invalid to mix //visibility:public with other rules in the
visibility property in a .bp file tt was possible, by overriding
defaults, to have //visibility:public mixed in with other rules in the
effective visibility rules. That caused problems when those effective
rules were used in an sdk snapshot.
This change replaces any set of rules that include //visibility:public
with just the //visibility:public rule. That simplifies those rules,
making them cheaper to process and ensures that the effective rules are
valid in the visibility property.
Adding test support required some refactoring of the
effectiveVisibilityRules(BaseModuleContext, ...) and underlying methods
to take a Config instead of BaseModuleContext as the tests do not have
access to BaseModuleContext.
Bug: 142935992
Test: m nothing - new tests failed without change, work with it
Add dex2oat to art-module-host-exports, build it and check the
generated Android.bp file in the snapshot to ensure the
visibility property for the dex2oat prebuilt does not mix
//visibility:public with other rules.
Change-Id: I08e7f0dcb40838d426fe88fedf69eae27b77473c
Move the NDK modules defined in app_test.go into cc/testing.go in
preparation for using sdk_version: "current" in more tests.
Test: all soong tests
Change-Id: I80bc31225fb4562fce42781219cefbbd6affae06
Internal sdk members are used by an sdk member but not exported by the
sdk. The exported sdk members are those listed explicitly in one of the
sdk member list properties, e.g. java_header_libs.
The prebuilts of an internal sdk member use a unique name so that they
do not clash with the source module. The use of the module internally
is an implementation detail that must not have any effect outside the
snapshot. Having the same name as the source module could cause it to
override the source module, hence why it needs a unique name.
Similarly, they are marked as private so as to prevent their accidental
use from outside the snapshot.
Bug: 142940300
Test: m nothing
Change-Id: Id5364b410be0592f65666afb3e40e9d3f020251c
Adds an SdkMemberType implementation for java_system_modules. It
specifies that java_system_modules can be used with sdk as well as
module_exports, and also that the libs property should be included
as transitive members in the sdk.
It also adds support for treating appropriate tagged properties in
the snapshot prebuilts module as references to sdk members so that
they are correctly transformed when creating the versioned modules.
Bug: 142940300
Test: m nothing
Change-Id: Ic10b5a6d5b92b6018334fe876f06feaf79cc55e9
Paul Duffin