Previously, it was assumed that generated headers must be arch specific
and so prevented the fields referencing the paths to those headers from
being automatically optimized by the sdk generation code. That is not
always the case, e.g. with headers generated from protos so this change
allows those fields to be optimized.
Bug: 180427921
Test: m nothing
Change-Id: Id2af419d58ae3c30ea6d9e87f71e33a9ff6ba13b
Previously, the snapshot handling code did not preserve the directory
structure of generated include directories and instead just copied the
headers into the same module specific directory and added that single
directory to the export_include_dirs (or similar) property.
That had a couple of issues:
* The include directory was repeated in the ..._include_dirs property.
* It did not work when the include directories overlapped.
In the latter case it had a couple of issues:
* Code which compiled fine against the source would not compile against
the prebuilt.
* Header files were duplicated in the output.
e.g. assume the following generated header file structure:
foo/
foo.h
bar/
bar.h
baz/
baz.h
When the sdk snapshot was passed include directories of "foo", "bar" and
headers of "foo/foo.h", "bar/bar.h", "bar/baz/baz.h" it would generate a
snapshot with the structure:
include_gen/
foo.h
bar.h
baz/
baz.h
And:
export_include_dirs: ["include_gen", "include_gen"]
However, when the include directories overlapped and include directories
of "foo", "bar" and "bar/baz" were passed in the directory structure
would be the same and the export_include_dirs would contain 3 usages of
"include_gen".
That meant that source code which used the following would build
against the source (because it would find "baz.h" in the "bar/baz"
include directory) but would fail when built against the prebuilts
because the "include_gen" directory did not contain "baz.h":
#include "baz.h"
This change preserves the input directory structure for generated files
in a similar way to how it does it for source files. So, the snapshot
structure looks something like this:
include_gen/
foo/
foo.h
bar/
bar.h
baz/
baz.h
And:
export_include_dirs: [
"include_gen/foo",
"include_gen/bar",
"include_gen/bar/baz",
],
Bug: 180427921
Test: m nothing
Change-Id: Id69eef8cf5eecd033841d3b7cd0c044a697ce404
'vendor_available: true' creates a vendor variant from a system
module. The vendor variant of the module is installed to /vendor.
However, we may want to install the vendor variant to /odm, instead.
'device_specific: true' does not work for this purpose because
'vendor_available: true' is allowed only for the system or product
modules to create a vendor variant. But 'device_specific: true'
itself creates a vendor variant that may not work with
'vendor_available: true'.
To install the vendor variant to /odm, we define a new property
'odm_available'. 'odm_available' is exactly the same as the
'vendor_available' except the install path of the vendor variant.
By defining 'odm_available: true', the vendor variant of the module
will be installed to /odm or /vendor/odm instead of /vendor.
Bug: 176147321
Bug: 176079978
Test: check if a module with 'odm_available: true' is installed to
/vendor/odm
Change-Id: I2d16bd2c515796597b2fbd1eb66f7c2736434697
To make a module available to product variants, it must define
`product_available: true`. `vendor_available: true` will not create
product variants any more.
However, in this CL, we don't change the behavior of
`vendor_available` property. It still creates both variants. After we
update all Android.bp files that need to provide product variants
with `product_available: true`, we may upload the remaining patches.
Bug: 150902910
Test: lunch aosp_arm64-userdebug && m
Change-Id: I0fd5be7bbae2c45d5cab3c3c2ca49f53a9b6f975
The stubs methods are very specific to cc for now, move them out
of LinkableInterface so they are not shared with rust. Instead,
create a cc.Module.library field that contains the libraryInterface
to simplify calling libraryInterface methods on cc modules.
Test: all Soong tests
Test: no change to Soong outputs
Change-Id: I0289d866ce1f7a765631fe3101a62b1b4988ba1c
Export information about static libraries, shared libraries and
exported flags through Providers instead of accessing the module
directly. Much more is left to be converted, but this significantly
simplifies the dependencies on libraries with stubs by making it easy
for a module to masquerade as another by simply exporting the
providers from the other module. Instead of depending on all the
versions of a library and then picking which one to use later, it
can depend only on the implementation variant and then select the
right SharedLibraryInfo from the variant.
Test: m checkbuild
Test: only expected changes to build.ninja
Change-Id: I1fd9eb4d251cf96ed8398d586efc3e0817663c76
Don't create empty version variants for binaries, objects, rust
rlibs or rust dylibs.
Test: no change to build.ninja
Change-Id: I62d4d43da476eafdb258a08b5ada758bb2971a1a
This is a quick fix to avoid build failures due to prebuilts lacking
stubs versions that the sources have, which causes dependencies on
older version stubs fail on master-art-host where the sources of e.g.
Bionic are no longer available.
More work is required to make the version handling of stubs DTRT - a
TODO comment outlines it.
Test: m nothing
Test: `m runtime-module-sdk`, check that the generated Android.bp
has the full stubs.versions list like the sources.
Bug: 169373910
Change-Id: I0ebfaf94f9d45a0e58d9785c40e7cea626f0ab83
liblog snapshot needs to sanitizer.address=false to avoid cycle in asan
builds. Adding that separately in library_sdk_member.go would start to
feel like whack-a-mole, so the snapshot generation is instead extended
to handle nested property structs.
This uses the BpPropertySet.AddProperty extension in
https://r.android.com/1423510, and common value optimisation now
recurses into non-anonymous structs, instead of comparing them as a
whole.
Test: m nothing
Test: `m SANITIZE_TARGET=address nothing` with prebuilts/runtime
present in the manifest and a fresh snapshot made with this
Bug: 151303681
Change-Id: I472554117a488e6c800045cb2ed59377778571a4
A per-context variable is used to store the list of modules that
contain stubs and their available versions. Stores the list of the
stubs versions on the implementation module, and then use the new
return values from AddVariationDependencies to expand dependencies
on implementation libraries to also depend on the stubs libraries.
Adds a new mutator pass to propagate list of stub versions to llndk
libraries.
Also creates an alias version variation called "latest" to allow
depending on the latest version without having to know what it is.
Test: all Soong tests
Test: no change to build.ninja, Android-${TARGET_PRODUCT}.mk, make_vars-${TARGET_PRODUCT}.mk or late-${TARGET_PRODUCT}.mk
Change-Id: If19659e2e5828c860fd4d679ef79a414b7ea2efc
Needed for jni_headers which is marked as recovery_available because libchrome
is also marked and that library is used by others that are the recovery
partition. There may be other dependency paths that require jni_headers to be
recovery_available that was just the first one found.
Bug: 168301990
Test: m nothing
Change-Id: I916ecfe7700b9c153b29115e849da148fc7e7a76
AddFarVariationDependencies was broken, which allowed sdk modules to
request dependencies using image and version variations, even for
host modules that do not have image or version variations. Make
the image and version variations conditional on device sdk modules.
Test: go test ./sdk
Change-Id: I59b7a32a3782254fd5feb828a5258ee13d4db812
This is necessary to not get dependencies on libclang_rt sanitizer libs
for the CRT objects which lead to cyclic dependencies in sanitizer
builds.
Test: `lunch aosp_blueline_hwasan-userdebug && m nothing`
in a tree that has the prebuilts created and unzipped from
`build/soong/scripts/build-aml-prebuilts.sh runtime-module-{sdk,host-exports}`
Bug: 151303681
Change-Id: I3f848a084280bdc3ade4b74df03e981d8cc61222
Necessary to avoid problems with implicitly enabled host OS'es, e.g.
linux_glibc getting enabled when we only supply a linux_bionic linker
in runtime-module-host-exports. That will then cause a non-functional
prebuilt to take precedence over source if the prebuilt is preferred.
We don't do this for device since we so far only support a single
device OS (android).
This introduces the notion that SDK member types can be host OS
dependent or not. That way java members with host prebuilts don't get
restricted to a specific host OS.
Test: m nothing
Test: build/soong/scripts/build-aml-prebuilts.sh runtime-module-host-exports
Check that the generated Android.bp correctly disables the bionic
linker prebuilt for linux_glibc.
Test: art/build/apex/runtests.sh
on master-art with an updated runtime SDK snapshot
Test: art/tools/buildbot-build.sh {--host,--target}
on master-art with an updated runtime SDK snapshot
Bug: 160349757
Change-Id: Idad7ef138cdbcbd209d390bf6c10ca8365d4619f
It is not clear whether this is causing the bug but it is a potential
source of problems and if it does not fix the bug then at least it will
have eliminated this as the culprit.
Bug: 157656545
Test: m nothing
Change-Id: I3abcb9ae501977924d0d514f4d9007ead24fe0a5
Test: m nothing
Test: Check on go/Android.bp that cc_prebuilt_library supports
unique_host_soname
Bug: 158743135
Change-Id: Ie02a5cae057fb9092c226d8c5b7f63e1f66ad066
Change since last attempt: Disable test that breaks on darwin.
Adds a test that fails with unknown property android.stubs.versions
and then fixes that by marking the field from which that property is
created with 'ignored-on-host' and implemented the isHostVariant on
*osTypeSpecificInfo.
Bug: 155628860
Test: m nothing
Change-Id: Ia4e744c9e799d5adaf8a2f761516f568ec363ad4
Previously, the snapshot code did not know whether a specific property
could be arch specific or not and assumed that they all were which
meant that it could generate snapshots containing arch specific values
for properties that are not arch specific and so would fail when
unpacked.
This change requires arch specific fields in SdkMemberProperties to be
tagged as such using `android:"arch_variant"` (just as in module input
property structures). Any property without that must have properties
that are common across all variants.
Bug: 155628860
Test: m nothing
Change-Id: I3df60f0b53ba02ec2c55a80c7da058eac5909d26
Revert submission 1302576
Bug: 156054601
Reason for revert: Presumed root cause of build break.
Reverted Changes:
Ifc8116e11:Detect invalid arch specific properties in snapsho...
I7ebd33307:Adds support for 'ignored-on-host'
I167b47a13:Fix snapshot of a host/device cc_library with stub...
Change-Id: Id7eba0bdde5c579e10e9b42d94a7cfab5f34995f
Revert submission 1302576
Bug: 156054601
Reason for revert: Presumed root cause of build break.
Reverted Changes:
Ifc8116e11:Detect invalid arch specific properties in snapsho...
I7ebd33307:Adds support for 'ignored-on-host'
I167b47a13:Fix snapshot of a host/device cc_library with stub...
Change-Id: Ibccce5286605bb71c6be3b3550ba86d8b7e24fa7
Adds a test that fails with unknown property android.stubs.versions
and then fixes that by marking the field from which that property is
created with 'ignored-on-host' and implemented the isHostVariant on
*osTypeSpecificInfo.
Bug: 155628860
Test: m nothing
Change-Id: I167b47a1374f541aa09d7e045972d740f1d9009c
Previously, the snapshot code did not know whether a specific property
could be arch specific or not and assumed that they all were which
meant that it could generate snapshots containing arch specific values
for properties that are not arch specific and so would fail when
unpacked.
This change requires arch specific fields in SdkMemberProperties to be
tagged as such using `android:"arch_variant"` (just as in module input
property structures). Any property without that must have properties
that are common across all variants.
Bug: 155628860
Test: m nothing
Change-Id: Ifc8116e11d987cfe7aec2eeaa964f3bbf36b5dc2
Test: m nothing
Test: Build an SDK snapshot including libc, check that the generated
cc_prebuilt_library_shared for it is a stub that doesn't depend on
ld-android.
Bug: 152255951
Bug: 154310674
Change-Id: Ie5ec02aebdf00b75756a6eda40db821488d30065
This partially reverts https://r.android.com/1278193. The propagated
symbol files contain all versions, even though only one is applicable
in a given SDK snapshot.
It's uncertain what repercussions this might have, but one is that if
we were to update a snapshot for a fixed version then it might change
because the symbol file contains new versions that aren't applicable.
Since the symbol file isn't actually needed at this point it's better
to wait with this step until the use cases for it are more clear.
Test: m nothing
Test: Create an SDK snapshot with Bionic libs, drop it into a
master-art tree without bionic/ in it, build ART APEXes, and check
that the Soong phase completes (specifically that the stubs are
detected even without symbol files).
Bug: 152481980
Change-Id: Ic79f89bc6d11d0b6552fa20791f5680ff9a40c0d
Necessary to make the APEX build logic treat the libraries as API
boundaries rather than dependencies to bundle.
The .so files in the snapshots are the compiled stub libraries in this
case. They are strictly speaking redundant since they can be generated
from the .map.txt files in the snapshots, but doing that would require
extending the cc_prebuilt_library(_shared) module types with a full
compiler pass etc, and that would break a lot of assumptions in the cc
package.
Test: m nothing
Test: Create an SDK snapshot with Bionic libs, drop it into a
master-art tree without bionic/ in it, build ART APEXes, and check
that the Soong phase completes (specifically no errors about various
APEX libs requiring libc that is not available to them).
Bug: 152481980
Change-Id: I31b928e6261198b6dd6f6b17196e714f07b64172
Necessary to get correct prebuilts for many Bionic libs.
Cleaned up numerious "system_shared_libs: []" from test fixtures, since
they otherwise would need correction in the expected results, and it is
better to have a single test focused on testing system_shared_libs
propagation.
Test: m nothing
Bug: 152255951
Change-Id: If2e8a5296223e6281d833312660e8e9e4cd184c0
Previously passing additional information to the implementations of
AddPrebuiltModule() or the SdkMemberProperties interface would have
required making changes to the API. This change added an
SdkMemberContext object into which additional information can easily
be added without requiring changes to existing implementations.
The BuildSnapshot() method was not modified because it is deprecated
and will be removed in a follow up change.
It also switches the API from passing variants as android.SdkAware to
android.Module. That is for a couple of reasons:
1) SdkAware is designed for managing the relationship between the
module and the SDK, not for generating the output snapshot. As such
there is nothing in SdkAware that is needed for generating the
output snapshot.
2) Accepting android.Module instead makes it easier to use the
underlying code for generating the snapshot module as well as the
individual member modules.
This is in preparation for a number of improvements and bug fixes in
both the snapshot creation code and implementations to address found
while trying to built the platform against ART prebuilts.
Bug: 151937654
Test: m nothing
Change-Id: Iac10f1200c0f283aa35402167eec8f9aeb65a38e
Added link type support in module creation code as cc_prebuilt_library
has to provide both shared and static libraries.
Had to add some calls to FirstUniquePaths() around the include dirs
settings in library_sdk_member as otherwise the shared variant contains
duplicate include dirs (presumably because it gets one set from its
dependency on the static variant and one set of its own). That
difference in include dirs causes problems in the generated
cc_prebuilt_library.
Test: m nothing
Bug: 142918168
Change-Id: Ie7f23fc2341c83c7814cc98e3970df4f5d4c8423
Test: m nothing
Test: Add
sdk {
name: "runtime-module-sdk",
native_shared_libs: [
"libc",
"libdl",
"libm",
"ld-android",
],
native_objects: [
"crtbegin_dynamic",
"crtbegin_static",
"crtend_android",
],
}
to bionic/apex/Android.bp. Then:
build/soong/scripts/build-aml-prebuilts.sh runtime-module-sdk
Take the generated runtime-module-sdk-current.zip and unzip into a
master-art tree without bionic/, edit the generated Android.bp to
extend cc_prebuilt_* modules with:
nocrt: true,
stl: "none",
system_shared_libs: [],
apex_available: ["//apex_available:anyapex"],
recovery_available: true,
vendor_available: true,
ramdisk_available: true,
Then "m com.android.art.debug". This passes Soong but fails in the
build step because more members are required.
Bug: 148934017
Change-Id: I2ab8f6aadb1440b325697cae4a8ed761c62d15d2
This was only being used to set the "stl" property for cc library sdk
member type and so that functionality was moved to AddPrebuiltModule()
and FinalizeModule was removed.
Required a few test changes to move the property to the correct
position in the generated module.
Bug: 142918168
Test: m nothing
Change-Id: If6400189833d4ff3285e7a7adf63a9b509e2a03b
This change ensures that the runtime dependencies between a
binary/shared library are correctly specified in the snapshot so that
the build can ensure that shared libraries are built before the targets
that use them.
It adds support for differentiating between references that are
required to refer to another sdk member (required) and those that may
refer to either an sdk member or a non-sdk member (optional). The
latter is used for shared library references as the libraries used by
an sdk member may be provided from outside the sdk. e.g. liblog is not
part of the ART module but is used by some members of the ART sdk.
Bug: 142935992
Test: m nothing
Change-Id: Ia8509ffe79b208c23beba1880fe9c8a92b732685
Updates the member snapshot creation code to support multiple os types.
It basically sorts the variants by os type, then applies the code to
optimize the arch properties and then it optimizes the properties that
are common across architectures and extracts any properties that are
common across os types.
The java and cc member types needed to be modified to make the location
of the generated files within the snapshot os type dependent when there
is more than one os type. That was done by adding an OsPrefix() method
to the SdkMemberPropertiesBase which returns the os prefix to use when
there is > 1 os type and otherwise returns an empty string.
Added three tests, one for cc shared libraries, one for cc binary and
one for java header libraries.
Bug: 150451422
Test: m nothing
Change-Id: I08f5fbdd7852b06c9a9a2f1cfdc364338a3d5bac
Generalize the processing of arch specific properties to reduce
duplication in snapshot module creation and simplify addition of
support for handling multiple os types.
Supporting multiple os types with the current method for building
snapshot modules would require every affected module type to add
support for it. Rather than duplicate multiple os type handling code
across those module types this work generalizes the process cc modules
use for handling arch types as it can be used as a basis for handling
multiple os types. Migrating module types over to this new process
will insulate them from having to handle multiple os types.
OB
SdkMemberType changes:
* BuildSnapshot is deprecated in favour of the new AddPrebuiltModule()
method.
* Additional methods, CreateVariantPropertiesStruct() and
FinalizeModule() are added.
* A new interface SdkMemberProperties, is defined that handles
extracting information from the variant (prior to common value
optimization) and adding properties to a property set.
The sdk module type uses these new methods and types to delegate the
member type specific processing to the relevant member types while
handling the behavior that is common across all members types, e.g.
extracting common values across multiple architectures. A future change
will leverage this processing to add support for multiple os types.
This change also refactors the cc module processing to use the new
process.
Bug: 150451422
Test: m nothing
Change-Id: If6ab2498407b17f50391d062cd9afc01b5e01af4
Previously, the code for generating a snapshot of a cc library was
split into two separate phases. The first phase copied the files that
needed copying and the second phase added the properties for the
include dirs. This separation made it difficult to make sure that the
two phases were in sync.
This change merges those two phases together so the same paths used to
copy the files are used in the properties ensuring consistency. As the
various type of include dir and header were treated slightly different
to each other this parameterizes that behavior.
Bug: 142935992
Test: m nothing
Change-Id: I7877464987bbdae9662e5e3f02bb5e5a75dca5a3
The list of exported generated headers are all expected to be within
one of the exported generated include dirs. Previously, that was not
the case as ExportedGeneratedIncludeDirs was suitable for extracting
to common properties (which changes the output location) and
exportedGeneratedHeaders was not.
This would cause a problem if there was only one variant. In that case
the ExportedGeneratedIncludeDirs would be treated as a common property
and placed in include_gen/<x> directory while exportedGeneratedHeaders
would be treated as an arch specific property and placed in
<arch>/include_gen/<x>.
Bug: 142935992
Test: m nothing
Change-Id: Idf82a5ca551b44ec31971c7ff3bd957a4c38f396
Some additional, possibly arch specific, properties need adding to
cc library (e.g. recovery_available). This dedups and cleans up the
code involved to simply the addition of those new properties.
Bug: 142918168
Test: m nothing
Change-Id: I0963aa02e9504af1ae9b427ff1016e7c481465f4
By default SdkMemberTypes are only supported on module_exports module
type. Support for sdk module type has to be explicitly specified.
The java_header_libs, native_shared_libs and stubs_sources are
supported on sdk. The latter is required to provide the stubs source
for an API specified in java_header_libs as they should be kept in
sync.
Bug: 146341462
Test: m nothing
Change-Id: I19b9e60792780a797458d4a9e489506602b13144
The cc library and cc binary (and other cc module types) are all
instances of cc.Module. So, to differentiate between them and make
sure that only appropriate library instances can be added to
native_shared/static_libs this adds a special sdkMemberTypes field to
Module which if set specifies the SdkMemberTypes the module supports.
If it is not set then the module type cannot be used in the sdk at all.
Corrects an issue with one of the tests where a prebuilt cc
library was added to the sdk instead of a source cc library.
Adds a new test to ensure that cc_library_(shared|static)_host module
types work with the sdk as well and another test to ensure that
cc_library can be used as either.
Bug: 142918168
Test: m checkbuild
Change-Id: I359cdbdd15328ca571f276d2b6ce9a229ebb2c86
Previously, adding a new SdkMemberType would require adding a new
sdkMemberListProperty instance to the sdkMemberListProperties as well
as adding a new property into the sdkProperties struct. They are
potential sources of conflict and couple the sdk code with all the
packages that add members to it. This change switched to a
registration model that allows each package to register its sdk
member types decoupling them from the sdk code.
Adds an SdkPropertyName() method to SdkMemberType that specifies the
name of the property to use in the sdk/sdk_snapshot. Also provides
an SdkMemberTypeBase struct to be used by providers of SdkMemberType
implementations.
SdkMemberType instances are registered using the
RegisterSdkMemberType() func which sorts the registered instances
by their SdkPropertyName() to ensure the behavior is consistent and
not affected by order of registration.
When creating a new sdk module a dynamicSdkMemberTypes instance is
created that contains the following:
* A properties struct is created dynamically that contains a field for
each registered SdkMemberType, corresponding to that type's
SdkPropertyName().
* A list of sdkMemberListProperty instances is also created, one for
each registered SdkMemberType.
The dynamicSdkMemberTypes instance is cached using a key that uniquely
identifies the set of registered types just in case new types are
registered after one has been created, e.g. by tests.
Bug: 142918168
Test: m checkbuild
Change-Id: I4bf2bf56a2a49025aa41454048bc1e8ccc6baca2
Paul Duffin