This change ensures that the runtime dependencies between a
binary/shared library are correctly specified in the snapshot so that
the build can ensure that shared libraries are built before the targets
that use them.
It adds support for differentiating between references that are
required to refer to another sdk member (required) and those that may
refer to either an sdk member or a non-sdk member (optional). The
latter is used for shared library references as the libraries used by
an sdk member may be provided from outside the sdk. e.g. liblog is not
part of the ART module but is used by some members of the ART sdk.
Bug: 142935992
Test: m nothing
Change-Id: Ia8509ffe79b208c23beba1880fe9c8a92b732685
This change also added support for excluding properties from common
value extraction by using a struct tag of `sdk:"keep"` That was needed
to prevent the fields in SdkMemberPropertiesBase from having their
values cleared.
The purpose of this change is to make it easier to share functionality
across sdk member types.
Bug: 142935992
Test: m nothing
Change-Id: Ie5160a8f854056920e411801ca20721eab7c8578
This change pulled the common value extraction functionality out into
its own structure that can be used to extract common values from
multiple sets of properties.
Precursor to follow up changes that will allow properties to contain
embedded structures making it easier to share functionality across
sdk member types.
Bug: 142935992
Test: m nothing
Change-Id: Ic902ed61e40ced7a2c2fa4b5f793f532c6fc0034
Makes sure that the module snapshots do not rely on the white list
of apex available settings so that when those lists are removed it is
not necessary to update any snapshots.
Bug: 142935992
Test: m nothing
Change-Id: Iedcff7dfc2646a4da77258d16e06657dd2f411f9
The sdk snapshot creates two prebuilts for each member one that is
versioned and one that is not. If they are both installed then they
lead to duplicate rules in make for creating the same installed file.
This change adds an installable property to cc modules that will
prevent the installation of the file and then adds installable: false
on the versioned prebuilt for cc modules.
Bug: 142935992
Test: m nothing
Change-Id: I4cb294c2b0c8a3f411eea569775835d9e41726d6
Checking apex_available was missing some corner cases.
For example, the deps of share deps of cc_library modules are missed
while those from cc_library_shared are correctly tracked.
This was due to..
* calling DepIsInSameApex in WalkDeps: both work fine separately, but
when they are used together, it fails to work. It's due to how WalkDeps
works. (We might fix this bug too risky since it is used very widely)
* incorrect receiver for DepIsInSameApex in apex_deps mutator: receiver
is supposed to be parent, but child was used before. Interestingly lots
of deps are within the same group of module types(cc to cc, java to
java), it has worked. (note that receiver's DepIsInSameApex
implementation can be different).
This change fixes them by..
* walkPayloadDeps is now relying on ApexVariation, which is calculated
correctly by TopDown apex_deps mutator.
* use correct receiver for DepIsInSameApex in apex_deps mutator, which
requires for java.SdkLibrary to override the method and for
java.Library/Import to use passed dep instead of receiver to check its
membership of sdk.
Bug: 151071238
Test: build/boot
Change-Id: I0569ef4bb8e79635e4d97a89f421a8d8b7d26456
Adds support for specifying separate members to an sdk/module_exports
for different os types, e.g. separate ones for android and host.
Adds 'android:"arch_variant"' tag to the dynamically generated fields
for the sdk member types.
Merges the exported members from all variants together.
Determines the device/host_supported flags of the member snapshots by
whether the OsClasses used by their variants rather than the sdk's
host/device supported properties as they may be different.
Bug: 150451422
Test: m nothing
Change-Id: I41fbbcd8723aafd54826aad9b78eced9f66ef51c
Updates the member snapshot creation code to support multiple os types.
It basically sorts the variants by os type, then applies the code to
optimize the arch properties and then it optimizes the properties that
are common across architectures and extracts any properties that are
common across os types.
The java and cc member types needed to be modified to make the location
of the generated files within the snapshot os type dependent when there
is more than one os type. That was done by adding an OsPrefix() method
to the SdkMemberPropertiesBase which returns the os prefix to use when
there is > 1 os type and otherwise returns an empty string.
Added three tests, one for cc shared libraries, one for cc binary and
one for java header libraries.
Bug: 150451422
Test: m nothing
Change-Id: I08f5fbdd7852b06c9a9a2f1cfdc364338a3d5bac
Previously, while sdk tests would pass the sdk code would often fail
in androidmk processing. This change makes the tests more realistic
and will catch the errors earlier.
Bug: 142935992
Test: m nothing
Change-Id: Ifd0b2d7cf24e941c919f6b6e0beb2403a67d4308
Adds support for handling the common arch to the sdk module type
snapshot generation code and then refactors the java_library
and java_test snapshot processing to take advantage of that.
Bug: 150451422
Test: m nothing
Change-Id: If746f375f1c4288ab6b67c7d216593b70258b043
Generalize the processing of arch specific properties to reduce
duplication in snapshot module creation and simplify addition of
support for handling multiple os types.
Supporting multiple os types with the current method for building
snapshot modules would require every affected module type to add
support for it. Rather than duplicate multiple os type handling code
across those module types this work generalizes the process cc modules
use for handling arch types as it can be used as a basis for handling
multiple os types. Migrating module types over to this new process
will insulate them from having to handle multiple os types.
OB
SdkMemberType changes:
* BuildSnapshot is deprecated in favour of the new AddPrebuiltModule()
method.
* Additional methods, CreateVariantPropertiesStruct() and
FinalizeModule() are added.
* A new interface SdkMemberProperties, is defined that handles
extracting information from the variant (prior to common value
optimization) and adding properties to a property set.
The sdk module type uses these new methods and types to delegate the
member type specific processing to the relevant member types while
handling the behavior that is common across all members types, e.g.
extracting common values across multiple architectures. A future change
will leverage this processing to add support for multiple os types.
This change also refactors the cc module processing to use the new
process.
Bug: 150451422
Test: m nothing
Change-Id: If6ab2498407b17f50391d062cd9afc01b5e01af4
Following how NDK also creates its own install path.
Bug: 142935992
Test: cc_sdk_test.go, java_sdk_test.go
Change-Id: I98a3656903f37f6d7c90e6cf609431b2461a6161
Adds a CommonOS variant for sdk that depends on the os specific
variants and is used to generate a single sdk for multiple OsTypes,
e.g. host linux and android.
At the minute the member types only support a single OsType but the
basic mechanism for managing the CommonOS variant and collating the
variants across all of them is there.
The only visible effect of this change is that the location of the
generated snapshot is changed, it is no longer os specific and instead
is in the same location irrespective of which os it is built for.
A lot of tests needed to be changed to specify "common_os" as the
variant type instead of the specific os type. As that is the same across
all tests it is hard coded in CheckSnapshot method.
Test: m nothing
Bug: 150451422
Change-Id: If36be39b06d6910453649f7c288c2d34f688b2f4
Simplifies debugging sdk membership code when it is only called if an
appropriate member is present.
Test: m nothing
Change-Id: I32bc93e0484c829bdbd6b050822249feb6404f89
Makes the tests more realistic as they cover what happens when an sdk
OS specific variant is disabled (Windows is disabled by default). This
will allow issues around disabled variants to be detected early during
Soong bootstrapping.
Test: m nothing
Change-Id: I7ec47dbca52e38750166d755daf706aadea12d15
Ensures that the generated prebuilt modules do not contain empty
property sets.
Bug: 148933848
Test: m nothing
Change-Id: I99d5da1c4e3a72836e81e021ef043df53888e66b
This change adds support for transforming a property set after its
contents. This allows a transform to recursively prune empty property
sets that were created for a module.
The transformPropertySet method was renamed to
transformPropertySetBeforeContents and a new
transformPropertySetAfterContents method was added.
Bug: 148933848
Test: m nothing
Change-Id: Ia198d47e042b98c69406db4bc12859869816a387
Returning nil from transformPropertySet in order to remove it did
not work because it ends up comparing as (*bpPropertySet, nil) and
not an untyped nil which causes the test against nil to fail.
This change adds tests to check that returning nil will delete a
property/property set from the containing property set and fixes the
code so that it passes the tests. It extracts common code to transform
a property set and its contents as well as code for creating new
property sets.
Bug: 148933848
Test: m nothing
Change-Id: I35dc3c39c76e701821891622615c09b094cf697f
This is needed to allow the art-host-module-exports to restrict itself
to just managing the linux 64 bit version of the host tools as that is
the only variant that is currently supported by all host tools. This
greatly simplifies that process and allows us to make progress on the
unbundling.
Bug: 142935992
Test: m nothing
Change-Id: I62d016d97c2df73e5feecf912638f477fedd97c9
If an sdk variant is disabled (e.g. say windows) then it should not
add dependencies on corresponding sdk member variants as if those
variants have not been created then the build breaks unnecessarily.
This is needed to make progress on unbundling art which provides at
least one host tool (dex2oat) that is not supported on all platforms
and some (hiddenapi) that only produce 64 bit versions.
Initially, the art prebuilts will only support building 64 bit
versions of linux host tools.
Bug: 142935992
Test: m art-module-sdk art-module-host-exports art-module-test-exports
Change-Id: Idfbb40bb3cabc6eb731a2b38f045ed14f0a713cc
While it is invalid to mix //visibility:public with other rules in the
visibility property in a .bp file tt was possible, by overriding
defaults, to have //visibility:public mixed in with other rules in the
effective visibility rules. That caused problems when those effective
rules were used in an sdk snapshot.
This change replaces any set of rules that include //visibility:public
with just the //visibility:public rule. That simplifies those rules,
making them cheaper to process and ensures that the effective rules are
valid in the visibility property.
Adding test support required some refactoring of the
effectiveVisibilityRules(BaseModuleContext, ...) and underlying methods
to take a Config instead of BaseModuleContext as the tests do not have
access to BaseModuleContext.
Bug: 142935992
Test: m nothing - new tests failed without change, work with it
Add dex2oat to art-module-host-exports, build it and check the
generated Android.bp file in the snapshot to ensure the
visibility property for the dex2oat prebuilt does not mix
//visibility:public with other rules.
Change-Id: I08e7f0dcb40838d426fe88fedf69eae27b77473c
Move the NDK modules defined in app_test.go into cc/testing.go in
preparation for using sdk_version: "current" in more tests.
Test: all soong tests
Change-Id: I80bc31225fb4562fce42781219cefbbd6affae06
Internal sdk members are used by an sdk member but not exported by the
sdk. The exported sdk members are those listed explicitly in one of the
sdk member list properties, e.g. java_header_libs.
The prebuilts of an internal sdk member use a unique name so that they
do not clash with the source module. The use of the module internally
is an implementation detail that must not have any effect outside the
snapshot. Having the same name as the source module could cause it to
override the source module, hence why it needs a unique name.
Similarly, they are marked as private so as to prevent their accidental
use from outside the snapshot.
Bug: 142940300
Test: m nothing
Change-Id: Id5364b410be0592f65666afb3e40e9d3f020251c
Adds an SdkMemberType implementation for java_system_modules. It
specifies that java_system_modules can be used with sdk as well as
module_exports, and also that the libs property should be included
as transitive members in the sdk.
It also adds support for treating appropriate tagged properties in
the snapshot prebuilts module as references to sdk members so that
they are correctly transformed when creating the versioned modules.
Bug: 142940300
Test: m nothing
Change-Id: Ic10b5a6d5b92b6018334fe876f06feaf79cc55e9
Allow an sdk member type to treat some of its dependencies as being
members of the sdk.
Needed for the java_system_modules type whose libs property are an
implementation detail of the system module and so should not be
explicitly listed in the sdk module but still have to be included in
the sdk snapshot.
Bug: 142940300
Test: m nothing
Change-Id: I90f37dae269ef64a6fe9debd0bbaf29a64dd74d8
This change fixes a bug that apex_available is not enforced for static
dependencies. For example, a module with 'apex_available:
["//apex_available:platform"]' was able to be statically linked to any
APEX. This was happening because the check was done on the modules that
are actually installed to an APEX. Static dependencies of the modules
were not counted as they are not installed to the APEX as files.
Fixing this bug by doing the check by traversing the tree in the method
checkApexAvailability.
This change includes a few number of related changes:
1) DepIsInSameApex implementation for cc.Module was changed as well.
Previuosly, it returned false only when the dependency is actually a
stub variant of a lib. Now, it returns false when the dependency has one
or more stub variants. To understand why, we need to recall that when
there is a dependency to a lib having stubs, we actually create two
dependencies: to the non-stub variant and to the stub variant during the
DepsMutator phase. And later in the build action generation phase, we
choose one of them depending on the context. Also recall that an APEX
variant is created only when DepIsInSameApex returns true. Given these,
with the previous implementatin of DepIsInSameApex, we did create apex
variants of the non-stub variant of the dependency, while not creating
the apex variant for the stub variant. This is not right; we needlessly
created the apex variant. The extra apex variant has caused no harm so
far, but since the apex_available check became more correct, it actually
breaks the build. To fix the issue, we stop creating the APEX variant
both for non-stub and stub variants.
2) platform variant is created regardless of the apex_available value.
This is required for the case when a library X that provides stub is in
an APEX A and is configured to be available only for A. In that case,
libs in other APEX can't use the stub library since the stub library is
mutated only for apex A. By creating the platform variant for the stub
library, it can be used from outside as the default dependency variation
is set to the platform variant when creating the APEX variations.
3) The ApexAvailableWhitelist is added with the dependencies that were
revealed with this change.
Exempt-From-Owner-Approval: cherry-pick from internal
Bug: 147671264
Test: m
Merged-In: Iaedc05494085ff4e8af227a6392bdd0c338b8e6e
(cherry picked from commit fa89944c79)
Change-Id: Iaedc05494085ff4e8af227a6392bdd0c338b8e6e
Moves the struct to android/sdk.go and abstracts it behind a factory
method and interface. That allows it to be used outside the sdk
package.
This change is in preparation for adding support for module types that
have transitive sdk members.
Bug: 142940300
Test: m nothing
Change-Id: I71e5e0adf839b28a3a0952f82637637887f02688
This is preparation for enhancing the versioning tranformer to support
applying per property transformations. Specifically, to allow
properties to reference other libraries within the sdk.
Bug: 142940300
Test: m nothing
Change-Id: I99cdff4b407763ed395ff358d8110a63c6cf5589
Moves the code for transforming an unversioned module into a
versioned module into a transformer.
This is preparation for enhancing the versioning tranformer to support
applying per property transformations. Specifically, to allow
properties to reference other libraries within the sdk.
Bug: 142940300
Test: m nothing
Change-Id: Ia6b5cff8e1b5cd7232e031769d9fc0019e815fcb
Adds support for transforming a bpModule using a bpTransformer
instance. Rewrites the deepCopy code as a transformer.
This is preparation for allowing the update process to perform
per property transformations when transforming an unversioned
prebuilt module to a versioned prebuilt module for use by a
versioned snapshot.
Bug: 142940300
Test: m nothing
Change-Id: I0f438509df23bbb8a5862541b6fbfb1d5fbc06f6
Previously, only a shallow copy was made so the copy ends up sharing
some contents with the original. That was a potential source of bugs
as the copy was being made in order to be mutated.
This change switches to a deep copy; renaming the methods from
copy -> deepCopy to clarify the intent.
Makes the bpPropertySet member of bpModule a *bpPropertySet to avoid
unnecessary copying of bpPropertySet.
Bug: 142940300
Test: m nothing
Change-Id: I3f2eaa9fffab4e61d5a7cec81aa42fee9fdfec44
This means everything that goes into apexes need to be
explicitly labeled to be available for apex.
Whitelist the current offenders. This list should be
trimmed down.
Bug: 147364041
Test: m
Test: multiproduct_kati -only-soong
Exempt-From-Owner-Approval: cherry-pick from internal
Change-Id: I837299c6a15d46f8a5ba544b613776b1cc27d7b8
Merged-In: I837299c6a15d46f8a5ba544b613776b1cc27d7b8
(cherry picked from commit 93488cbb10d4882845abb732f8e53714f0982031)
The implicit dependency from an unversioned sdk member to the snapshot
versions should be excluded from visibility enforcement as it provides
no benefit and would require snapshot members to be visible to their
original source modules which would complicate visibility handling.
Also, corrects a spelling mistake in sdkMemberVersionedDepTag.
Bug: 142940300
Test: m nothing
Change-Id: Ib7ea6a3d3904d042dca0ea12f6b9196d40de970b
Adds java_test_import module type for use by the sdk snapshot and
adds java_tests property to the sdk and sdk_snapshot module type.
This is needed for the conscrypt test sdk.
Bug: 143678475
Test: m nothing
Change-Id: Ied4c56c978dac2f92a9b3bc34b3235d7eeba2fd3
Paul Duffin