platform_build_soong/android/apex.go

866 lines
34 KiB
Go
Raw Normal View History

// Copyright 2018 Google Inc. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package android
import (
"fmt"
"sort"
"strconv"
"strings"
"sync"
"github.com/google/blueprint"
)
var (
// This is the sdk version when APEX was first introduced
SdkVersion_Android10 = uncheckedFinalApiLevel(29)
)
// ApexInfo describes the metadata about one or more apexBundles that an apex variant of a module is
// part of. When an apex variant is created, the variant is associated with one apexBundle. But
// when multiple apex variants are merged for deduping (see mergeApexVariations), this holds the
// information about the apexBundles that are merged together.
// Accessible via `ctx.Provider(android.ApexInfoProvider).(android.ApexInfo)`
type ApexInfo struct {
// Name of the apex variation that this module (i.e. the apex variant of the module) is
// mutated into, or "" for a platform (i.e. non-APEX) variant. Note that a module can be
// included in multiple APEXes, in which case, the module is mutated into one or more
// variants, each of which is for an APEX. The variants then can later be deduped if they
// don't need to be compiled differently. This is an optimization done in
// mergeApexVariations.
ApexVariationName string
// Serialized ApiLevel that this module has to support at minimum. Should be accessed via
// MinSdkVersion() method. Cannot be stored in its struct form because this is cloned into
// properties structs, and ApiLevel has private members.
MinSdkVersionStr string
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
// True if this module comes from an updatable apexBundle.
Updatable bool
// The list of SDK modules that the containing apexBundle depends on.
RequiredSdks SdkRefs
// List of apexBundles that this apex variant of the module is associated with. Initially,
// the size of this list is one because one apex variant is associated with one apexBundle.
// When multiple apex variants are merged in mergeApexVariations, ApexInfo struct of the
// merged variant holds the list of apexBundles that are merged together.
InApexes []string
// Pointers to the ApexContents struct each of which is for apexBundle modules that this
// module is part of. The ApexContents gives information about which modules the apexBundle
// has and whether a module became part of the apexBundle via a direct dependency or not.
ApexContents []*ApexContents
Export dex implementation jars from prebuilt_apex Dexpreopt and boot jars package check all require access to dex implementation jars created for java_library and java_sdk_library. They were available when building from source but not when building from prebuilts, even though they are embedded within the .apex files that are referenced from prebuilt_apex. This changes adds support to prebuilt_apex to export the dex implementation jars and updates java_import to use those exported dex implementation jars. In a source build dexpreopt/boot jars package check access the apex (or platform) specific variant of a java_library, e.g. core-oj, from which it retrieves the dex implementation jar path. After this change in a prebuilt build dexpreopt/boot jars package check behave in the same way except in this case they retrieve the dex implementation jar path from the apex (or platform) specific variant of the java_import, e.g. core-oj. The work to export files from a `.apex` file for use by other modules is performed by a new `deapexer` module type. It is not used directly in an `Android.bp` file but instead is created implicitly by `prebuilt_apex`, In order to do that this contains the following changes: * Adds a new `dexapexer` module type to handle the exporting of files from the `.apex` file. * Adds an exported_java_libs property to prebuilt_apex to specify the set of libraries whose dex implementation jars need exporting. * Creates apex specific variants of the libraries listed in the exported_java_libs property. * Adds the set of exported files to the ApexInfo to make them available to the apex specific variants. * Prevents the prebuilt_apex variants from being merged together as they will not be compatible. * Modifies java_import to use the exported file for variants of a prebuilt_apex. * Adds a ninja rule to unpack (using deapexer) the contents of the prebuilt_apex's apex file, verify that the required files are present and make them available as outputs for other rules to use. * Some minor refactorings to support these changes. * Adds tests to cover prebuilt only, prebuilt with source preferred, and prebuilt preferred with source. Test: m nothing Bug: 171061220 Change-Id: Ic9bed81fb65b92f0d59f64c0bce168a9ed44cfac
2020-11-03 01:32:38 +08:00
// True if this is for a prebuilt_apex.
//
// If true then this will customize the apex processing to make it suitable for handling
// prebuilt_apex, e.g. it will prevent ApexInfos from being merged together.
//
// See Prebuilt.ApexInfoMutator for more information.
ForPrebuiltApex bool
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
}
var ApexInfoProvider = blueprint.NewMutatorProvider(ApexInfo{}, "apex")
// mergedName gives the name of the alias variation that will be used when multiple apex variations
// of a module can be deduped into one variation. For example, if libfoo is included in both apex.a
// and apex.b, and if the two APEXes have the same min_sdk_version (say 29), then libfoo doesn't
// have to be built twice, but only once. In that case, the two apex variations apex.a and apex.b
// are configured to have the same alias variation named apex29.
func (i ApexInfo) mergedName(ctx PathContext) string {
name := "apex" + strconv.Itoa(i.MinSdkVersion(ctx).FinalOrFutureInt())
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
for _, sdk := range i.RequiredSdks {
name += "_" + sdk.Name + "_" + sdk.Version
}
return name
}
// MinSdkVersion gives the api level that this module has to support at minimum. This is from the
// min_sdk_version property of the containing apexBundle.
func (i ApexInfo) MinSdkVersion(ctx PathContext) ApiLevel {
return ApiLevelOrPanic(ctx, i.MinSdkVersionStr)
}
// IsForPlatform tells whether this module is for the platform or not. If false is returned, it
// means that this apex variant of the module is built for an APEX.
func (i ApexInfo) IsForPlatform() bool {
return i.ApexVariationName == ""
}
// InApex tells whether this apex variant of the module is part of the given apexBundle or not.
func (i ApexInfo) InApex(apex string) bool {
for _, a := range i.InApexes {
if a == apex {
return true
}
}
return false
}
// InApexByBaseName tells whether this apex variant of the module is part of the given APEX or not,
// where the APEX is specified by its canonical base name, i.e. typically beginning with
// "com.android.". In particular this function doesn't differentiate between source and prebuilt
// APEXes, where the latter may have "prebuilt_" prefixes.
func (i ApexInfo) InApexByBaseName(apex string) bool {
for _, a := range i.InApexes {
if RemoveOptionalPrebuiltPrefix(a) == apex {
return true
}
}
return false
}
// ApexTestForInfo stores the contents of APEXes for which this module is a test - although this
// module is not part of the APEX - and thus has access to APEX internals.
type ApexTestForInfo struct {
ApexContents []*ApexContents
}
var ApexTestForInfoProvider = blueprint.NewMutatorProvider(ApexTestForInfo{}, "apex_test_for")
// DepIsInSameApex defines an interface that should be used to determine whether a given dependency
// should be considered as part of the same APEX as the current module or not. Note: this was
// extracted from ApexModule to make it easier to define custom subsets of the ApexModule interface
// and improve code navigation within the IDE.
type DepIsInSameApex interface {
// DepIsInSameApex tests if the other module 'dep' is considered as part of the same APEX as
// this module. For example, a static lib dependency usually returns true here, while a
// shared lib dependency to a stub library returns false.
DepIsInSameApex(ctx BaseModuleContext, dep Module) bool
}
// ApexModule is the interface that a module type is expected to implement if the module has to be
// built differently depending on whether the module is destined for an APEX or not (i.e., installed
// to one of the regular partitions).
//
// Native shared libraries are one such module type; when it is built for an APEX, it should depend
// only on stable interfaces such as NDK, stable AIDL, or C APIs from other APEXes.
//
// A module implementing this interface will be mutated into multiple variations by apex.apexMutator
// if it is directly or indirectly included in one or more APEXes. Specifically, if a module is
// included in apex.foo and apex.bar then three apex variants are created: platform, apex.foo and
// apex.bar. The platform variant is for the regular partitions (e.g., /system or /vendor, etc.)
// while the other two are for the APEXs, respectively. The latter two variations can be merged (see
// mergedName) when the two APEXes have the same min_sdk_version requirement.
type ApexModule interface {
Module
DepIsInSameApex
apexModuleBase() *ApexModuleBase
// Marks that this module should be built for the specified APEX. Call this BEFORE
// apex.apexMutator is run.
BuildForApex(apex ApexInfo)
// Returns true if this module is present in any APEX either directly or indirectly. Call
// this after apex.apexMutator is run.
InAnyApex() bool
// Returns true if this module is directly in any APEX. Call this AFTER apex.apexMutator is
// run.
DirectlyInAnyApex() bool
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
// NotInPlatform tells whether or not this module is included in an APEX and therefore
// shouldn't be exposed to the platform (i.e. outside of the APEX) directly. A module is
// considered to be included in an APEX either when there actually is an APEX that
// explicitly has the module as its dependency or the module is not available to the
// platform, which indicates that the module belongs to at least one or more other APEXes.
NotInPlatform() bool
// Tests if this module could have APEX variants. Even when a module type implements
// ApexModule interface, APEX variants are created only for the module instances that return
// true here. This is useful for not creating APEX variants for certain types of shared
// libraries such as NDK stubs.
CanHaveApexVariants() bool
// Tests if this module can be installed to APEX as a file. For example, this would return
// true for shared libs while return false for static libs because static libs are not
// installable module (but it can still be mutated for APEX)
IsInstallableToApex() bool
// Tests if this module is available for the specified APEX or ":platform". This is from the
// apex_available property of the module.
AvailableFor(what string) bool
// Returns true if this module is not available to platform (i.e. apex_available property
// doesn't have "//apex_available:platform"), or shouldn't be available to platform, which
// is the case when this module depends on other module that isn't available to platform.
mark platform un-availability A module is marked unavailable for platform when 1) it does not have "//apex_available:platform" in its apex_available property, or 2) it depends on another module that is unavailable for platform. In that case, LOCAL_NOT_AVAILABLE_FOR_PLATFORM is set to true for the module in the Make world. Later, that flag is used to ensure that there is no module with the flag is installed to the device. The reason why this isn't entirely done in Soong is because Soong doesn't know if a module will be installed to the device or not. To explain this, let's have an example. cc_test { name: "mytest", static_libs: ["libfoo"]} cc_library_static { name: "libfoo", static_libs: ["libbar"]} cc_library { name: "libbar", apex_available: ["com.android.xxx"]} Here, libbar is not available for platform, but is used by libfoo which is available for platform (apex_available defaults to "//apex_available:platform"). libfoo is again depended on by mytest which again is available for platform. The use of libbar should be allowed in the context of test; we don't want to make libbar available to platform just for the dependency from test because it will allow non-test uses of the library as well. Soong by itself can't tell whether libfoo and libbar are used only in the context of a test. There could be another module depending them, e.g., cc_library_shared { name: "mylib", static_libs: ["libfoo"] } can exist and it might be installed to the device, in which case we really should trigger an error. Since Make has the knowledge of what's installed and what's not, the check should be done there. Bug: 153073816 Test: m Test: remove "//apex_available:platform" from libmdnssd (it is currently installed to /system/lib), and check that `m system_image` fails Change-Id: Ia304cc5f41f173229e8a154e90cea4dce46dcebe
2020-04-07 15:37:39 +08:00
NotAvailableForPlatform() bool
// Marks that this module is not available to platform. Set by the
mark platform un-availability A module is marked unavailable for platform when 1) it does not have "//apex_available:platform" in its apex_available property, or 2) it depends on another module that is unavailable for platform. In that case, LOCAL_NOT_AVAILABLE_FOR_PLATFORM is set to true for the module in the Make world. Later, that flag is used to ensure that there is no module with the flag is installed to the device. The reason why this isn't entirely done in Soong is because Soong doesn't know if a module will be installed to the device or not. To explain this, let's have an example. cc_test { name: "mytest", static_libs: ["libfoo"]} cc_library_static { name: "libfoo", static_libs: ["libbar"]} cc_library { name: "libbar", apex_available: ["com.android.xxx"]} Here, libbar is not available for platform, but is used by libfoo which is available for platform (apex_available defaults to "//apex_available:platform"). libfoo is again depended on by mytest which again is available for platform. The use of libbar should be allowed in the context of test; we don't want to make libbar available to platform just for the dependency from test because it will allow non-test uses of the library as well. Soong by itself can't tell whether libfoo and libbar are used only in the context of a test. There could be another module depending them, e.g., cc_library_shared { name: "mylib", static_libs: ["libfoo"] } can exist and it might be installed to the device, in which case we really should trigger an error. Since Make has the knowledge of what's installed and what's not, the check should be done there. Bug: 153073816 Test: m Test: remove "//apex_available:platform" from libmdnssd (it is currently installed to /system/lib), and check that `m system_image` fails Change-Id: Ia304cc5f41f173229e8a154e90cea4dce46dcebe
2020-04-07 15:37:39 +08:00
// check-platform-availability mutator in the apex package.
SetNotAvailableForPlatform()
// Returns the list of APEXes that this module is a test for. The module has access to the
// private part of the listed APEXes even when it is not included in the APEXes. This by
// default returns nil. A module type should override the default implementation. For
// example, cc_test module type returns the value of test_for here.
TestFor() []string
// Returns nil (success) if this module should support the given sdk version. Returns an
// error if not. No default implementation is provided for this method. A module type
// implementing this interface should provide an implementation. A module supports an sdk
// version when the module's min_sdk_version is equal to or less than the given sdk version.
ShouldSupportSdkVersion(ctx BaseModuleContext, sdkVersion ApiLevel) error
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
// Returns true if this module needs a unique variation per apex, effectively disabling the
// deduping. This is turned on when, for example if use_apex_name_macro is set so that each
// apex variant should be built with different macro definitions.
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
UniqueApexVariations() bool
}
// Properties that are common to all module types implementing ApexModule interface.
type ApexProperties struct {
// Availability of this module in APEXes. Only the listed APEXes can contain this module. If
// the module has stubs then other APEXes and the platform may access it through them
// (subject to visibility).
//
// "//apex_available:anyapex" is a pseudo APEX name that matches to any APEX.
// "//apex_available:platform" refers to non-APEX partitions like "system.img".
// "com.android.gki.*" matches any APEX module name with the prefix "com.android.gki.".
// Default is ["//apex_available:platform"].
Apex_available []string
// See ApexModule.InAnyApex()
InAnyApex bool `blueprint:"mutated"`
// See ApexModule.DirectlyInAnyApex()
DirectlyInAnyApex bool `blueprint:"mutated"`
// AnyVariantDirectlyInAnyApex is true in the primary variant of a module if _any_ variant
// of the module is directly in any apex. This includes host, arch, asan, etc. variants. It
// is unused in any variant that is not the primary variant. Ideally this wouldn't be used,
// as it incorrectly mixes arch variants if only one arch is in an apex, but a few places
// depend on it, for example when an ASAN variant is created before the apexMutator. Call
// this after apex.apexMutator is run.
AnyVariantDirectlyInAnyApex bool `blueprint:"mutated"`
mark platform un-availability A module is marked unavailable for platform when 1) it does not have "//apex_available:platform" in its apex_available property, or 2) it depends on another module that is unavailable for platform. In that case, LOCAL_NOT_AVAILABLE_FOR_PLATFORM is set to true for the module in the Make world. Later, that flag is used to ensure that there is no module with the flag is installed to the device. The reason why this isn't entirely done in Soong is because Soong doesn't know if a module will be installed to the device or not. To explain this, let's have an example. cc_test { name: "mytest", static_libs: ["libfoo"]} cc_library_static { name: "libfoo", static_libs: ["libbar"]} cc_library { name: "libbar", apex_available: ["com.android.xxx"]} Here, libbar is not available for platform, but is used by libfoo which is available for platform (apex_available defaults to "//apex_available:platform"). libfoo is again depended on by mytest which again is available for platform. The use of libbar should be allowed in the context of test; we don't want to make libbar available to platform just for the dependency from test because it will allow non-test uses of the library as well. Soong by itself can't tell whether libfoo and libbar are used only in the context of a test. There could be another module depending them, e.g., cc_library_shared { name: "mylib", static_libs: ["libfoo"] } can exist and it might be installed to the device, in which case we really should trigger an error. Since Make has the knowledge of what's installed and what's not, the check should be done there. Bug: 153073816 Test: m Test: remove "//apex_available:platform" from libmdnssd (it is currently installed to /system/lib), and check that `m system_image` fails Change-Id: Ia304cc5f41f173229e8a154e90cea4dce46dcebe
2020-04-07 15:37:39 +08:00
// See ApexModule.NotAvailableForPlatform()
mark platform un-availability A module is marked unavailable for platform when 1) it does not have "//apex_available:platform" in its apex_available property, or 2) it depends on another module that is unavailable for platform. In that case, LOCAL_NOT_AVAILABLE_FOR_PLATFORM is set to true for the module in the Make world. Later, that flag is used to ensure that there is no module with the flag is installed to the device. The reason why this isn't entirely done in Soong is because Soong doesn't know if a module will be installed to the device or not. To explain this, let's have an example. cc_test { name: "mytest", static_libs: ["libfoo"]} cc_library_static { name: "libfoo", static_libs: ["libbar"]} cc_library { name: "libbar", apex_available: ["com.android.xxx"]} Here, libbar is not available for platform, but is used by libfoo which is available for platform (apex_available defaults to "//apex_available:platform"). libfoo is again depended on by mytest which again is available for platform. The use of libbar should be allowed in the context of test; we don't want to make libbar available to platform just for the dependency from test because it will allow non-test uses of the library as well. Soong by itself can't tell whether libfoo and libbar are used only in the context of a test. There could be another module depending them, e.g., cc_library_shared { name: "mylib", static_libs: ["libfoo"] } can exist and it might be installed to the device, in which case we really should trigger an error. Since Make has the knowledge of what's installed and what's not, the check should be done there. Bug: 153073816 Test: m Test: remove "//apex_available:platform" from libmdnssd (it is currently installed to /system/lib), and check that `m system_image` fails Change-Id: Ia304cc5f41f173229e8a154e90cea4dce46dcebe
2020-04-07 15:37:39 +08:00
NotAvailableForPlatform bool `blueprint:"mutated"`
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
// See ApexModule.UniqueApexVariants()
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
UniqueApexVariationsForDeps bool `blueprint:"mutated"`
}
// Marker interface that identifies dependencies that are excluded from APEX contents.
type ExcludeFromApexContentsTag interface {
blueprint.DependencyTag
// Method that differentiates this interface from others.
ExcludeFromApexContents()
}
// Marker interface that identifies dependencies that should inherit the DirectlyInAnyApex state
// from the parent to the child. For example, stubs libraries are marked as DirectlyInAnyApex if
// their implementation is in an apex.
type CopyDirectlyInAnyApexTag interface {
blueprint.DependencyTag
// Method that differentiates this interface from others.
CopyDirectlyInAnyApex()
}
// Interface that identifies dependencies to skip Apex dependency check
type SkipApexAllowedDependenciesCheck interface {
// Returns true to skip the Apex dependency check, which limits the allowed dependency in build.
SkipApexAllowedDependenciesCheck() bool
}
// ApexModuleBase provides the default implementation for the ApexModule interface. APEX-aware
// modules are expected to include this struct and call InitApexModule().
type ApexModuleBase struct {
ApexProperties ApexProperties
canHaveApexVariants bool
apexInfos []ApexInfo
apexInfosLock sync.Mutex // protects apexInfos during parallel apexInfoMutator
}
// Initializes ApexModuleBase struct. Not calling this (even when inheriting from ApexModuleBase)
// prevents the module from being mutated for apexBundle.
func InitApexModule(m ApexModule) {
base := m.apexModuleBase()
base.canHaveApexVariants = true
m.AddProperties(&base.ApexProperties)
}
// Implements ApexModule
func (m *ApexModuleBase) apexModuleBase() *ApexModuleBase {
return m
}
// Implements ApexModule
func (m *ApexModuleBase) ApexAvailable() []string {
return m.ApexProperties.Apex_available
}
// Implements ApexModule
func (m *ApexModuleBase) BuildForApex(apex ApexInfo) {
m.apexInfosLock.Lock()
defer m.apexInfosLock.Unlock()
for _, v := range m.apexInfos {
if v.ApexVariationName == apex.ApexVariationName {
return
}
}
m.apexInfos = append(m.apexInfos, apex)
}
// Implements ApexModule
func (m *ApexModuleBase) InAnyApex() bool {
return m.ApexProperties.InAnyApex
}
// Implements ApexModule
func (m *ApexModuleBase) DirectlyInAnyApex() bool {
return m.ApexProperties.DirectlyInAnyApex
}
// Implements ApexModule
func (m *ApexModuleBase) NotInPlatform() bool {
return m.ApexProperties.AnyVariantDirectlyInAnyApex || !m.AvailableFor(AvailableToPlatform)
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
}
// Implements ApexModule
func (m *ApexModuleBase) CanHaveApexVariants() bool {
return m.canHaveApexVariants
}
// Implements ApexModule
func (m *ApexModuleBase) IsInstallableToApex() bool {
// If needed, this will bel overridden by concrete types inheriting
// ApexModuleBase
return false
}
// Implements ApexModule
func (m *ApexModuleBase) TestFor() []string {
// If needed, this will be overridden by concrete types inheriting
// ApexModuleBase
return nil
}
// Implements ApexModule
func (m *ApexModuleBase) UniqueApexVariations() bool {
// If needed, this will bel overridden by concrete types inheriting
// ApexModuleBase
return false
}
// Implements ApexModule
func (m *ApexModuleBase) DepIsInSameApex(ctx BaseModuleContext, dep Module) bool {
// By default, if there is a dependency from A to B, we try to include both in the same
// APEX, unless B is explicitly from outside of the APEX (i.e. a stubs lib). Thus, returning
// true. This is overridden by some module types like apex.ApexBundle, cc.Module,
// java.Module, etc.
return true
}
const (
AvailableToPlatform = "//apex_available:platform"
AvailableToAnyApex = "//apex_available:anyapex"
AvailableToGkiApex = "com.android.gki.*"
)
// CheckAvailableForApex provides the default algorithm for checking the apex availability. When the
// availability is empty, it defaults to ["//apex_available:platform"] which means "available to the
// platform but not available to any APEX". When the list is not empty, `what` is matched against
// the list. If there is any matching element in the list, thus function returns true. The special
// availability "//apex_available:anyapex" matches with anything except for
// "//apex_available:platform".
func CheckAvailableForApex(what string, apex_available []string) bool {
if len(apex_available) == 0 {
return what == AvailableToPlatform
}
return InList(what, apex_available) ||
(what != AvailableToPlatform && InList(AvailableToAnyApex, apex_available)) ||
(strings.HasPrefix(what, "com.android.gki.") && InList(AvailableToGkiApex, apex_available))
}
// Implements ApexModule
func (m *ApexModuleBase) AvailableFor(what string) bool {
return CheckAvailableForApex(what, m.ApexProperties.Apex_available)
}
// Implements ApexModule
mark platform un-availability A module is marked unavailable for platform when 1) it does not have "//apex_available:platform" in its apex_available property, or 2) it depends on another module that is unavailable for platform. In that case, LOCAL_NOT_AVAILABLE_FOR_PLATFORM is set to true for the module in the Make world. Later, that flag is used to ensure that there is no module with the flag is installed to the device. The reason why this isn't entirely done in Soong is because Soong doesn't know if a module will be installed to the device or not. To explain this, let's have an example. cc_test { name: "mytest", static_libs: ["libfoo"]} cc_library_static { name: "libfoo", static_libs: ["libbar"]} cc_library { name: "libbar", apex_available: ["com.android.xxx"]} Here, libbar is not available for platform, but is used by libfoo which is available for platform (apex_available defaults to "//apex_available:platform"). libfoo is again depended on by mytest which again is available for platform. The use of libbar should be allowed in the context of test; we don't want to make libbar available to platform just for the dependency from test because it will allow non-test uses of the library as well. Soong by itself can't tell whether libfoo and libbar are used only in the context of a test. There could be another module depending them, e.g., cc_library_shared { name: "mylib", static_libs: ["libfoo"] } can exist and it might be installed to the device, in which case we really should trigger an error. Since Make has the knowledge of what's installed and what's not, the check should be done there. Bug: 153073816 Test: m Test: remove "//apex_available:platform" from libmdnssd (it is currently installed to /system/lib), and check that `m system_image` fails Change-Id: Ia304cc5f41f173229e8a154e90cea4dce46dcebe
2020-04-07 15:37:39 +08:00
func (m *ApexModuleBase) NotAvailableForPlatform() bool {
return m.ApexProperties.NotAvailableForPlatform
}
// Implements ApexModule
mark platform un-availability A module is marked unavailable for platform when 1) it does not have "//apex_available:platform" in its apex_available property, or 2) it depends on another module that is unavailable for platform. In that case, LOCAL_NOT_AVAILABLE_FOR_PLATFORM is set to true for the module in the Make world. Later, that flag is used to ensure that there is no module with the flag is installed to the device. The reason why this isn't entirely done in Soong is because Soong doesn't know if a module will be installed to the device or not. To explain this, let's have an example. cc_test { name: "mytest", static_libs: ["libfoo"]} cc_library_static { name: "libfoo", static_libs: ["libbar"]} cc_library { name: "libbar", apex_available: ["com.android.xxx"]} Here, libbar is not available for platform, but is used by libfoo which is available for platform (apex_available defaults to "//apex_available:platform"). libfoo is again depended on by mytest which again is available for platform. The use of libbar should be allowed in the context of test; we don't want to make libbar available to platform just for the dependency from test because it will allow non-test uses of the library as well. Soong by itself can't tell whether libfoo and libbar are used only in the context of a test. There could be another module depending them, e.g., cc_library_shared { name: "mylib", static_libs: ["libfoo"] } can exist and it might be installed to the device, in which case we really should trigger an error. Since Make has the knowledge of what's installed and what's not, the check should be done there. Bug: 153073816 Test: m Test: remove "//apex_available:platform" from libmdnssd (it is currently installed to /system/lib), and check that `m system_image` fails Change-Id: Ia304cc5f41f173229e8a154e90cea4dce46dcebe
2020-04-07 15:37:39 +08:00
func (m *ApexModuleBase) SetNotAvailableForPlatform() {
m.ApexProperties.NotAvailableForPlatform = true
}
// This function makes sure that the apex_available property is valid
func (m *ApexModuleBase) checkApexAvailableProperty(mctx BaseModuleContext) {
for _, n := range m.ApexProperties.Apex_available {
if n == AvailableToPlatform || n == AvailableToAnyApex || n == AvailableToGkiApex {
continue
}
if !mctx.OtherModuleExists(n) && !mctx.Config().AllowMissingDependencies() {
mctx.PropertyErrorf("apex_available", "%q is not a valid module name", n)
}
}
}
type byApexName []ApexInfo
func (a byApexName) Len() int { return len(a) }
func (a byApexName) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
func (a byApexName) Less(i, j int) bool { return a[i].ApexVariationName < a[j].ApexVariationName }
// mergeApexVariations deduplicates apex variations that would build identically into a common
// variation. It returns the reduced list of variations and a list of aliases from the original
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
// variation names to the new variation names.
func mergeApexVariations(ctx PathContext, apexInfos []ApexInfo) (merged []ApexInfo, aliases [][2]string) {
sort.Sort(byApexName(apexInfos))
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
seen := make(map[string]int)
for _, apexInfo := range apexInfos {
Export dex implementation jars from prebuilt_apex Dexpreopt and boot jars package check all require access to dex implementation jars created for java_library and java_sdk_library. They were available when building from source but not when building from prebuilts, even though they are embedded within the .apex files that are referenced from prebuilt_apex. This changes adds support to prebuilt_apex to export the dex implementation jars and updates java_import to use those exported dex implementation jars. In a source build dexpreopt/boot jars package check access the apex (or platform) specific variant of a java_library, e.g. core-oj, from which it retrieves the dex implementation jar path. After this change in a prebuilt build dexpreopt/boot jars package check behave in the same way except in this case they retrieve the dex implementation jar path from the apex (or platform) specific variant of the java_import, e.g. core-oj. The work to export files from a `.apex` file for use by other modules is performed by a new `deapexer` module type. It is not used directly in an `Android.bp` file but instead is created implicitly by `prebuilt_apex`, In order to do that this contains the following changes: * Adds a new `dexapexer` module type to handle the exporting of files from the `.apex` file. * Adds an exported_java_libs property to prebuilt_apex to specify the set of libraries whose dex implementation jars need exporting. * Creates apex specific variants of the libraries listed in the exported_java_libs property. * Adds the set of exported files to the ApexInfo to make them available to the apex specific variants. * Prevents the prebuilt_apex variants from being merged together as they will not be compatible. * Modifies java_import to use the exported file for variants of a prebuilt_apex. * Adds a ninja rule to unpack (using deapexer) the contents of the prebuilt_apex's apex file, verify that the required files are present and make them available as outputs for other rules to use. * Some minor refactorings to support these changes. * Adds tests to cover prebuilt only, prebuilt with source preferred, and prebuilt preferred with source. Test: m nothing Bug: 171061220 Change-Id: Ic9bed81fb65b92f0d59f64c0bce168a9ed44cfac
2020-11-03 01:32:38 +08:00
// If this is for a prebuilt apex then use the actual name of the apex variation to prevent this
// from being merged with other ApexInfo. See Prebuilt.ApexInfoMutator for more information.
if apexInfo.ForPrebuiltApex {
merged = append(merged, apexInfo)
continue
}
// Merge the ApexInfo together. If a compatible ApexInfo exists then merge the information from
// this one into it, otherwise create a new merged ApexInfo from this one and save it away so
// other ApexInfo instances can be merged into it.
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
apexName := apexInfo.ApexVariationName
mergedName := apexInfo.mergedName(ctx)
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
if index, exists := seen[mergedName]; exists {
// Variants having the same mergedName are deduped
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
merged[index].InApexes = append(merged[index].InApexes, apexName)
merged[index].ApexContents = append(merged[index].ApexContents, apexInfo.ApexContents...)
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
merged[index].Updatable = merged[index].Updatable || apexInfo.Updatable
} else {
seen[mergedName] = len(merged)
apexInfo.ApexVariationName = mergedName
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
apexInfo.InApexes = CopyOf(apexInfo.InApexes)
apexInfo.ApexContents = append([]*ApexContents(nil), apexInfo.ApexContents...)
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
merged = append(merged, apexInfo)
}
aliases = append(aliases, [2]string{apexName, mergedName})
}
return merged, aliases
}
// CreateApexVariations mutates a given module into multiple apex variants each of which is for an
// apexBundle (and/or the platform) where the module is part of.
func CreateApexVariations(mctx BottomUpMutatorContext, module ApexModule) []Module {
base := module.apexModuleBase()
apex_available tracks static dependencies This change fixes a bug that apex_available is not enforced for static dependencies. For example, a module with 'apex_available: ["//apex_available:platform"]' was able to be statically linked to any APEX. This was happening because the check was done on the modules that are actually installed to an APEX. Static dependencies of the modules were not counted as they are not installed to the APEX as files. Fixing this bug by doing the check by traversing the tree in the method checkApexAvailability. This change includes a few number of related changes: 1) DepIsInSameApex implementation for cc.Module was changed as well. Previuosly, it returned false only when the dependency is actually a stub variant of a lib. Now, it returns false when the dependency has one or more stub variants. To understand why, we need to recall that when there is a dependency to a lib having stubs, we actually create two dependencies: to the non-stub variant and to the stub variant during the DepsMutator phase. And later in the build action generation phase, we choose one of them depending on the context. Also recall that an APEX variant is created only when DepIsInSameApex returns true. Given these, with the previous implementatin of DepIsInSameApex, we did create apex variants of the non-stub variant of the dependency, while not creating the apex variant for the stub variant. This is not right; we needlessly created the apex variant. The extra apex variant has caused no harm so far, but since the apex_available check became more correct, it actually breaks the build. To fix the issue, we stop creating the APEX variant both for non-stub and stub variants. 2) platform variant is created regardless of the apex_available value. This is required for the case when a library X that provides stub is in an APEX A and is configured to be available only for A. In that case, libs in other APEX can't use the stub library since the stub library is mutated only for apex A. By creating the platform variant for the stub library, it can be used from outside as the default dependency variation is set to the platform variant when creating the APEX variations. 3) The ApexAvailableWhitelist is added with the dependencies that were revealed with this change. Exempt-From-Owner-Approval: cherry-pick from internal Bug: 147671264 Test: m Merged-In: Iaedc05494085ff4e8af227a6392bdd0c338b8e6e (cherry picked from commit fa89944c79f19552e906b41fd03a4981903eee7e) Change-Id: Iaedc05494085ff4e8af227a6392bdd0c338b8e6e
2020-01-31 01:49:53 +08:00
// Shortcut
if len(base.apexInfos) == 0 {
return nil
}
// Do some validity checks.
// TODO(jiyong): is this the right place?
base.checkApexAvailableProperty(mctx)
apex_available tracks static dependencies This change fixes a bug that apex_available is not enforced for static dependencies. For example, a module with 'apex_available: ["//apex_available:platform"]' was able to be statically linked to any APEX. This was happening because the check was done on the modules that are actually installed to an APEX. Static dependencies of the modules were not counted as they are not installed to the APEX as files. Fixing this bug by doing the check by traversing the tree in the method checkApexAvailability. This change includes a few number of related changes: 1) DepIsInSameApex implementation for cc.Module was changed as well. Previuosly, it returned false only when the dependency is actually a stub variant of a lib. Now, it returns false when the dependency has one or more stub variants. To understand why, we need to recall that when there is a dependency to a lib having stubs, we actually create two dependencies: to the non-stub variant and to the stub variant during the DepsMutator phase. And later in the build action generation phase, we choose one of them depending on the context. Also recall that an APEX variant is created only when DepIsInSameApex returns true. Given these, with the previous implementatin of DepIsInSameApex, we did create apex variants of the non-stub variant of the dependency, while not creating the apex variant for the stub variant. This is not right; we needlessly created the apex variant. The extra apex variant has caused no harm so far, but since the apex_available check became more correct, it actually breaks the build. To fix the issue, we stop creating the APEX variant both for non-stub and stub variants. 2) platform variant is created regardless of the apex_available value. This is required for the case when a library X that provides stub is in an APEX A and is configured to be available only for A. In that case, libs in other APEX can't use the stub library since the stub library is mutated only for apex A. By creating the platform variant for the stub library, it can be used from outside as the default dependency variation is set to the platform variant when creating the APEX variations. 3) The ApexAvailableWhitelist is added with the dependencies that were revealed with this change. Exempt-From-Owner-Approval: cherry-pick from internal Bug: 147671264 Test: m Merged-In: Iaedc05494085ff4e8af227a6392bdd0c338b8e6e (cherry picked from commit fa89944c79f19552e906b41fd03a4981903eee7e) Change-Id: Iaedc05494085ff4e8af227a6392bdd0c338b8e6e
2020-01-31 01:49:53 +08:00
var apexInfos []ApexInfo
var aliases [][2]string
if !mctx.Module().(ApexModule).UniqueApexVariations() && !base.ApexProperties.UniqueApexVariationsForDeps {
apexInfos, aliases = mergeApexVariations(mctx, base.apexInfos)
} else {
apexInfos = base.apexInfos
}
// base.apexInfos is only needed to propagate the list of apexes from apexInfoMutator to
// apexMutator. It is no longer accurate after mergeApexVariations, and won't be copied to
// all but the first created variant. Clear it so it doesn't accidentally get used later.
base.apexInfos = nil
sort.Sort(byApexName(apexInfos))
var inApex ApexMembership
for _, a := range apexInfos {
for _, apexContents := range a.ApexContents {
inApex = inApex.merge(apexContents.contents[mctx.ModuleName()])
}
}
base.ApexProperties.InAnyApex = true
base.ApexProperties.DirectlyInAnyApex = inApex == directlyInApex
defaultVariation := ""
mctx.SetDefaultDependencyVariation(&defaultVariation)
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
variations := []string{defaultVariation}
for _, a := range apexInfos {
variations = append(variations, a.ApexVariationName)
}
modules := mctx.CreateVariations(variations...)
for i, mod := range modules {
platformVariation := i == 0
if platformVariation && !mctx.Host() && !mod.(ApexModule).AvailableFor(AvailableToPlatform) {
// Do not install the module for platform, but still allow it to output
// uninstallable AndroidMk entries in certain cases when they have side
// effects. TODO(jiyong): move this routine to somewhere else
mod.MakeUninstallable()
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
}
if !platformVariation {
mctx.SetVariationProvider(mod, ApexInfoProvider, apexInfos[i-1])
}
}
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
for _, alias := range aliases {
mctx.CreateAliasVariation(alias[0], alias[1])
}
return modules
}
// UpdateUniqueApexVariationsForDeps sets UniqueApexVariationsForDeps if any dependencies that are
// in the same APEX have unique APEX variations so that the module can link against the right
// variant.
func UpdateUniqueApexVariationsForDeps(mctx BottomUpMutatorContext, am ApexModule) {
// anyInSameApex returns true if the two ApexInfo lists contain any values in an InApexes
// list in common. It is used instead of DepIsInSameApex because it needs to determine if
// the dep is in the same APEX due to being directly included, not only if it is included
// _because_ it is a dependency.
anyInSameApex := func(a, b []ApexInfo) bool {
collectApexes := func(infos []ApexInfo) []string {
var ret []string
for _, info := range infos {
ret = append(ret, info.InApexes...)
}
return ret
}
aApexes := collectApexes(a)
bApexes := collectApexes(b)
sort.Strings(bApexes)
for _, aApex := range aApexes {
index := sort.SearchStrings(bApexes, aApex)
if index < len(bApexes) && bApexes[index] == aApex {
return true
}
}
return false
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
}
// If any of the dependencies requires unique apex variations, so does this module.
mctx.VisitDirectDeps(func(dep Module) {
if depApexModule, ok := dep.(ApexModule); ok {
if anyInSameApex(depApexModule.apexModuleBase().apexInfos, am.apexModuleBase().apexInfos) &&
(depApexModule.UniqueApexVariations() ||
depApexModule.apexModuleBase().ApexProperties.UniqueApexVariationsForDeps) {
am.apexModuleBase().ApexProperties.UniqueApexVariationsForDeps = true
}
}
})
}
// UpdateDirectlyInAnyApex uses the final module to store if any variant of this module is directly
// in any APEX, and then copies the final value to all the modules. It also copies the
// DirectlyInAnyApex value to any direct dependencies with a CopyDirectlyInAnyApexTag dependency
// tag.
func UpdateDirectlyInAnyApex(mctx BottomUpMutatorContext, am ApexModule) {
base := am.apexModuleBase()
// Copy DirectlyInAnyApex and InAnyApex from any direct dependencies with a
// CopyDirectlyInAnyApexTag dependency tag.
mctx.VisitDirectDeps(func(dep Module) {
if _, ok := mctx.OtherModuleDependencyTag(dep).(CopyDirectlyInAnyApexTag); ok {
depBase := dep.(ApexModule).apexModuleBase()
base.ApexProperties.DirectlyInAnyApex = depBase.ApexProperties.DirectlyInAnyApex
base.ApexProperties.InAnyApex = depBase.ApexProperties.InAnyApex
}
})
if base.ApexProperties.DirectlyInAnyApex {
// Variants of a module are always visited sequentially in order, so it is safe to
// write to another variant of this module. For a BottomUpMutator the
// PrimaryModule() is visited first and FinalModule() is visited last.
mctx.FinalModule().(ApexModule).apexModuleBase().ApexProperties.AnyVariantDirectlyInAnyApex = true
}
// If this is the FinalModule (last visited module) copy
// AnyVariantDirectlyInAnyApex to all the other variants
if am == mctx.FinalModule().(ApexModule) {
mctx.VisitAllModuleVariants(func(variant Module) {
variant.(ApexModule).apexModuleBase().ApexProperties.AnyVariantDirectlyInAnyApex =
base.ApexProperties.AnyVariantDirectlyInAnyApex
})
}
}
// ApexMembership tells how a module became part of an APEX.
type ApexMembership int
const (
notInApex ApexMembership = 0
indirectlyInApex = iota
directlyInApex
)
// ApexContents gives an information about member modules of an apexBundle. Each apexBundle has an
// apexContents, and modules in that apex have a provider containing the apexContents of each
// apexBundle they are part of.
type ApexContents struct {
Export dex implementation jars from prebuilt_apex Dexpreopt and boot jars package check all require access to dex implementation jars created for java_library and java_sdk_library. They were available when building from source but not when building from prebuilts, even though they are embedded within the .apex files that are referenced from prebuilt_apex. This changes adds support to prebuilt_apex to export the dex implementation jars and updates java_import to use those exported dex implementation jars. In a source build dexpreopt/boot jars package check access the apex (or platform) specific variant of a java_library, e.g. core-oj, from which it retrieves the dex implementation jar path. After this change in a prebuilt build dexpreopt/boot jars package check behave in the same way except in this case they retrieve the dex implementation jar path from the apex (or platform) specific variant of the java_import, e.g. core-oj. The work to export files from a `.apex` file for use by other modules is performed by a new `deapexer` module type. It is not used directly in an `Android.bp` file but instead is created implicitly by `prebuilt_apex`, In order to do that this contains the following changes: * Adds a new `dexapexer` module type to handle the exporting of files from the `.apex` file. * Adds an exported_java_libs property to prebuilt_apex to specify the set of libraries whose dex implementation jars need exporting. * Creates apex specific variants of the libraries listed in the exported_java_libs property. * Adds the set of exported files to the ApexInfo to make them available to the apex specific variants. * Prevents the prebuilt_apex variants from being merged together as they will not be compatible. * Modifies java_import to use the exported file for variants of a prebuilt_apex. * Adds a ninja rule to unpack (using deapexer) the contents of the prebuilt_apex's apex file, verify that the required files are present and make them available as outputs for other rules to use. * Some minor refactorings to support these changes. * Adds tests to cover prebuilt only, prebuilt with source preferred, and prebuilt preferred with source. Test: m nothing Bug: 171061220 Change-Id: Ic9bed81fb65b92f0d59f64c0bce168a9ed44cfac
2020-11-03 01:32:38 +08:00
// map from a module name to its membership in this apexBundle
contents map[string]ApexMembership
}
Export dex implementation jars from prebuilt_apex Dexpreopt and boot jars package check all require access to dex implementation jars created for java_library and java_sdk_library. They were available when building from source but not when building from prebuilts, even though they are embedded within the .apex files that are referenced from prebuilt_apex. This changes adds support to prebuilt_apex to export the dex implementation jars and updates java_import to use those exported dex implementation jars. In a source build dexpreopt/boot jars package check access the apex (or platform) specific variant of a java_library, e.g. core-oj, from which it retrieves the dex implementation jar path. After this change in a prebuilt build dexpreopt/boot jars package check behave in the same way except in this case they retrieve the dex implementation jar path from the apex (or platform) specific variant of the java_import, e.g. core-oj. The work to export files from a `.apex` file for use by other modules is performed by a new `deapexer` module type. It is not used directly in an `Android.bp` file but instead is created implicitly by `prebuilt_apex`, In order to do that this contains the following changes: * Adds a new `dexapexer` module type to handle the exporting of files from the `.apex` file. * Adds an exported_java_libs property to prebuilt_apex to specify the set of libraries whose dex implementation jars need exporting. * Creates apex specific variants of the libraries listed in the exported_java_libs property. * Adds the set of exported files to the ApexInfo to make them available to the apex specific variants. * Prevents the prebuilt_apex variants from being merged together as they will not be compatible. * Modifies java_import to use the exported file for variants of a prebuilt_apex. * Adds a ninja rule to unpack (using deapexer) the contents of the prebuilt_apex's apex file, verify that the required files are present and make them available as outputs for other rules to use. * Some minor refactorings to support these changes. * Adds tests to cover prebuilt only, prebuilt with source preferred, and prebuilt preferred with source. Test: m nothing Bug: 171061220 Change-Id: Ic9bed81fb65b92f0d59f64c0bce168a9ed44cfac
2020-11-03 01:32:38 +08:00
// NewApexContents creates and initializes an ApexContents that is suitable
// for use with an apex module.
// * contents is a map from a module name to information about its membership within
// the apex.
func NewApexContents(contents map[string]ApexMembership) *ApexContents {
return &ApexContents{
contents: contents,
Reland: Deduplicate APEX variants that would build identically APEX variants that share the same SDK version and updatability almost always use identical command line arguments to build but with different intermediates directories. This causes unnecessary build time and disk space for duplicated work. Deduplicate APEX variants that would build identically. Create aliases from the per-APEX variations to the new shared variations so that the APEX modules can continue to depend on them via the APEX name as the variation. This has one significant change in behavior. Before this change, if an APEX had two libraries in its direct dependencies and one of those libraries depended on the other, and the second library had stubs, then the first library would depend on the implementation of the second library and not the stubs. After this change, if the first library is also present in a second APEX but the second library is not, then the common variant shared between the two APEXes would use the stubs, not the implementation. In a correctly configured set of build rules this change will be irrelevant, because if the compilation worked for the second APEX using stubs then it will work for the common variant using stubs. However, if an incorrect change to the build rules is made this could lead to confusing errors, as a previously-working common variant could suddenly stop building when a module is added to a new APEX without its dependencies that require implementation APIs to compile. This change reduces the number of modules in an AOSP arm64-userdebug build by 3% (52242 to 50586), reduces the number of variants of the libcutils module from 74 to 53, and reduces the number of variants of the massive libart[d] modules from 44 to 32. This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix to always mark permissions XML files of java_sdk_library modules as unique per apex since they contain the APEX filename, and a fix to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes instead of DepIsInSameApex to check if two modules are in the same apex to account for a module that depends on another in a way that doesn't normally include the dependency in the APEX (e.g. a libs property), but the dependency is directly included in the APEX. Bug: 164216768 Test: go test ./build/soong/apex/... Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-12 03:17:01 +08:00
}
}
// Updates an existing membership by adding a new direct (or indirect) membership
func (i ApexMembership) Add(direct bool) ApexMembership {
if direct || i == directlyInApex {
return directlyInApex
}
return indirectlyInApex
}
// Merges two membership into one. Merging is needed because a module can be a part of an apexBundle
// in many different paths. For example, it could be dependend on by the apexBundle directly, but at
// the same time, there might be an indirect dependency to the module. In that case, the more
// specific dependency (the direct one) is chosen.
func (i ApexMembership) merge(other ApexMembership) ApexMembership {
if other == directlyInApex || i == directlyInApex {
return directlyInApex
}
if other == indirectlyInApex || i == indirectlyInApex {
return indirectlyInApex
}
return notInApex
}
// Tests whether a module named moduleName is directly included in the apexBundle where this
// ApexContents is tagged.
func (ac *ApexContents) DirectlyInApex(moduleName string) bool {
return ac.contents[moduleName] == directlyInApex
}
// Tests whether a module named moduleName is included in the apexBundle where this ApexContent is
// tagged.
func (ac *ApexContents) InApex(moduleName string) bool {
return ac.contents[moduleName] != notInApex
}
// Tests whether a module named moduleName is directly depended on by all APEXes in an ApexInfo.
func DirectlyInAllApexes(apexInfo ApexInfo, moduleName string) bool {
for _, contents := range apexInfo.ApexContents {
if !contents.DirectlyInApex(moduleName) {
return false
}
}
return true
}
////////////////////////////////////////////////////////////////////////////////////////////////////
//Below are routines for extra safety checks.
//
// BuildDepsInfoLists is to flatten the dependency graph for an apexBundle into a text file
// (actually two in slightly different formats). The files are mostly for debugging, for example to
// see why a certain module is included in an APEX via which dependency path.
//
// CheckMinSdkVersion is to make sure that all modules in an apexBundle satisfy the min_sdk_version
// requirement of the apexBundle.
// A dependency info for a single ApexModule, either direct or transitive.
type ApexModuleDepInfo struct {
// Name of the dependency
To string
// List of dependencies To belongs to. Includes APEX itself, if a direct dependency.
From []string
// Whether the dependency belongs to the final compiled APEX.
IsExternal bool
// min_sdk_version of the ApexModule
MinSdkVersion string
}
// A map of a dependency name to its ApexModuleDepInfo
type DepNameToDepInfoMap map[string]ApexModuleDepInfo
type ApexBundleDepsInfo struct {
flatListPath OutputPath
fullListPath OutputPath
}
type ApexBundleDepsInfoIntf interface {
Updatable() bool
FlatListPath() Path
FullListPath() Path
}
func (d *ApexBundleDepsInfo) FlatListPath() Path {
return d.flatListPath
}
func (d *ApexBundleDepsInfo) FullListPath() Path {
return d.fullListPath
}
// Generate two module out files:
// 1. FullList with transitive deps and their parents in the dep graph
// 2. FlatList with a flat list of transitive deps
// In both cases transitive deps of external deps are not included. Neither are deps that are only
// available to APEXes; they are developed with updatability in mind and don't need manual approval.
func (d *ApexBundleDepsInfo) BuildDepsInfoLists(ctx ModuleContext, minSdkVersion string, depInfos DepNameToDepInfoMap) {
var fullContent strings.Builder
var flatContent strings.Builder
fmt.Fprintf(&fullContent, "%s(minSdkVersion:%s):\n", ctx.ModuleName(), minSdkVersion)
for _, key := range FirstUniqueStrings(SortedStringKeys(depInfos)) {
info := depInfos[key]
toName := fmt.Sprintf("%s(minSdkVersion:%s)", info.To, info.MinSdkVersion)
if info.IsExternal {
toName = toName + " (external)"
}
fmt.Fprintf(&fullContent, " %s <- %s\n", toName, strings.Join(SortedUniqueStrings(info.From), ", "))
fmt.Fprintf(&flatContent, "%s\n", toName)
}
d.fullListPath = PathForModuleOut(ctx, "depsinfo", "fulllist.txt").OutputPath
WriteFileRule(ctx, d.fullListPath, fullContent.String())
d.flatListPath = PathForModuleOut(ctx, "depsinfo", "flatlist.txt").OutputPath
WriteFileRule(ctx, d.flatListPath, flatContent.String())
ctx.Phony(fmt.Sprintf("%s-depsinfo", ctx.ModuleName()), d.fullListPath, d.flatListPath)
}
// TODO(b/158059172): remove minSdkVersion allowlist
var minSdkVersionAllowlist = func(apiMap map[string]int) map[string]ApiLevel {
list := make(map[string]ApiLevel, len(apiMap))
for name, finalApiInt := range apiMap {
list[name] = uncheckedFinalApiLevel(finalApiInt)
}
return list
}(map[string]int{
"adbd": 30,
"android.net.ipsec.ike": 30,
"apache-commons-compress": 29,
"bouncycastle_ike_digests": 30,
"brotli-java": 29,
"captiveportal-lib": 28,
"flatbuffer_headers": 30,
"framework-permission": 30,
"gemmlowp_headers": 30,
"ike-internals": 30,
"kotlinx-coroutines-android": 28,
"kotlinx-coroutines-core": 28,
"libadb_crypto": 30,
"libadb_pairing_auth": 30,
"libadb_pairing_connection": 30,
"libadb_pairing_server": 30,
"libadb_protos": 30,
"libadb_tls_connection": 30,
"libadbconnection_client": 30,
"libadbconnection_server": 30,
"libadbd_core": 30,
"libadbd_services": 30,
"libadbd": 30,
"libapp_processes_protos_lite": 30,
"libasyncio": 30,
"libbrotli": 30,
"libbuildversion": 30,
"libcrypto_static": 30,
"libcrypto_utils": 30,
"libdiagnose_usb": 30,
"libeigen": 30,
"liblz4": 30,
"libmdnssd": 30,
"libneuralnetworks_common": 30,
"libneuralnetworks_headers": 30,
"libneuralnetworks": 30,
"libprocpartition": 30,
"libprotobuf-java-lite": 30,
"libprotoutil": 30,
"libqemu_pipe": 30,
"libsync": 30,
"libtextclassifier_hash_headers": 30,
"libtextclassifier_hash_static": 30,
"libtflite_kernel_utils": 30,
"libwatchdog": 29,
"libzstd": 30,
"metrics-constants-protos": 28,
"net-utils-framework-common": 29,
"permissioncontroller-statsd": 28,
"philox_random_headers": 30,
"philox_random": 30,
"service-permission": 30,
"tensorflow_headers": 30,
"xz-java": 29,
})
// Function called while walking an APEX's payload dependencies.
//
// Return true if the `to` module should be visited, false otherwise.
type PayloadDepsCallback func(ctx ModuleContext, from blueprint.Module, to ApexModule, externalDep bool) bool
// UpdatableModule represents updatable APEX/APK
type UpdatableModule interface {
Module
WalkPayloadDeps(ctx ModuleContext, do PayloadDepsCallback)
}
// CheckMinSdkVersion checks if every dependency of an updatable module sets min_sdk_version
// accordingly
func CheckMinSdkVersion(m UpdatableModule, ctx ModuleContext, minSdkVersion ApiLevel) {
// do not enforce min_sdk_version for host
if ctx.Host() {
return
}
// do not enforce for coverage build
if ctx.Config().IsEnvTrue("EMMA_INSTRUMENT") || ctx.DeviceConfig().NativeCoverageEnabled() || ctx.DeviceConfig().ClangCoverageEnabled() {
return
}
// do not enforce deps.min_sdk_version if APEX/APK doesn't set min_sdk_version
if minSdkVersion.IsNone() {
return
}
m.WalkPayloadDeps(ctx, func(ctx ModuleContext, from blueprint.Module, to ApexModule, externalDep bool) bool {
if externalDep {
// external deps are outside the payload boundary, which is "stable"
// interface. We don't have to check min_sdk_version for external
// dependencies.
return false
}
if am, ok := from.(DepIsInSameApex); ok && !am.DepIsInSameApex(ctx, to) {
return false
}
if err := to.ShouldSupportSdkVersion(ctx, minSdkVersion); err != nil {
toName := ctx.OtherModuleName(to)
if ver, ok := minSdkVersionAllowlist[toName]; !ok || ver.GreaterThan(minSdkVersion) {
Friendly error message on apex_available and min_sdk_version checks 1) suggest a fix at the end of the message 2) add new lines around the dependency path so that they are visually separated from rest of the error message Bug: N/A Test: m with an intentional break error: bionic/apex/Android.bp:32:1: module "com.android.runtime" variant "android_common_com.android.runtime_image": "libutils_headers" requires "libsystem_headers" that doesn't list the APEX under 'apex_available'. Dependency path: via tag apex.dependencyTag: { name:executable payload:true} -> crash_dump{os:android,image:,arch:arm_armv8-a,sdk:,apex:apex10000} via tag cc.libraryDependencyTag: { Kind:staticLibraryDependency Order:normalLibraryDependency wholeStatic:false reexportFlags:false explicitlyVersioned:false dataLib:false ndk:false staticUnwinder:false makeSuffix: skipApexAllowedDependenciesCheck:false excludeInApex:false} -> libtombstoned_client_static{os:android,image:,arch:arm_armv8-a,sdk:,link:static,apex:apex10000} via tag cc.libraryDependencyTag: { Kind:staticLibraryDependency Order:normalLibraryDependency wholeStatic:true reexportFlags:true explicitlyVersioned:false dataLib:false ndk:false staticUnwinder:false makeSuffix: skipApexAllowedDependenciesCheck:false excludeInApex:false} -> libcutils{os:android,image:,arch:arm_armv8-a,sdk:,link:static,asan:,apex:apex10000} via tag cc.libraryDependencyTag: { Kind:headerLibraryDependency Order:normalLibraryDependency wholeStatic:false reexportFlags:false explicitlyVersioned:false dataLib:false ndk:false staticUnwinder:false makeSuffix: skipApexAllowedDependenciesCheck:false excludeInApex:false} -> libutils_headers{os:android,image:,arch:arm_armv8-a,sdk:,asan:,apex:apex10000} via tag cc.libraryDependencyTag: { Kind:headerLibraryDependency Order:normalLibraryDependency wholeStatic:false reexportFlags:true explicitlyVersioned:false dataLib:false ndk:false staticUnwinder:false makeSuffix: skipApexAllowedDependenciesCheck:false excludeInApex:false} -> libsystem_headers{os:android,image:,arch:arm_armv8-a,sdk:,asan:,apex:apex10000} Consider adding "com.android.runtime" to 'apex_available' property of "libsystem_headers" Change-Id: I09f92c3086ea433780133a33ba0ad73baee6dc41
2021-03-04 12:03:10 +08:00
ctx.OtherModuleErrorf(to, "should support min_sdk_version(%v) for %q: %v."+
"\n\nDependency path: %s\n\n"+
"Consider adding 'min_sdk_version: %q' to %q",
minSdkVersion, ctx.ModuleName(), err.Error(),
ctx.GetPathString(false),
minSdkVersion, ctx.ModuleName())
return false
}
}
return true
})
}