Previously, when building from prebuilts boot no rules were created to
produce the boot image files for the host, i.e. the OS on which the
build was running. That caused problems with checkbuilds. No rules were
produced as there was no host variant of a prebuilt apex to provide
them.
This change restructures the code to allow the prebuilt bootclasspath
fragment to build the host variants of the files from the dex files
provided by the prebuilt APEX. The generated files will not be the same
as they would be if built from source as there is no boot image profile
to use but it should be sufficient to satisfy the checkbuild target and
allow for host side testing.
Bug: 192575099
Test: m SOONG_CONFIG_art_module_source_build=false droid dist checkbuild
Merged-In: I6af00f19bb71aa18dd462f5eac6aa38e3e721023
Change-Id: I6af00f19bb71aa18dd462f5eac6aa38e3e721023
(cherry picked from commit a56be7d7815ad164cdd854f8fc6b1cbc3bbf1918)
Previously, boot image generation used GetCachedGlobalSoongConfig(ctx)
in order to get access to the GlobalSoongConfig. That required that
some other part of the code had called GetGlobalSoongConfig(ctx) to
initialize the cached value. That was left over from when the boot
image generation was done in a singleton which could not call
GetGlobalSoongConfig(ctx) directly. That is no longer true.
This change switches the uses of GetCachedGlobalSoongConfig(ctx) to
GetGlobalSoongConfig(ctx) and removes the now unnecessary call to
GetGlobalSoongConfig(ctx) from outside the functions.
Bug: 192575099
Test: m nothing
Merged-In: I34b7b1526d072d8b09fda7caa96e381695888e16
Change-Id: I34b7b1526d072d8b09fda7caa96e381695888e16
(cherry picked from commit 8fc51a8eb6ca20dbb4b65822123201568bb297f1)
Previously, the boot zip file, containing all the boot image files for
all the supported architectures, was only created from source. It was
not created when building from a prebuilt_bootclasspath_fragment. That
lead to build failures when building from ART prebuilts.
This change pulls the boot zip file creation out so that it can be done
for both source and prebuilt bootclasspath_fragment modules as well as
for the platform_bootclasspath module.
Bug: 192575099
Test: m out/target/product/generic_arm64/boot.zip
m SOONG_CONFIG_art_module_source_build=false out/target/product/generic_arm64/boot.zip
- Compare the output of the first command from before the change
with the output from them both after and confirm that when the
ART prebuilts are up to date with the source that there are no
differences.
Merged-In: Ie7dd5e2ca4a865d06fd9ebf87320cf68c4d05bc3
Change-Id: Ie7dd5e2ca4a865d06fd9ebf87320cf68c4d05bc3
(cherry picked from commit 56afb27fb099cb79c1537c661628db1776f1fcc3)
Bug: 158843648
Test: check if dexpreopt.config for the module defined in mk file has
DexPreoptImageLocationsOnDevice field
Change-Id: Idb2de398871ff114245393a9dd92b5a1b5c942e7
Merged-In: Idb2de398871ff114245393a9dd92b5a1b5c942e7
(cherry picked from commit e3165c8de302839c8ab15f53d8024b585c62061f)
Previously, the prebuilt art-bootclasspath-fragment did not provide any
boot image files. That meant it was impossible to build any module that
requires access to those files from prebuilts, e.g. any module that
needs to be dexpreopt-ed.
This change enables that module to retrieve those files from the
prebuilt_apex.
Bug: 177892522
Bug: 189298093
Test: m nothing
m droid SOONG_CONFIG_art_module_source_build=false SKIP_BOOT_JARS_CHECK=true
- the previous command does not work but this change does fix one
of the issues reported.
Merged-In: I1d4d9545172d79282918130df6b9aa55471bffc1
Change-Id: I1d4d9545172d79282918130df6b9aa55471bffc1
(cherry picked from commit 5466a3699c550d1fa67066db48951b765156be4f)
Previously, copyBootJarsToPredefinedLocations relied on all its
parameters having the same length and the same order. That made it
quite fragile as changes to one of the parameters without corresponding
changes to the other would cause failures. It also combined the
retrieval of the boot dex jars from the modules, handling of missing
boot dex jar files and the generation of the rules to copy the files.
This change separates the retrieval of boot dex jars and handling of
missing files from the copying of those files while at the same time
making the function less fragile by replacing the three ordered
parameters with two maps that shared common keys.
Bug: 179354495
Test: m nothing
Merged-In: Idbcd24a7e8af89f7895a20aeddc58502dcbaad03
Change-Id: Idbcd24a7e8af89f7895a20aeddc58502dcbaad03
(cherry picked from commit 5f148ca7cf2e9a9478922577b7ed70b1caacb55e)
Previously, when building a framework boot image variant it added an
implicit dependency onto the first file in the primary boot image
variant to ensure that the primary boot image variant files that the
dex2oat command needs have been created. That works when generating
from source as in that case all the files for a boot image variant are
created by a single command. However, it does not work for prebuilts as
each prebuilt file will be copied into the required location by
separate copy commands.
This change adds all the files that the dex2oat command uses implicitly
when building an extension boot image as implicit dependencies.
Bug: 177892522
Test: m SOONG_CONFIG_art_module_source_build=false droid
- the previous command only works in combination with a number of
other build changes.
Merged-In: I183748fd17f8f3003890675b8c6bb9fcab331443
Change-Id: I183748fd17f8f3003890675b8c6bb9fcab331443
(cherry picked from commit bff50e2b803ede32c9b17b6b7bf3054424703bf5)
oatdumpd isn't available as a prebuilt.
Cherry-picked from https://r.android.com/1711291.
Test: m SOONG_CONFIG_art_module_source_build=false droid
Bug: 172480615
Change-Id: I70317eb8f253272d629f23063cbe265d556caad3
Merged-In: I70317eb8f253272d629f23063cbe265d556caad3
The AlwaysUsePrebuiltSdks() causes all java_sdk_library_import modules
to be preferred over the source, i.e. as if they had prefer: true set.
That interacts badly with the work that is being done to integrate the
bootclasspath_fragment/platform_bootclasspath modules into the build.
It would work fine once that integration has been completed but in the
interim it causes problems. e.g. it does not cause a problem in AOSP
because those java_sdk_library_import modules that are affected have
already been integrated into the build properly.
Unfortunately, internally that is not the case because there are
java_sdk_library/java_sdk_library_import modules that still need to
be updated.
Before the java_sdk_library_import can be safely preferred each
java_sdk_library/java_sdk_library_import module that contributes to the
bootclasspath must:
* Be in the contents of matching bootclasspath_fragment and
prebuilt_bootclasspath_fragment modules.
* Have an apex and one of a prebuilt_apex/apex_set that contains the
dex implementation jar and lists the prebuilt_bootclasspath_fragment
name in its exported_bootclasspath_fragments property.
Safely preferred in this context means that the whole build will
continue to work rather than the current situation which is that only
some of the build will work and some will fail if an attempt is
actually made to build it.
Unfortunately, many java_sdk_library_import modules are missing:
* The prebuilt_bootclasspath_fragment.
* The exported_bootclasspath_fragments property on the
prebuilt_apex/apex_set that contains them.
Together these cause the following symptoms:
1. The java_sdk_library_import does not have a dex implementation jar.
2. The java_sdk_library_import does not have a myapex variant.
These workarounds will avoid Soong reporting build failures. However,
the build will still fail if an attempt is made to build anything
produced by the platform-bootclasspath, e.g. hidden API processing or
a system image.
Bug: 188505921
Bug: 179354495
Test: m TARGET_BUILD_APPS=Calendar
Change-Id: I3226e21cd6a7f9e4d6bbe94e54129ac5e1d4c679
For running dex2oat on the target_files, the paths should be use the
device install path instead of the path starting with $(OUT).
So add usesTargetFiles option and basePath option which indicates
extracted path. With those options, the path is replaced with
$(basePath)/$(device path)
And also, add DexPreoptImageDeviceLocations in the config which refers
to the boot image path(without arch) on the device. Because
DexPreoptImage related device path was missing.
Bug: 158843648
Test: dexpreopt_gen -usesTargetFiles -basePath (extract path) and then
check if paths in the generated shell script are based on on-device
path.
Change-Id: I9667fadbf3b7c6f770e0d1bcbee5d67c1ecd8a3d
Merged-In: I9667fadbf3b7c6f770e0d1bcbee5d67c1ecd8a3d
(cherry picked from commit 4dda75e73e3e52e11b1cd37af33645fcfe5ed980)
Moves the building of boot images from the dexpreopt_bootjars singleton
to the bootclasspath_fragment and platform_bootclasspath.
The art boot image is generated by the art-bootclasspath-fragment
module and the framework boot image by the platform-bootclasspath
module.
This does temporarly duplicate the generation of an identical boot
profile for each image. As part of the work to modularize the boot
image profile each image will have its own custom default boot profile.
Bug: 177892522
Bug: 186455808
Test: m droid and TreeHugger
Change-Id: I23cf05ec7648749b21c7cf6fcba282b46649a981
Previously, the initialization of the dexpreoptBootJars fields used to
store information needed in its MakeVars method was interleaved with
the calls to buildBootImage(). In fact those fields were initialized
from the *bootImageConfig returned by buildBootImage(). However, the
method simply returned the *bootImageConfig value that was passed in.
Separating the initialization from the calls to buildBootImage() allows
the functionality to be moved to platform_bootclasspath separately.
Bug: 177892522
Test: m nothing
Change-Id: I8ba460f60be553516de5f186e14377bb0ec1b2cd
The art dex files are copied in the bootclasspath_fragment and the
non-updatable and updatable dex files are copied in the
platform_bootclasspath.
Bug: 177892522
Test: m nothing
Change-Id: I5d3d533d1a7a9f8e7ae20c12eb33029a898a2cd6
Add 'host' into the name of fields regarding path on the host side to
distinguish between paths on the device(which will be added in the
following commit), and paths on the host.
Bug: 158843648
Test: build and flash, and then
adb wait-for-device \
&& adb -s $S root \
&& adb -s $S logcat \
| grep -E 'ClassLoaderContext [a-z ]+ mismatch' -C 1
Change-Id: Ib2645ed51591ba2f4b726c115b401ad2bd6675da
Previous refactorings of the code for creating these two files caused
them to be created before the boot.prof file instead of afterwards. As
a result even though they were appended to the list of files to install
they were discarded when the boot.prof file set the list rather than
appended to it. That worked before the refactoring because the
boot.prof file set the list first before the others were appended to
it.
This change appends the boot.prof file to the list making the order in
which they are added irrelevant.
Bug: 187494247
Test: m nothing
- check DEXPREOPT_IMAGE_PROFILE_BUILT_INSTALLED in generated
make_vars file to ensure it includes all 3 files.
Change-Id: Idb94531daf61b1b047c72eb5c67a57d3fdced05c
The main difference between the dexpreopt_bootjars singleton and the
platform_bootclasspath singleton module is the way they find the
modules to use. The former searches all modules, the latter adds
dependencies on the modules that they need. This change separates the
finding of the modules from the copying of the boot jars for those
modules to make it easier to move the remaining functionality to
platform_bootclasspath.
This temporarily creates a singleton specific copy of the hidden API
function isModuleInConfiguredList() to select the modules in place of
the logic in the getBootJar() method. There is a slight loss of context
information from the error messages but as these methods will be
removed once the boot image creation has moved this is not an issue.
While switching the isModuleInConfiguredListForSingleton() to use the
SingletonContext the error message was fixed to include the name of
the module with the issue.
Bug: 177892522
Test: m nothing
Change-Id: Iaea906da95d9da5301fb964fc593890f2216d336
The getBootImageJar function will be removed once the boot image
creation has been moved to the platform_bootclasspath and
bootclasspath_fragment module types. However, the consistency checks
that it performs are still useful so this change moves them out
first.
The ART boot image related checks are now performed in the
bootclasspath_fragment module type. A previous change accidentally
disabled the checks when the contents property was not empty which has
been fixed. Also, the error messages have been tweaked to make it clear
that the art-bootclasspath-fragment is now the source of truth as to
its contents not the configuration.
The framework boot image related checks are now performed in the
platform_bootclasspath module type.
Initially, this change included an extra check to make sure that
UpdatableBootJars comes from updatable APEXes but that broke because
framework-wifi and framework-tethering are not currently marked as
updatable in AOSP.
Bug: 177892522
Test: m nothing
Change-Id: I80fb600fa2c7cec4566b3461c6a33c4c6f0743f4
Bug: 177892522
Test: m oat-dump-boot
- test output to make sure that this change does not change the
generated files, at least no more than no changes do as the
output from this rule is not deterministic. See b/186459873.
Change-Id: Ib2b4203d9bb1fd0ee9443aee4e58b54b38b491cf
Changes generateUpdatableBcpPackagesRule to use ModuleContext instead
of SingletonContext and moves the call from dexpreoptBootJar's
GenerateSingletonBuildActions method to platform_bootclasspath's
GenerateAndroidBuildActions.
Bug: 177892522
Test: lunch art_module_arm64
m out/soong/module_arm64/dex_bootjars/updatable-bcp-packages.txt
- make sure it is not affected by this change
Change-Id: I9741ae1eb9573cafe12dd7a17dc1d8449b224dc8
This simplifies the process of moving the updatableBcpPackagesRule to
the platform_bootclasspath by separating the gathering of the list of
updatable modules (which differs between the singleton and the
platform_bootclasspath module) from the gathering of the permitted
packages list and generation of the rule which are generally common.
Bug: 177892522
Test: lunch art_module_arm64
m out/soong/module_arm64/dex_bootjars/updatable-bcp-packages.txt
- make sure it is not affected by this change
Change-Id: I3cb64310f618059758a32cfe00d3745d52388e49
Previously, the method was run in repo manifests that did not include
the frameworks/base repository so it had to handle the file being
missing. However, now that this is being called from
platform_bootclasspath which is only defined in frameworks/base that
is no longer required.
Bug: 177892522
Test: m droid
Change-Id: I77fa5a204d1426a6be24a6f0b48e18f48f3dd908
Changes bootFrameworkProfileRule to use ModuleContext instead of
SingletonContext and moves the call from dexpreoptBootJar's
GenerateSingletonBuildActions method to platform_bootclasspath's
GenerateAndroidBuildActions.
Changing the context also allows the code to switch from
bootFrameworkProfileRule to GetGlobalSoongConfig.
Also extracts the shouldBuildBootImages function so it can be used by
platform_bootclasspath to preserve the existing behavior.
Bug: 177892522
Test: m droid
Change-Id: I30d3ca10be7f84348ad3aa9cc984dd15b8f6f4e9
Previously, buildBootImage had to be called for the default boot image
config before the art boot image config as it created some rules which
were specific to default boot image. The creation of these rules was
encapsulated in Config().Once(..) to ensure that they were only ever
created once.
This change extracts that functionality out of buildBootImage to the
calling method which means that they are only called once and so no
longer need protecting against being called again.
Bug: 177892522
Test: m droid
Change-Id: I464477de1a08df15e577873a9accf7db2bc088d1
Previously, the buildBootImage function maintained a list of
missingDeps that was returned from findAndCopyBootJars and was passed
to various other methods that created ninja rules that depended on the
files created by findAndCopyBootJars. The purpose of that list was
basically to record that findAndCopyBootJars could not find some files
so that the subsequent ninja rules could be marked as being in error.
This changes avoids having to explicitly propagate the missing deps to
each rule by instead having findAndCopyBootJars generate an error rule
instead of a Cp rule for any missing file. That means that any attempt
to build any rule that depends on the missing file will fail when the
missing file is created.
Bug: 177892522
Test: Add an unknown module to PRODUCT_BOOT_JARS
lunch art_module_arm64
- Run each of the following targets and made sure it failed with a
message indicating that the unknown module was missing.
m SOONG_ALLOW_MISSING_DEPENDENCIES=true out/soong/module_arm64/dex_bootjars/boot.prof
m SOONG_ALLOW_MISSING_DEPENDENCIES=true out/soong/generic_arm64/dex_bootjars/boot.bprof
m SOONG_ALLOW_MISSING_DEPENDENCIES=true out/soong/generic_arm64/dex_bootjars/updatable-bcp-packages.txt
Change-Id: Ib7196c128419fbffdfeb61c1c5d47a3b08d1e132
When Soong looks for boot jars among all modules, it applies certain
constraints to each module that looks like a boot jar (e.g. that it
comes from the right apex or platform). Previously these constraints did
not handle boot jars on /system_ext correctly (they were handled like
apex jars, while they should be handled like platform jars).
Bug: 154976937
Test: m nothing (the modified Soong test would fail previously)
Change-Id: I0746a2fd276ab5ef0400340c5b61cf26c2570e5a
This CL handles updatable boot jars in the same hacky way as we handle
non-updatable boot jars: it creates a set of predefined paths to the dex
jars in a global config, then traverses all modules in a singleton
context, finds updatable boot jars and adds copy rules from these jars
to the predefined paths. A proper way would be to register dependencies
of the dexpreopted modules on the boot jars and extracting paths to dex
files by walking these dependencies.
Bug: 178467404
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Test: added new Soong test
Change-Id: I87f764109315f79315d73bf43799b70eb010fc0b
This CL adds no functional changes, it only splits a few functions into
parts that will be reused in a follow-up CL.
Bug: 178467404
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Change-Id: I2bcb3c03d51175130da29fdfaf6dd747e3aea964
Previously, it ignored any module that was not a platform apex variant
however that required every module that was referenced from the boot
jars to have a platform variant which is not the case when building
from prebuilts.
This change switches it to explicitly check that the variant is for
either the appropriate apex or the platform depending on what is
configured in the BootJars or UpdatableBootJars.
It partially duplicates some logic from the getBootImageJar() function.
It intentionally does not refactor the getBootImageJar() to allow for
reuse because coupling the hiddenapi and dexpreopt logic would make
refactoring either of them more difficult. Any duplicated code will be
deduped once they have both been refactored.
Bug: 178361284
Test: m droid
Change-Id: I4b4e0dc8ee40c1ba1713d7ef72df13d175e84af6
Previously, the apex module had to hard code behavior specific to the
art apex module in order to include the art boot image. This change
adds support to the apex module to allow the boot images to be
specified per apex.
In combination with a change to add the "art-boot-image" to the ART
apex this allows the custom code for handling the art boot image in
apex to be removed.
That custom apex code also included the logic to ensure that the
GlobalSoongConfig was initialized for use by the dex_bootjars
singleton. That logic has been moved from the APEX to the boot_image
module. That ensures that it will be run if and only if a boot_image
module is present in the checked out repos. So, limited manifest
checkouts which do not contain the art or frameworks/base repos (which
is where the boot_image modules are defined) will not attempt to run
this logic, which would fail because dex2oat would not be present.
Bug: 177892522
Test: m droid
Change-Id: I02d25fbef6e864e31eb5e0f4eb50358c79486db0
are present.
1) The boot jar to APEX mapping is maintained by the base names for
both of them. When building with prebuilt modules and APEXes, that
means we need to take care to compare them without regard to any
"prebuilt_" prefixes.
2) VisitAllModules can visit disabled modules and both source and
prebuilt modules, so they need some conditions to skip modules that
aren't applicable for boot jars.
Test: `m droid`
Test: `m droid SOONG_CONFIG_art_module_source_build=false`
with fresh ART Module prebuilts in place
Bug: 171061220
Change-Id: Iced269d29127bc8b8f9b3171adb60a97d115628b
Adds a boot_image module type. Follow on changes will add instances of
boot_image to represent the ART and framework boot images in the art
and frameworks/base repositories respectively.
For the moment they retrieve their associated boot image configuration
from the map returned by genBootImageConfigs() and leave the actual
work of creating the ninja rules to create the boot images to the
dex_bootjars singleton.
Bug: 177892522
Test: m droid
Change-Id: Ib69701de0d24d996bb8e8be7a20b941be907390e
The main difference between a singleton and a singleton module that
affects this change is that the latter requires a module to be defined
in order for the singleton to be instantiated. This change defines the
module in both the tests and the standard build.
Bug: 177892522
Test: m droid
Change-Id: Id6669835cf3a2ab7359fa53047eb9042992a9362
Bug: 176171716
Test: build_mainline_modules.sh and ensure that the ART boot image is
built: `find out -type f -name '*.art'` is nonempty for every arch.
Change-Id: Ibc45581eef2b205c750a30709780cf659ba7cfa1
Dexpreopt and boot jars package check all require access to dex
implementation jars created for java_library and java_sdk_library. They
were available when building from source but not when building from
prebuilts, even though they are embedded within the .apex files that
are referenced from prebuilt_apex.
This changes adds support to prebuilt_apex to export the dex
implementation jars and updates java_import to use those exported dex
implementation jars.
In a source build dexpreopt/boot jars package check access the apex (or
platform) specific variant of a java_library, e.g. core-oj, from which
it retrieves the dex implementation jar path.
After this change in a prebuilt build dexpreopt/boot jars package check
behave in the same way except in this case they retrieve the dex
implementation jar path from the apex (or platform) specific variant of
the java_import, e.g. core-oj.
The work to export files from a `.apex` file for use by other modules
is performed by a new `deapexer` module type. It is not used directly
in an `Android.bp` file but instead is created implicitly by
`prebuilt_apex`,
In order to do that this contains the following changes:
* Adds a new `dexapexer` module type to handle the exporting of files
from the `.apex` file.
* Adds an exported_java_libs property to prebuilt_apex to specify the
set of libraries whose dex implementation jars need exporting.
* Creates apex specific variants of the libraries listed in the
exported_java_libs property.
* Adds the set of exported files to the ApexInfo to make them available
to the apex specific variants.
* Prevents the prebuilt_apex variants from being merged together as
they will not be compatible.
* Modifies java_import to use the exported file for variants of a
prebuilt_apex.
* Adds a ninja rule to unpack (using deapexer) the contents of the
prebuilt_apex's apex file, verify that the required files are present
and make them available as outputs for other rules to use.
* Some minor refactorings to support these changes.
* Adds tests to cover prebuilt only, prebuilt with source preferred,
and prebuilt preferred with source.
Test: m nothing
Bug: 171061220
Change-Id: Ic9bed81fb65b92f0d59f64c0bce168a9ed44cfac
Previously, only the <jar> part of the boot image configuration pair of
<apex>:<jar> was used to select the module that provides the boot image
dex. The name was sufficient to select the module but not the module
variant and it relied on the platform variant not being installed to
select a unique variant. Unfortunately, when modules are provided as
prebuilts they are not installed and so they get ignored.
This change removes the install check and instead checks the <apex> as
well to select a unique variant.
It also moves the DexJarBuildPath() and ApexModule checks to after
checking the name and treats them as errors. It is safe to do so
because every module that has the same name has to have the same module
type and in order for the module to be valid in the boot image
configuration it must pass those two checks. Treating these as errors
rather than silently ignoring them makes it easier to diagnose problems
with the configuration and/or the boot image modules.
Test: m droid
Bug: 171061220
Change-Id: Ie51f097e081907ea5b75d75840736b10d8d883e8
This change does not break modules like "core-oj" that are in multiple
apexes, i.e. com.android.art, com.android.art.debug and
com.android.art.testing because they all shared the same variant. So,
the getBootImageJar() is only called once for that variant.
Test: m droid
Bug: 171061220
Change-Id: Ibf26da147af2b49ab9e4588030e8cd4002d04a7a
Adds some additional information into the paths that are created when
modules are missing in Soong but SOONG_ALLOW_MISSING_DEPENDENCIES=true.
Test: try and build platform against art prebuilts
Bug: 171061220
Change-Id: Ifbcc0af5bdbd15409758a3b6f216cf9b3b5dba31
Enable the RuleBuilder and RuleBuilderCommand methods to access
the BuilderContext by passing it to NewRuleBuilder instead of
RuleBuilder.Build.
Test: genrule_test.go
Test: rule_builder_test.go
Test: m checkbuild
Change-Id: I63e6597e19167393876dc2259d6f521363b7dabc
The android.WriteFile rule takes careful escaping to produce the
right contents. Wrap it in an android.WriteFileRule that handles
the escaping.
Test: compare all android.WriteFile outputs
Change-Id: If71a5843af47a37ca61714e1a1ebb32d08536c31
Paul Duffin