dependencies.
dexpreopt.RegisterToolDeps runs late after prebuilt dependencies have
been resolved, and there's special code in dex2oatPathFromDep to
resolve the prebuilt from the source module. However, if the source
module is disabled then the dependencies check in validateAndroidModule
will complain, so we need to disable that check in this particular
situation.
Also add a comment to explain why dexpreopt.RegisterToolDeps needs to
run so late.
Cherry-picked from https://r.android.com/1711292.
Test: m nothing
Bug: 145934348
Bug: 172480615
Change-Id: Ibc673303d0336768fa23261a2068e91a08f46a30
Merged-In: Ibc673303d0336768fa23261a2068e91a08f46a30
For running dex2oat on the target_files, the paths should be use the
device install path instead of the path starting with $(OUT).
So add usesTargetFiles option and basePath option which indicates
extracted path. With those options, the path is replaced with
$(basePath)/$(device path)
And also, add DexPreoptImageDeviceLocations in the config which refers
to the boot image path(without arch) on the device. Because
DexPreoptImage related device path was missing.
Bug: 158843648
Test: dexpreopt_gen -usesTargetFiles -basePath (extract path) and then
check if paths in the generated shell script are based on on-device
path.
Change-Id: I9667fadbf3b7c6f770e0d1bcbee5d67c1ecd8a3d
Merged-In: I9667fadbf3b7c6f770e0d1bcbee5d67c1ecd8a3d
(cherry picked from commit 4dda75e73e3e52e11b1cd37af33645fcfe5ed980)
Add 'host' into the name of fields regarding path on the host side to
distinguish between paths on the device(which will be added in the
following commit), and paths on the host.
Bug: 158843648
Test: build and flash, and then
adb wait-for-device \
&& adb -s $S root \
&& adb -s $S logcat \
| grep -E 'ClassLoaderContext [a-z ]+ mismatch' -C 1
Change-Id: Ib2645ed51591ba2f4b726c115b401ad2bd6675da
Consistent with plumbing of boot jars. SystemServerJars only support /system/framework/ jars.
Bug: 180105615, 155630745
Test: m && launch_cvd
Change-Id: I58b005b7c4103c8e250090e995b1d9b2f9ed4a76
This fixes a discrepancy between clean and incremental
builds exposed by
https://android-review.googlesource.com/1671710 .
Test: m
Bug: 148067697
Change-Id: I0de078045075417a4123f7c3821c723f1e8e6b68
Dedups the many repetitions of the code to obtain a *Prebuilt from a
module.
Bug: 177892522
Test: m nothing
Change-Id: I1ededbe9ee79e89ea6dd8882dfee4be0bf0b51b7
The field isn't used anymore.
And DexPreoptImagesDeps will have the slice of which size is the same as
Archs to avoid an error.
Bug: 158843648
Test: m
Change-Id: I520063ff7376811febbc82e1a0a43785feb5bbb2
Both out/soong/dexpreopt.config and out/soong/soong.variables contain
configuration for the BootJars and UpdatableBootJars that MUST be
identical. If they are not then it can lead to broken builds.
This change adds a consistency check for them that will make the issue
more obvious.
Bug: 186195980
Test: DIST_DIR=out/bionic-dist ./art/tools/dist_linux_bionic.sh -j80 com.android.art.host com.android.support.apexer --skip-soong-tests
Ran the previous command with and without the fix in
https://r.android.com/1684877. Without the fix the build reported
an inconsistency in the configuration, with that fix the build
passed.
Change-Id: I10fbe328ba4f1fbd9db4708409979e9824c196ef
toJsonClassLoaderContextRec's size is twice as large than expected. And
the initial values is filled with null value.
Bug: 158843648
Test: m nothing(testcase is added)
Change-Id: I24c23269bd16baa18481f34b85c543d41f26d0e0
1. Instead of 'slim' config, use full config even for libs
2. Define moduleJSONConfig for fields which cannot be converted to JSON
field directly(Path type field, ProfileBootListing,
DexPreoptImagesDeps are added in this CL) and exclude fields which is
convertible(DexPreoptImageLocations)
Bug: 158843648
Test: m dist
Change-Id: I3f9192ab5292bd079be1b686bb3b25735a836cbc
Dexpreopt uses AAPT to parse `targetSdkVersion` from the manifest, so it
the generated dexpreopt.sh script that calls AAPT must depend on it,
otherwise AAPT might not be built by the time it is used. Tool
dependencies are handled via the global Soong config and the
DEXPREOPT_GEN_DEPS variable that Soong generates for Make. This config
always uses Soong tool paths, like out/soong/host/linux-x86/bin/aapt
rather than out/host/linux-x86/bin/aapt.
This CL fixes a mistake in dexpreopt rule that used context-dependent
AAPT path (so, when called from Make, the dependency was on Soong tool,
but a Make tool was actually used, so it failed sometimes).
Bug: 132357300
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Change-Id: I1f0ab4afac98e6239f324e7f3571d670fd7a36cd
Creates a new deptag type for it so that it can implement the marker
interfaces that will exclude it from being added to the APEX and from
visibility enforcement. The latter is probably not an issue ATM because
the dependencies are added after visibility checks are enforced but
this code is undergoing lots of refactoring so that may change.
Bug: 177892522
Test: m nothing
Change-Id: Ibd167d557adec761a2e3eed78f4d334c40a04fb9
This CL handles updatable boot jars in the same hacky way as we handle
non-updatable boot jars: it creates a set of predefined paths to the dex
jars in a global config, then traverses all modules in a singleton
context, finds updatable boot jars and adds copy rules from these jars
to the predefined paths. A proper way would be to register dependencies
of the dexpreopted modules on the boot jars and extracting paths to dex
files by walking these dependencies.
Bug: 178467404
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Test: added new Soong test
Change-Id: I87f764109315f79315d73bf43799b70eb010fc0b
Make it easier to test dexpreopt functionality by grouping all the
fixture preparations together.
Bug: 177892522
Test: m nothing
Change-Id: I94f66e3ec82efc4fd791f4fdab678d298565e452
There are modules that would have passed verify_uses_library check, but
don't run it for some reason (the check gets enabled either with an
explicit setting, or if the module has nonempty <ues-library> lists in
the build properties). Previously all such modules were assumed to have
empty CLC, which is not always true. In particular, compatibility
libraries are ignored, which affected e.g. Calendar and messaging apps.
This CL gives such apps a chance to have correct CLC. The goal for the
future is to enforce verify_library_check by default.
Bug: 132357300
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Change-Id: Iea3be0fc9d7775c52950848b5a3fd3b7fcd36c53
Previously there were two different scripts that did similar things:
1) build/soong/scripts/manifest_check.py
2) build/make/core/verify_uses_libraries.sh
Both scripts extracted <uses-library> tags and `targetSdkVersion` from
the manifests of Java modules, but 1) worked for XML manifests, and 2)
worked for APKs. This CL reimplements the functionality from 2) in 1),
so that one script can handle both XML manifests and APKs.
Bug: 132357300
Test: lunch cf_x86_64_phone-userdebug && m && launch_cvd \
adb wait-for-device && adb root && adb logcat \
| grep -E 'ClassLoaderContext [a-z ]+ mismatch'
# empty grep output, no errors
Change-Id: Id1b66e4f3f30f307dba70cb111c7571762cb546a
The tests in dexpreopt are unit tests that don't require the use of
test fixtures but there are a number of tests in java and apex packages
that need to configure dexpreopt behavior so this changes adds support
for modifying the dexpreopt configuration in test fixtures.
Bug: 181070625
Test: m nothing
Change-Id: I7b64c9cf0a94cc548c27b55c5a7875be5ff0e18d
In cases when class loader context cannot be computed at build time and
verify_uses_libraries check is relaxed (RELAX_USES_LIBRARY_CHECK=true)
dexpreopt uses a special compiler filter that suppresses AOT-compilation
to native code and applies only those optimizations that do not require
class loader context. Previously the "extract" filter was used. Now ART
supports using "verify" filter in this configuration.
Bug: 132357300
Test: lunch cf_x86_64_phone-userdebug && m && launch_cvd \
adb wait-for-device && adb root && adb logcat \
| grep -E 'ClassLoaderContext [a-z ]+ mismatch'
# empty grep output, no errors
Change-Id: Id4933aa94ea96894278355283383c16d103e98c9
The new mode is enabled with environment variable
RELAX_USES_LIBRARY_CHECK. If the variable is set to true, then a
verify_uses_libraries check failure does not fail the build, instead it
sets a special compiler filter "extract" for dexpreopt, which means that
the DEX file will be extracted, but it won't be compiled to native code.
Class loader context will be set to empty in this case (not &, as it is
going to be deprecated soon).
If the variable RELAX_USES_LIBRARY_CHECK is unset or set to something
other than "true", then the old behaviour of the verify_uses_libraries
check is preserved.
The intended use case for this flag is to have a smoother migration path
for the Java modules that need to add <uses-library> information in
the build files. The flag allows to quickly silence build errors. This
flag should be used with caution and only as a temporary measure, as it
masks real errors and affects performance.
verify_uses_libraries check is reworked so that it writes the error
message to a status file (which is used instead of the former timestamp
file). Currently the stored error message is not used, but it may be
used later to produce a warning. Dexpreopt command checks if the status
file exists and is nonempty; if that is the case, then compiler filter
is set to "extract".
Bug: 132357300
Test: Manually add some mismatch between the libraries in the Android.bp
and Android.mk files for dexpreopted apps, build with
RELAX_USES_LIBRARY_CHECK=true and obsserve that the build doesn't
fail and they are compiled with compiler-filter "extract".
Unset RELAX_USES_LIBRARY_CHECK and observe that the build fails.
Change-Id: Ibb5d993a25b1df1d2e70b7d5aafc6997f9d64e67
Library order is important because it is used to construct class loader
context, which is then written into OAT/ODEX files and chacked against
class loader context constructed by PackageManager on the device. If the
orders are different, dexpreopted code is rejected.
Soong avoids using Go maps for class loader context representation
precisely for that reason. However, for the modules defined in makefiles
dexpreopt configs were serialized to JSON and unmarshaled to Go maps,
which resulted in wrong order of libraries. This CL changes Go
representation of class loader contexts imported from JSON and makes the
order stable.
Bug: 132357300
Test: lunch cf_x86_64_phone-userdebug && m && launch_cvd \
adb wait-for-device && adb root && adb logcat \
| grep -E 'ClassLoaderContext [a-z ]+ mismatch'
# empty grep output, no errors
Change-Id: I15f51617f9573c0bbcb324cf2592daf719cad586
This is needed for Java libraries that are <uses-library> dependencies
of Java libraries and apps defined as Make modules. Each dexpreopted
module in Make generates a dexpreopt.config file, which incorporates
information from its dependencies' dexpreopt.config files. For
dependencies that are Make modules their dexpreopt.config files are
generated by Make, and for Soong modules they are generated by Soong.
Since Soong doesn't know which libraries are used by Make, it generates
build rules for a superset of the necessary libraries.
Bug: 132357300
Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: I325b1037658736ee3c02450b08c00eca1a175962
Previously Make class loader contexts used a simplified one-level map.
Now they use the same tree representation as in Soong. The difference
between Make and Soong represenataions is insubstantial and caused only
by the language differences between JSON and Go.
Bug: 132357300
Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: Ia273afb7f026dbe3b25d4327d55143657b026b98
Previously, the createGlobalSoongConfig() function was explicitly
prevented from being used in tests because it would fail. However, it
turns out that is no longer the case and it does now work.
That allows the following changes to be made:
* Tests no longer need to use GlobalSoongConfigForTests() to
prepopulate the cache.
* GlobalSoongConfigForTests() is only needed in the dexpreopt
package.
Bug: 177892522
Test: m nothing
Change-Id: Ifcbb1a44254c5d2d10c1d02ab23227488d1d1ed1
Bug: 176171716
Test: build_mainline_modules.sh and ensure that the ART boot image is
built: `find out -type f -name '*.art'` is nonempty for every arch.
Change-Id: Ibc45581eef2b205c750a30709780cf659ba7cfa1
The removed API has been unused since https://r.android.com/1533342
(except for unit tests).
Changes in the unit tests reflect the change of API in
https://r.android.com/1533342: early errors caused by unknown library
paths at CLC construction time have been replaced with late errors at
the time of CLC validation.
Bug: 132357300
Test: m nothing
Change-Id: I739c7c41b6f882b7e28cdd6acd05961d754d8687
Previously CLC construction was scattered across different module types
and dependency tags. This CL moves all logic to one function, which
handles all special cases. This will allow to simplify CLC API and
reduce the number of different ways in which CLC is constructed.
Previously some of the cases failed early (at the time when a library is
added to CLC) if the build/install paths were unknown. Other cases did
not fail early, but were validated later before CLC was used. Late
failures are necessary because some of the libraries with unknown paths
still have to be processed by manifest_fixer (which doesn't need library
paths), but they do not use dexpreopt (which needs library paths). This
CL removes the early failures (all paths are still validated later).
The CLC tests do not fail because they use a private method that toggles
the "strict" flag (that enforces early/late failure mode) manually in
the method call.
The CL also makes a functional change in the way CLC is constructed for
component libraries that have an OptionalImplicitSdkLibrary(), or
libraries that are disguised as SDK libraries via `provides_uses_lib`.
Previously such a component/disguised library X was added to its own CLC
as a sibling element of X's own <uses-library> dependencies, which
created incorrect CLC structure. Now this is handled by addCLCFromDep,
when X is processed as dependency and added as a top-level CLC element
with its sub-CLC properly nested under it.
Bug: 132357300
Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: I6a512209b87b81d785875f10f76b21c81b2ed579
Enable the RuleBuilder and RuleBuilderCommand methods to access
the BuilderContext by passing it to NewRuleBuilder instead of
RuleBuilder.Build.
Test: genrule_test.go
Test: rule_builder_test.go
Test: m checkbuild
Change-Id: I63e6597e19167393876dc2259d6f521363b7dabc
This patch reworks the approach introduced in
https://r.android.com/1450819. That patch based the decision which
libraries should be added to the manifest <uses-library> tags by the
manifest_fixer on the "shared" status of the library.
That approach is incorrect for two reasons:
- It doesn't make sense to have a non-shared library in class loader
context ("shared" libraries are those specified in
frameworks/base/data/etc/platform.xml, and they are the only ones
that PackageManager knows about).
- It doesn't make sense to add anything but the top-level of the
class loader context tree to the manifest, because this part of the
tree is flattened to a sequence, and PackageManager cannot restore
it to the previous tree shape (there is an information loss).
This patch removes the "shared" bit of information from class loader
context elements and assumes that all libraries that end up in class
loader context are shared. Consequently, only the top-level libraries
should be passed to manifest_fixer.
Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Bug: 168686456
Change-Id: I902690f0f38f1047fa79cf6ccbe956077eceaab0
Martin Stjernholm