Commit Graph

57 Commits

Author SHA1 Message Date
David Srbecky 1aacc6c3df Fix dump-oat-boot build target.
Implement support of dumping host boot.art.

That was not possible before because imageLocation is different on host.

This CL makes imageLocation per-variant so can access for the host variant.

Test: m dump-oat-boot
Change-Id: Id82161e4901a6b77f220238aec460b8609fd765f
2020-03-26 15:28:27 +00:00
Ulya Trafimovich dacc6c56f1 Simplify the construction of class loader contexts for system server jars.
This reworks CL https://r.android.com/1180134 as follows:

1) Do not reorder the list of system server jars passed from Make to
  Soong via the product variable PRODUCT_SYSTEM_SERVER_JARS. This means
  that for some products the order of jars on the system server
  classpath may be non-optimal: a jar X that depends on Y may be
  dexpreopted before Y, so that all references to the classes and
  methods from Y wil be unresolved.

  Unfortunately for such products, fixing the order is not a simple
  matter of rearranging their PRODUCT_SYSTEM_SERVER_JARS, because the
  conflicts may arise when the product-specific variable gets merged
  with the common variable.

2) As a consequence of 1), do not add artificial dependencies between
  system server jars: this is now impossible, as it would create
  circular dependencies for those products that have non-optimal order
  of jars.

3) Copy dex files for system server jars to a predefined build location.
  This is necessary because at the time when Soong generates class
  loader context for k-th jar, it needs to know the paths to jars 1 ..
  (k-1), and it might have not processed those jars yet (so Soong can't
  query the paths from the modules).

  This approach is similar to the way Soong handles bootclasspath jars.

4) Do not exclude from dexpreopting system server jars that are not
  Soong modules (those that are Make modules). The previous CL excluded
  them because Make modules do not have ModuleContext. But it turns out
  that ModuleContext is not necessary, as all the information is passed
  via the dexpreopt config.

Test: aosp_walleye-userdebug boots and there are no messages in the
  logcat regarding class loader context mismatch:

  $ adb logcat | grep 'mismatch'
  # empty

Test: Class loader contexts in the oat files for system server jars
  match expectations:

  $ oatdump --oat-file=out/target/product/walleye/system/framework/oat/arm64/com.android.location.provider.odex 2>/dev/null | grep '^classpath'
  classpath = PCL[]
  $ oatdump --oat-file=out/target/product/walleye/system/framework/oat/arm64/services.odex 2>/dev/null | grep '^classpath'
  classpath = PCL[/system/framework/com.android.location.provider.jar*1989208671]
  $ oatdump --oat-file=out/target/product/walleye/system/framework/oat/arm64/ethernet-service.odex 2>/dev/null | grep '^classpath'
  classpath = PCL[/system/framework/com.android.location.provider.jar*1989208671:/system/framework/services.jar*4040443083:/system/framework/services.jar!classes2.dex*2210087472]

Test: The phone boots and logcat has no scary messages related to
  class loader contexts:

  $ lunch aosp_walleye-userdebug && m
  $ adb reboot bootloader && fastboot flashall -w && adb wait-for-device
  $ adb root
  $ adb shell stop
  $ adb logcat -c
  $ adb shell setprop dalvik.vm.extra-opts -verbose:oat
  $ adb shell start
  $ adb logcat | egrep -io 'system_server: .*load.*/system/framework.*'
  system_server: Loading /system/framework/oat/arm64/com.android.location.provider.odex with executable: 0
  system_server: Successfully loaded /system/framework/oat/arm64/com.android.location.provider.odex with executable: 0
  system_server: Loading /system/framework/oat/arm64/services.odex with executable: 0
  system_server: Successfully loaded /system/framework/oat/arm64/services.odex with executable: 0
  system_server: Loading /system/framework/oat/arm64/ethernet-service.odex with executable: 0
  system_server: Successfully loaded /system/framework/oat/arm64/ethernet-service.odex with executable: 0
  system_server: Loading /system/framework/oat/arm64/wifi-service.odex with executable: 0
  system_server: Successfully loaded /system/framework/oat/arm64/wifi-service.odex with executable: 0
  system_server: Loading /system/framework/oat/arm64/com.android.location.provider.odex with executable: 1
  system_server: Successfully loaded /system/framework/oat/arm64/com.android.location.provider.odex with executable: 1
  system_server: Loading /system/framework/oat/arm64/services.odex with executable: 1
  system_server: Successfully loaded /system/framework/oat/arm64/services.odex with executable: 1
  system_server: Loading /system/framework/oat/arm64/ethernet-service.odex with executable: 1
  system_server: Successfully loaded /system/framework/oat/arm64/ethernet-service.odex with executable: 1
  system_server: Loading /system/framework/oat/arm64/wifi-service.odex with executable: 1
  system_server: Successfully loaded /system/framework/oat/arm64/wifi-service.odex with executable: 1
  system_server: Loading /system/framework/oat/arm64/com.android.location.provider.odex with executable: 1
  system_server: Successfully loaded /system/framework/oat/arm64/com.android.location.provider.odex with executable: 1

Bug: 141785760
Bug: 140451054
Bug: 148944771
Bug: 147017252

Change-Id: I33c4087f8bfacd0ecb89877aa150b75360d06710
Merged-In: I33c4087f8bfacd0ecb89877aa150b75360d06710
(cherry picked from commit a4a83b0ef9)

Exempt-From-Owner-Approval: cherry-pick.
2020-03-23 12:27:31 +00:00
David Srbecky 7f8dac1254 Generate the ART boot image for host as well (used by ART tests).
This will allow us to remove the core.art image compiled in make.

Test: lunch aosp_x86_64 && m test-art-host-gtest
Bug: 147817558
Bug: 147819342
Change-Id: I864f72198d875dc1421c9c029a683bafbe7b71ac
2020-03-10 13:35:27 +00:00
David Srbecky c177ebeca9 Distinguish boot images by target rather than arch
We plan to add boot image variants for host tests.
Distinguishing the variants by arch does not work,
since both host and device can have the same arch.

Change-Id: Iea73c77367affb074f97a0fc318389417ce537da
2020-02-26 17:04:21 +00:00
Nicolas Geoffray 47cbfcdd3e Add system server jars expressed in make in the system server classpath.
This was missed in:
https://android-review.googlesource.com/c/platform/build/soong/+/1180134

Test: lunch and build ARC, check SYSTEMSERVERCLASSPATH
Bug: 148944771
Bug: 141785760
Bug: 140451054
Change-Id: I619aee5441f7233010067a6c780e978f38ba7796
2020-02-17 19:47:05 +00:00
Martin Stjernholm 40f9f3c061 Reland: Move the Once cache for dexpreopt.GlobalConfig into the
dexpreopt package.

Preparation for a future CL that will need to get the make-written
GlobalConfig from within dexpreopt.

Also rename the Load*Config functions to Parse*Config, since they don't
actually load the config files anymore.

This relands https://r.android.com/1211982.

Bug: 145934348
Test: m
Change-Id: Icb9332a93811d77d2d8b06e983b92501b180a358
2020-02-13 17:46:48 +00:00
Martin Stjernholm 75a48d8ae2 Reland: Separate dexpreopt.GlobalSoongConfig to allow independent
caching of it.

Introduce a Once cache for GlobalSoongConfig to allow it to get binary
tool paths from ordinary module dependencies (coming in a future CL)
that are then reused in singletons.

This relands https://r.android.com/1205729.

Bug: 145934348
Test: m
Change-Id: I039d6e204bee5ddc16d8e2d85057fbec20e326fe
2020-02-13 17:21:13 +00:00
Ulya Trafimovich f3ff010bd4 Use precise class loader context for system server jars.
This patch excludes non-Soong system server jars from dexpreopting.

System server jars should be dexpreopted together for better
performance: each jar should have all preceding system server jars
in its class loader context (that is passed to dex2oat and recorded
in the resulting .oat file to be used by the loader).

This means that we need a total order on system server jars. The
default order provided by PRODUCT_SYSTEM_SERVER_JARS is not good, as
it does not always respect genuine dependencies between jars (counter-
examples are rare, but non-trivial to fix: b/148219586).

This patch adds a post deps mutator pass that creates additional
dependencies and enforces global order. These are later used to
generate precise class loader contexts and system server classpath.

Test: Class loader contexts in the oat files for system server jars
  match expectations:

  $ oatdump --oat-file=out/target/product/walleye/system/framework/oat/arm64/com.android.location.provider.odex | grep '^classpath'
  classpath = PCL[]
  $ oatdump --oat-file=out/target/product/walleye/system/framework/oat/arm64/services.odex | grep '^classpath'
  classpath = PCL[/system/framework/com.android.location.provider.jar*1919890654]
  $ oatdump --oat-file=out/target/product/walleye/system/framework/oat/arm64/wifi-service.odex | grep '^classpath'
  classpath = PCL[/system/framework/com.android.location.provider.jar*1919890654:/system/framework/services.jar*4269704903:/system/framework/services.jar!classes2.dex*134345935]
  ...

Test: The phone boots and logcat has no scary messages related to
  class loader contexts:

  $ lunch aosp_walleye-userdebug && m
  $ adb reboot bootloader && fastboot flashall -w && adb wait-for-device
  $ adb root
  $ adb shell stop
  $ adb logcat -c
  $ adb shell setprop dalvik.vm.extra-opts -verbose:oat
  $ adb shell start
  $ adb logcat | egrep -i 'system_server: .*load.*/system/framework'
  02-03 14:14:26.912  5016  5016 I system_server: Loading /system/framework/oat/arm64/com.android.location.provider.odex with executable: 1
  02-03 14:14:26.914  5016  5016 I system_server: Successfully loaded /system/framework/oat/arm64/com.android.location.provider.odex with executable: 1
  02-03 14:14:26.914  5016  5016 I system_server: Loading /system/framework/oat/arm64/services.odex with executable: 1
  02-03 14:14:26.916  5016  5016 I system_server: Successfully loaded /system/framework/oat/arm64/services.odex with executable: 1
  02-03 14:14:26.927  5016  5016 I system_server: Loading /system/framework/oat/arm64/wifi-service.odex with executable: 1
  02-03 14:14:26.933  5016  5016 I system_server: Successfully loaded /system/framework/oat/arm64/wifi-service.odex with executable: 1
  02-03 14:14:26.933  5016  5016 I system_server: Loading /system/framework/oat/arm64/ethernet-service.odex with executable: 1
  02-03 14:14:26.934  5016  5016 I system_server: Successfully loaded /system/framework/oat/arm64/ethernet-service.odex with executable: 1
  02-03 14:14:26.946  5016  5016 I system_server: Loading /system/framework/oat/arm64/com.android.location.provider.odex with executable: 0
  02-03 14:14:26.947  5016  5016 I system_server: Successfully loaded /system/framework/oat/arm64/com.android.location.provider.odex with executable: 0
  02-03 14:14:26.947  5016  5016 I system_server: Loading /system/framework/oat/arm64/services.odex with executable: 0
  02-03 14:14:26.948  5016  5016 I system_server: Successfully loaded /system/framework/oat/arm64/services.odex with executable: 0
  02-03 14:14:26.948  5016  5016 I system_server: Loading /system/framework/oat/arm64/wifi-service.odex with executable: 0
  02-03 14:14:26.948  5016  5016 I system_server: Successfully loaded /system/framework/oat/arm64/wifi-service.odex with executable: 0
  02-03 14:14:26.949  5016  5016 I system_server: Loading /system/framework/oat/arm64/ethernet-service.odex with executable: 0
  02-03 14:14:26.949  5016  5016 I system_server: Successfully loaded /system/framework/oat/arm64/ethernet-service.odex with executable: 0
  02-03 14:14:30.480  5016  5016 I system_server: Loading /system/framework/oat/arm64/com.android.location.provider.odex with executable: 1
  02-03 14:14:30.481  5016  5016 I system_server: Successfully loaded /system/framework/oat/arm64/com.android.location.provider.odex with executable: 1

Bug: 141785760
Bug: 140451054
Bug: 148944771

Change-Id: Idac678dbd1f5fe0e381ce8e0e3561423f8a31389
2020-02-13 14:39:03 +00:00
Ulya Trafimovich 4cdada2d1f Make PRODUCT_BOOT_JARS and PRODUCT_UPDATABLE_BOOT_JARS disjoint.
This is to unify boot jars with system server jars:
PRODUCT_SYSTEM_SERVER_JARS and PRODUCT_UPDATABLE_SYSTEM_SERVER_JARS
do not overlap.

Test: aosp_walleye-userdebug boots

Exempt-From-Owner-Approval: cherry-pick.
Bug: 146363577
Change-Id: If001049a46f8e9d38921dabeedbd61e971ebfe73
Merged-In: If001049a46f8e9d38921dabeedbd61e971ebfe73
(cherry picked from commit ef4358e536)
2020-02-13 13:35:39 +00:00
Nicolas Geoffray 73d5983af0 Remove UseApexImage and GenerateApexImage.
These options are not used anymore.

Bug: 119800099
Test: m
Change-Id: I4568ff23ed71a5c288ed87828aed6e4bc4f8bd4d
2020-02-07 14:19:28 +00:00
Ulya Trafimovich b0a2d37dcb Fix --boot-image argument in dex2oat command for boot image extension.
The primary image name should be "boot.art" in case of
"boot-framework.art", and "apex.art" in case of "apex-framework.art".
Although "boot.art" and "apex.art" are identical binaries, dex2oat
has hard-coded logic based on the image name.

This has been broken since boot image extension has been enabled for
JIT-zygote config in CL I5493e575ebf90bad1d5ad2850004d54590bbc079.

Test: compare boot-framework.art and apex-framework.art,
  they were identical before this CL, and differ after this CL.

Test: JIT-zygote config boots, steps 1-2:

  1. Temporarily enable Jit zygote in the product device config (in
    this case device/google/muskie/aosp_walleye.mk):

    +# System server should not contain compiled code.
    +PRODUCT_SYSTEM_SERVER_COMPILER_FILTER := verify
    +
    +# Use the apex image for preopting.
    +DEXPREOPT_USE_APEX_IMAGE := true
    +
    +# Have the runtime pick up the apex image.
    +PRODUCT_PROPERTY_OVERRIDES += \
    +    dalvik.vm.boot-image=/apex/com.android.art/javalib/apex.art:/system/framework/apex-framework.art

  2. Build and flash:
    $ lunch aosp_walleye-userdebug && m \
    && adb reboot bootloader && fastboot flashall -w

Change-Id: I98de271852ecc33feb9fd4c9b0addf0feba01856
2020-01-28 14:42:41 +00:00
Hans Boehm e4b5342d43 Revert "Separate dexpreopt.GlobalSoongConfig to allow independen..."
Revert submission 1211982-dex2oat-soong-dep

Reason for revert: Build failures. See b/148312086.

Reverted Changes:
Ibc427a9a8: Make dex2oat(d) visible for use as implicit dexpre...
I71df11c1e: Move the Once cache for dexpreopt.GlobalConfig int...
I38317f2d5: Get the dex2oat host tool path from module depende...
I440a09dba: Separate dexpreopt.GlobalSoongConfig to allow inde...

Bug: 148312086
Bug: 145934348
Exempt-From-Owner-Approval: Plain revert
Change-Id: Ice3990225635a737e49e9aed7373f06516fccea3
2020-01-27 13:44:24 +00:00
Hans Boehm 453bf09853 Revert "Move the Once cache for dexpreopt.GlobalConfig into the ..."
Revert submission 1211982-dex2oat-soong-dep

Reason for revert: Build failures. See b/148312086.

Reverted Changes:
Ibc427a9a8: Make dex2oat(d) visible for use as implicit dexpre...
I71df11c1e: Move the Once cache for dexpreopt.GlobalConfig int...
I38317f2d5: Get the dex2oat host tool path from module depende...
I440a09dba: Separate dexpreopt.GlobalSoongConfig to allow inde...

Bug: 148312086
Bug: 145934348
Exempt-From-Owner-Approval: Plain revert
Change-Id: I6b656afb5feaad70d958b9d38b6c6eab7b03fba1
2020-01-27 13:44:03 +00:00
Martin Stjernholm dae8a80426 Move the Once cache for dexpreopt.GlobalConfig into the dexpreopt
package.

Preparation for a future CL that will need to get the make-written
GlobalConfig from within dexpreopt.

Also rename the Load*Config functions to Parse*Config, since they don't
actually load the config files anymore.

Bug: 145934348
Test: m
Change-Id: I71df11c1e042ca8135d273a7263e9539ea3cd68f
2020-01-24 22:18:42 +00:00
Martin Stjernholm be9d0d21d1 Separate dexpreopt.GlobalSoongConfig to allow independent caching of
it.

Introduce a Once cache for GlobalSoongConfig to allow it to get binary
tool paths from ordinary module dependencies (coming in a future CL)
that are then reused in singletons.

Bug: 145934348
Test: m
Change-Id: I440a09dba7d337965a196527566b0966a18e3653
2020-01-24 22:09:54 +00:00
Ulyana Trafimovich 4d9c97bd77 Merge "Use boot image extension in the JIT-zygote experiment." 2020-01-13 10:38:01 +00:00
Treehugger Robot ae7cddd5ad Merge "Sandbox soong_build by changing to root directory" 2020-01-11 06:00:21 +00:00
Colin Cross 988414c2cf Sandbox soong_build by changing to root directory
This relands I12a0f907753fefd1997ab8b4ea2ac331234093cf along with
a fix to blueprint for absolute paths.

Store the current working directory and then change to the root
directory so that all file accesses must go through helpers in
the android package that properly track dependencies.

Change-Id: I24ac485677aa102eec1a2521d16820da6ee1ae77
Fixes: 146437378
Test: m checkbuild
Test: m OUT_DIR=/tmp/out nothing
2020-01-11 01:11:46 +00:00
Colin Cross e734832d25 Merge "Revert "Sandbox soong_build by changing to root directory"" 2020-01-10 22:00:59 +00:00
Colin Cross 47e4f9e1e8 Revert "Sandbox soong_build by changing to root directory"
This reverts commit 05c25ccb4a.

Reason for revert: broke absolute OUT_DIR
Bug: 146437378

Change-Id: I523ed79d40e1c1ef040212ba794a7a084abea75d
2020-01-10 18:51:04 +00:00
Colin Cross e35e961aa9 Merge "Sandbox soong_build by changing to root directory" 2020-01-10 17:35:22 +00:00
Ulya Trafimovich 57547452cf Use boot image extension in the JIT-zygote experiment.
Test: temporarily enable JIT-zygote configuration, build, boot the
  device and ensure that the JIT-zygote specific boot image
  apex-framework.art is mapped in the zygote address space:

  1. enable Jit zygote in the product device config (in my case
    device/google/muskie/aosp_walleye.mk):

    +# System server should not contain compiled code.
    +PRODUCT_SYSTEM_SERVER_COMPILER_FILTER := verify
    +
    +# Use the apex image for preopting.
    +DEXPREOPT_USE_APEX_IMAGE := true
    +
    +# Have the runtime pick up the apex image.
    +PRODUCT_PROPERTY_OVERRIDES += \
    +    dalvik.vm.boot-image=/apex/com.android.art/javalib/apex.art:/system/framework/apex-framework.art

  2. lunch aosp_walleye-userdebug \
    && m \
    && adb reboot bootloader \
    && fastboot flashall -w

  3. adb shell cat /proc/`adb shell ps | grep zygote64 | awk {'print $2'}`/maps | grep apex-framework.art
  6fe44000-7025c000 rw-p 00000000 fc:02 1179718                            /data/dalvik-cache/arm64/apex@com.android.art@javalib@apex-framework.art
  70571000-70696000 rw-p 0072d000 fc:02 1179718                            /data/dalvik-cache/arm64/apex@com.android.art@javalib@apex-framework.art
  75a339f000-75a33ac000 r--p 00852000 fc:02 1179718                        /data/dalvik-cache/arm64/apex@com.android.art@javalib@apex-framework.art

Change-Id: I5493e575ebf90bad1d5ad2850004d54590bbc079
2020-01-10 15:29:25 +00:00
Colin Cross 05c25ccb4a Sandbox soong_build by changing to root directory
Store the current working directory and then change to the root
directory so that all file accesses must go through helpers in
the android package that properly track dependencies.

Fixes: 146437378
Test: m checkbuild
Change-Id: I12a0f907753fefd1997ab8b4ea2ac331234093cf
2020-01-09 14:19:46 -08:00
Ulyana Trafimovich 5a4ccd1816 Revert^2 "Use boot image extension for framework libraries."
This reverts commit 3fae7662ee
Reason: re-applying the change after resolving the problem with
coverage builds (in a related CL).

Use boot image extension for framework libraries.
This patch splits the system boot image in two parts:

  - The ART boot image. This is the primary boot image that is
    included in the ART apex and contains dexpreopted Core Libraries.

  - The framwework boot image extension. It depends on the ART boot
    image and contains framework libraries.

The third "apex" boot image (used in the JIT-zygote experiment)
remains unchanged; it is a monolithic primary boot image that
contains both libcore and framework libraries.

Dexpreopting of APKs now uses the framework boot image extension
(which in turn pulls in the ART boot image as a dependency).

Bug: 146462581
Bug: 119800099

Test: aosp_walleye-userdebug boots.

Change-Id: I06c5ac5fca011fa639ed208735462ab32451df3a
2020-01-08 15:56:52 +00:00
Ulyana Trafimovich 11e1ae025f Merge "Include jacoco in the ART boot image if EMMA_INSTRUMENT_FRAMEWORK=true." 2020-01-08 10:58:24 +00:00
Martin Stjernholm c52aaf11d0 Move generation of the dexpreopt tool paths into Soong.
The dexpreopt global config is now split into the part that is generated
from make (in build/make/core/dex_preopt_config.mk) and the part that is
generated from Soong. Since the goal is to generate the dex2oat path from
Soong dependencies, the old GlobalConfig.Tools struct is simply repurposed
for the Soong generated config, although the intention is to allow more
settings to migrate from make to Soong, and hence from GlobalConfig to
GlobalSoongConfig.

Since the new dexpreopt_soong.config is written from a Soong-created ninja
rule, it doesn't need to be rewritten to out/soong/<device>/ like the old
make-created config file.

Test: m
Test: env USE_DEX2OAT_DEBUG=false m
  (check that out/soong/dexpreopt_soong.config points to dex2oat instead of dex2oatd)
Bug: 145934348

Change-Id: Ifd45c4a08e2ec55b86f4a93f0d85bd39cf2cf189
2020-01-07 14:15:42 +00:00
Ulya Trafimovich 4456188939 Include jacoco in the ART boot image if EMMA_INSTRUMENT_FRAMEWORK=true.
Test: aosp_walleye-userdebug boots.

Test: Coverage build boots:
  $ lunch aosp_walleye-userdebug
  $ cd $ANDROID_BUILD_TOP
  $ build/soong/soong_ui.bash --make-mode droid \
    SKIP_ABI_CHECKS=true \
    TARGET_PRODUCT=aosp_walleye \
    TARGET_BUILD_VARIANT=userdebug \
    EMMA_INSTRUMENT=true \
    NATIVE_COVERAGE=true
  $ adb reboot bootloader && fastboot flashall -w

Test: Coverage build with framework coverage boots:
  $ lunch aosp_walleye-userdebug
  $ cd $ANDROID_BUILD_TOP
  $ build/soong/soong_ui.bash --make-mode droid \
    SKIP_ABI_CHECKS=true \
    TARGET_PRODUCT=aosp_walleye \
    TARGET_BUILD_VARIANT=userdebug \
    EMMA_INSTRUMENT=true \
    EMMA_INSTRUMENT_FRAMEWORK=true \
    NATIVE_COVERAGE=true
  $ adb reboot bootloader && fastboot flashall -w

Test: Static coverage build with framework coverage boots:
  $ lunch aosp_walleye-userdebug
  $ cd $ANDROID_BUILD_TOP
  $ build/soong/soong_ui.bash --make-mode droid \
    SKIP_ABI_CHECKS=true \
    TARGET_PRODUCT=aosp_walleye \
    TARGET_BUILD_VARIANT=userdebug \
    EMMA_INSTRUMENT=true \
    EMMA_INSTRUMENT_FRAMEWORK=true \
    EMMA_INSTRUMENT_STATIC=true \
    NATIVE_COVERAGE=true
  $ adb reboot bootloader && fastboot flashall -w

Change-Id: Iaa198b8505aaff36e6685559642ff721637ce55f
2020-01-06 16:52:56 +00:00
Lingfeng Yang 54191fae42 Revert "Consistently use either "boot" or "apex" boot image as the default."
This reverts commit 48b3b3c71f.

Reason for revert: breaks emulator

BUG: 146558375

Change-Id: I2af3d11ad9a014b0d73a73540bc814f245bc6418
2019-12-19 16:41:40 +00:00
Ulya Trafimovich 48b3b3c71f Consistently use either "boot" or "apex" boot image as the default.
Previous CL Ia9b34aa92ebb1b4de96ea0f8f290d798be19b2cf introduced
asymmetry in handling "boot" and "apex" boot images: in JIT-zygote
experiment, though the "apex" boot image was installed, the "boot"
boot image was still used for generating dexpreopt configs.

It is unclear why this asymmetry was needed at that point; it seems
incorrect to use different boot images for dexpreopting and for
installing on device. After recent changes the asymmetry started
breaking walleye_jitzygote-userdebug on git_rvc-release, because APK
dexpreopt commands refer to inexistent boot image files.

Test: lunch aosp_walleye-userdebug && m
Test: cherry-pick CL in internal, walleye_jitzygote-userdebug boots

Change-Id: Id877c10269cf79caf6ae74b1dc169a31e6a2211b
2019-12-18 16:09:39 +00:00
Ulyana Trafimovich 3fae7662ee Revert "Use boot image extension for framework libraries."
This reverts commit 4d2eeed0da.

Reason for revert: breaks avd/avd_boot_health_check test
  on cf_x86_phone-userdebug_coverage on branch rvc-release
  (the device fails to boot).

Test: m
Test: aosp_walleye-userdebug boots

Bug: 145749668

Exempt-From-Owner-Approval: revert

Change-Id: Ie1d93a200222e26461c1bcd384fdb69b7351e259
2019-12-11 18:29:19 +00:00
Colin Cross 31bf00dac3 Export art dexpreopt image zip file
Test: m  out/target/product/generic_arm64/boot.zip
Bug: 144491400
Change-Id: I84342703ac614e072b0cde8c6f5597dc8cffbc65
2019-12-04 16:50:52 -08:00
Ulya Trafimovich 4d2eeed0da Use boot image extension for framework libraries.
This patch splits the system boot image in two parts:

  - The ART boot image. This is the primary boot image that is
    included in the ART apex and contains dexpreopted Core Libraries.

  - The framweork boot image extension. It depends on the ART boot
    image and contains framework libraries.

The third "apex" boot image (used in the JIT-zygote experiment)
remains unchanged; it is a monolithic primary boot image that
contains both libcore and framework libraries.

Dexpreopting of APKs now uses the framework boot image extension
(which in turn pulls in the ART boot image as a dependency).

Test: m
Test: phone boots:
    lunch aosp_walleye-userdebug && m \
        && adb reboot bootloader && fastboot flashall -w

Bug: b/119800099

Exempt-From-Owner-Approval: rebased after getting approval.

Change-Id: Ida40dfae8c83bf7c2e737d5c7ea418e1197ad826
2019-12-03 13:59:25 +00:00
Roshan Pius ccc26ef316 soong: Align update boot jar variable with system_server jar
Use apex:jar pair values for updatable boot jars to align with
updatable system_server values.

Test: Compiles & flashed device. Ensured that the bootclasspath & system_server class
paths remain the same.

Change-Id: I1d6dfe3325d9091b7c724458be4e6471f9086666
2019-12-02 08:37:57 -08:00
Ulyana Trafimovich f2cb7e959b Revert^2 "Do not dexpreopt system server jars from updatable modules."
This reverts commit 01f6b0a656.

Reason for revert: Build failure is not reproducible.
Forrest build on the same build ID 6033773 and same target
cf_x86_phone-userdebug_coverage finished successfully.

Change-Id: I5077f8332aa0b8037e324b89d41f35b86b8cf216
2019-11-27 12:29:50 +00:00
Ulyana Trafimovich 01f6b0a656 Revert "Do not dexpreopt system server jars from updatable modules."
This reverts commit 61c325ebcc.

Reason for revert: broken build git_master/cf_x86_phone-userdebug_coverage (likely caused by these changes).

Exempt-From-Owner-Approval: revert.

Change-Id: I88ddd3af3a6c4ffdaa1fbb881d965356c5c75ad3
2019-11-26 13:06:38 +00:00
Ulya Trafimovich 61c325ebcc Do not dexpreopt system server jars from updatable modules.
Test: m

Test: The list of updatable system server jars is empty now,
  so I tested that the filer works with a manual experiment:
  - temporarily add ethernet-service to PRODUCT_UPDATABLE_SYSTEM_SERVER_JARS
  - m nothing
  - fgrep -e 'ethernet-service' $ANDROID_BUILD_TOP/out/soong/build.ninja | grep dexpreopt
  - expect empty output (no dexpreopt command is generated)

Change-Id: I0b231e823d5a5a97632daa2b5eb7be3e06782004
2019-11-25 16:36:57 +00:00
Roshan Pius 9b51a4078a soong: Support system_server jars from apexes
Allow system_server jars delivered via apex. Regular system_server
jars are located in /system/framework folder. But, jars delivered via
apex are mounted at /apex/<module_name>/javalib.

Bug: 144722612
Bug: 141785760
Test: Compiles
Change-Id: Ia40bb91d2e05b2601a52eac28a985fe2d8da3481
2019-11-22 12:47:11 -08:00
Ulyana Trafimovich de534414b3 Revert^4 "Package dexpreopt artifacts for libcore jars in the ART apex."
This reverts commit bf0e47648a.

Reason for revert: coverage build with EMMA_INSTRUMENT_FRAMEWORK=true
is fixed by inspecting the environment variable and not generating
boot image in case it is set.

Dexpreopt artifacts for the libcore part of the boot class path are
now packaged in the ART apex. The system image still contains
dexpreopt artifacts for the full set of boot class path libraries
(both libcore and framework); the libcore part will be removed and
boot image extension will be used in a follow-up CL.

Since this is specific to the ART apex and makes no sense for other
apexes, the implementation adds a boolean flag "is ART apex" rather
than a new apex module property.

Build rules for the new set of dexpreopt artifacts are created using
a new variant of the global boot image config. Previously we had two
variants: "default" (for the system image) and "apex" (for the
JIT-zygote experiment). This patch adds a third "art" variant.

Test: m
Test: m art/build/apex/runtests.sh

Bug: 144091989
Change-Id: I113c0d39222d6d697cb62cd09d5010607872fc2b
2019-11-08 12:48:53 +00:00
Ulyana Trafimovich bf0e47648a Revert^3 "Package dexpreopt artifacts for libcore jars in the ART apex."
Reason for revert: breaks the coverage build.
Exempt-From-Owner-Approval: revert

Change-Id: I4c3db7b085b101738813a7f3da6f0078bbaa587b
2019-11-07 16:23:47 +00:00
Ulyana Trafimovich 66b3e99649 Revert^2 "Package dexpreopt artifacts for libcore jars in the ART apex."
This reverts commit 24babe3a66.

Build failures are fixed after reworking the patch:

  - 'missing dependencies' on aosp-master-art branch is fixed by
    disabling profile generation in case default profile is not
    found (it is part of framework absent in aosp-master-art)

  - invalid dex2oat invocation should no longer happen after
    disabling dexpreopt on targets that do not use default ART config
    (fixed in CL If2d4fe2cdcb6a81c7c6d730d18c2b681a74fb0b7)

Dexpreopt artifacts for the libcore part of the boot class path are
now packaged in the ART apex. The system image still contains
dexpreopt artifacts for the full set of boot class path libraries
(both libcore and framework); the libcore part will be removed and
boot image extension will be used in a follow-up CL.

Since this is specific to the ART apex and makes no sense for other
apexes, the implementation adds a boolean flag "is ART apex" rather
than a new apex module property.

Build rules for the new set of dexpreopt artifacts are created using
a new variant of the global boot image config. Previously we had two
variants: "default" (for the system image) and "apex" (for the
JIT-zygote experiment). This patch adds a third "art" variant.

Bug: 143594594
Bug: 143593500

Test: m
Test: m com.android.art deapexer \
    && find $ANDROID_BUILD_TOP -type f -name 'com.android.art.*.apex \
        | xargs deapexer | grep boot \
    Expect to find javalib/$ARCH/boot*.{art,oat,vdex} files.
Test: m art/build/apex/runtests.sh

Change-Id: Ib37acaec8401bd23c8d547dadf773565406ef448
2019-11-07 10:34:45 +00:00
Jiyong Park 0b2387551b stem property of java modules are propagated to Make
62c7829595c0df53e96addcd347c11ac01012eee introduced the new stem
property to java modules, but it wasn't propagated to Make. Fixing the
problem.

This change also fixes a problem that (module name) == (file name) is
assumed in dexpreopt_config.go, which no longer is the case. A mutator
runs to build a map from module name to its stem. The map is then used
when filling up the file paths in the bootImageConfig struct.

Bug: 139391334
Bug: 143494499
Test: m
Test: BootImageProfileTest
Change-Id: Idbc894f877692401471130de6cbfe5e0dd129da9
2019-10-31 22:13:05 +09:00
Nicolas Geoffray 24babe3a66 Revert "Package dexpreopt artifacts for libcore jars in the ART apex."
This reverts commit d5df949385.

Bug: 143594594
Bug: 143593500
Reason for revert: Some builds are failing.

Change-Id: I69986b472bce39266095e526fcd7ef5f48ece85e
Exempt-From-Owner-Approval: Going back to green.
2019-10-30 11:27:58 +00:00
Ulya Trafimovich d5df949385 Package dexpreopt artifacts for libcore jars in the ART apex.
This patch adds dexpreopt files for the libore part of the
bootclasspath to the ART apex.

Since this is specific to the ART apex and makes not sense for other
apexes, the patch does not add a new module property, but only a
boolean flag denoting that this is an ART apex.

Dexpreopt artifacts packaged into the ART apex differ from those that
are packaged in the system image: it inludes only the libcore part of
bootclasspath jars, but not the framework part. When the boot image
extension is implementd, dexpreopt artifacts for the libcore jars will
be removed from the system image (but for now they are both in the
apex and in the system image).

Build rules for the new set of dexpreopt artifacts are created using
a new variant of the global boot image config. Previously we had two
variants: "default" (for the system image) and "apex" (for the
JIT-zygote experiment). This patch adds the third "libcore" variant.

Test: m
Test: m com.android.art deapexer \
    && find $ANDROID_BUILD_TOP -type f -name 'com.android.art.*.apex \
        | xargs deapexer | grep boot \
    Expect to find dexpreopt/$ARCH/boot-art*.{art,oat,vdex} files.
Test: m art/build/apex/runtests.sh

Change-Id: I353ef90304bc5e18c3055ea379b3b223e5c38948
2019-10-28 14:03:08 +00:00
Ulya Trafimovich 18263381ed Deduplicate config generation for boot images.
Both default and apex configs are used for dexpreopting bootclasspath
jars. The difference between the two configs is in the naming (default
paths contain "boot", and apex paths contain "apex"). Another
difference is that apex config does not have a zip archive.

Test: m
Test: `find $ANDROID_BUILD_TOP/out -name '*.art' | sort`
    returns the same list of files before and after the patch

Change-Id: I5b2e8d83ab94fd0b1b3d4dc3f0db243ef70bfb08
2019-10-23 17:19:17 +01:00
Colin Cross 3b19f5d71d Remove old-style support for translated second architectures
Translated second architectures now go in NativeBridgeArch instead
of DeviceSecondaryArch.

This reapplies I568046330abc002d4eed582cb999b62a5eaba790 with
ctx.Config().HasMulitlibConflict() added to fix the NDK build,
which has arm64, arm, x86_64, and x86 architectures enabled.

Test: m checkbuild
Test: OUT_DIR=out_ndk build/soong/scripts/build-ndk-prebuilts.sh
Test: no change to build.ninja or Android-aosp_cf_x86_phone.mk
Change-Id: Iadcafbd64bfb9579ae7c86914927c43a062b0c8e
2019-09-19 12:54:32 -07:00
Martin Stjernholm cc4b0ad706 Update dexpreopt for the ART APEX name change.
Test: Build & boot
Bug: 135753770
Exempt-From-Owner-Approval: Approved internally
Change-Id: I3aac9112b022cce3ef8cfdf4bdbbed8bb0c8f4c7
Merged-In: I3aac9112b022cce3ef8cfdf4bdbbed8bb0c8f4c7
2019-08-30 17:47:30 +01:00
Mathieu Chartier 6adeee1131 Revert "Revert "Remove option to create boot image based on preloaded classes""
Test: make
Test: build/soong/scripts/build-ndk-prebuilts.sh

This reverts commit d3f688b521.

Change-Id: Ia8898d11d67e5f0c4d8886f13d29f5dbd5f8609b
2019-06-26 14:54:47 -07:00
Dan Willemsen 0f41678d00 Expand the dexpreopt image dependencies to entire image
Instead of just depending on the main .art file (boot.art, etc), also
expose the dependencies to the .oat/.vdex files (boot.oat/boot.vdex),
and all of the module files that get implicitly loading (boot-ext.*,
boot-framework.*, etc)

This is necessary for RBE, where the rule only gets the files that it
depends upon.

Test: treehugger
Test: build a system image with RBE
Change-Id: I0c7051f18582f1891d3398b46763b1521e4326c8
2019-06-18 06:40:38 +00:00
Colin Cross 2d00f0dcae Write dexpreopt.config again during the build
Writing $OUT/dexpreopt.config during build setup and then using
it during the build causes bad incremental builds if a cleanspec
removes $OUT/dexpreopt.config.  Create a rule in Soong to write
out a copy so that it is a normal build artifact.

Bug: 132379813
Test: m checkbuild
Change-Id: I7755a7f8c1d6561fc83022a7f4cf7e4a64e989c9
2019-05-10 07:07:21 -07:00
Colin Cross c11e0c5a39 Fix dangling rules on aosp_cf_x86_phone
aosp_cf_x86_phone uses SecondArchIsTranslated, which was leaving
a dangling rule from the oatdump rules to
out/soong/vsoc_x86/dex_bootjars/system/framework/arm/boot.art.
Consolidate the code to select targets for dexpreopting and use it
in more places to prevent references to boot images that were not
generated.

Test: m checkbuild
Change-Id: Ia4945a99ff5e575e759299106559c85f38489acc
2019-05-08 22:26:14 +00:00