Commit Graph

38266 Commits

Author SHA1 Message Date
Treehugger Robot 404af2ca4b Merge "Build repeatable system images with mke2fs." 2017-10-18 17:20:00 +00:00
Treehugger Robot f4f94ab567 Merge "Follow renames of conscrypt and bouncycastle to remove -host suffix" 2017-10-18 09:41:11 +00:00
Treehugger Robot e02a37d046 Merge "Add strace back onto userdebug builds" 2017-10-18 05:56:17 +00:00
Treehugger Robot 2023783f4f Merge "Switch envsetup.sh dumpvars to soong_ui" 2017-10-18 00:54:44 +00:00
Tom Cherry 1d6e8e31d0 Merge "Allow ro. properties to have arbitrary lengths" 2017-10-17 22:33:11 +00:00
Colin Cross b34d45ad65 Follow renames of conscrypt and bouncycastle to remove -host suffix
Test: m checkbuild
Change-Id: I72bd7249cf6bdc4a2251f6877c776f439649da81
2017-10-17 15:27:52 -07:00
Dan Willemsen af88c41795 Switch envsetup.sh dumpvars to soong_ui
Use soong_ui instead of make&config.mk directly. This way we use a
consistent make parser and sandboxing configurations.

Test: printconfig
Test: get_build_var TARGET_DEVICE
Test: get_abs_build_var PRODUCT_OUT
Test: get_abs_build_var ALL_PRODUCTS
Test: lunch aosp_marlin-userdebug
Test: build_build_var_cache; set | grep var_cache (diff before/after)
Change-Id: I22abdf95669b7a4fb8162377e013a9013155f249
2017-10-17 13:13:30 -07:00
Zach Riggle 7d1f81a048 Add strace back onto userdebug builds
Blueprint stopped supporting the "debug" tag, per dwillemsen

Change-Id: I6c206e058ce32cbea3fd8ae922ac1ba5a0598410
2017-10-17 14:19:53 -05:00
Colin Cross 58cd98dc47 Merge "Use system modules generated by soong when using javac -target 1.9" 2017-10-17 18:37:35 +00:00
Tao Bao d86e3114fc Build repeatable system images with mke2fs.
We have added support in mkuserimg_mke2fs.sh that allows specifying
filesystem UUID and hash seed UUID. This CL generates and passes the
UUIDs based on the image name and build fingerprint. This way we can
rebuild and get identical images.

Note that this only applies to images generated with `m dist` and with
signing tools (sign_target_files_apks.py and
add_img_to_target_files.py). Images under $OUT (built with `make`) are
not affected.

Bug: 64994964
Test: lunch marlin-userdebug; run `m dist` twice (after deleting the
      intermediate files under $OUT/obj/PACKAGING), and compare the
      generated images.
Change-Id: I41cf4e5869582bb930af2f35a8e9c79bff43b2a2
(cherry picked from commit 3aa21e6bb9)
2017-10-16 22:48:33 -07:00
Treehugger Robot 51e7a0cca4 Merge "Make SOONG_JAVAC_WRAPPER use JAVAC_WRAPPER to call javac." 2017-10-17 05:08:10 +00:00
Colin Cross f960257c14 Use system modules generated by soong when using javac -target 1.9
Soong has support for building system modules.  Use the directories
produced by Soong with --system to replace -bootclasspath arguments
when using javac -target 1.9.

Since soong can't generate current SDK stubs yet, and no existing
SDK stubs need -target, only use -target 1.9 for modules that are
not compiling against the SDK.  That means in practice the only
system modules that will be used for now is the default one,
core-system-modules.

Bug: 63986449
Test: m -j EXPERIMENTAL_USE_OPENJDK9=true makes some progress
Change-Id: I350ef50aedf36fdd72458c23d4fe8a2edf1a9a02
2017-10-16 17:55:24 -07:00
Elliott Hughes ff806ece8d Merge "external/svox is going away." 2017-10-14 15:12:39 +00:00
Dan Willemsen a0d0ea3b5c Merge "Switch to $(file) instead of a rule to create soong.variables" 2017-10-13 23:22:01 +00:00
Elliott Hughes 271f6ffa04 external/svox is going away.
Bug: N/A
Test: builds
Change-Id: I2583ebd8bc745fb76397e44e214c730863e55700
2017-10-13 15:41:35 -07:00
Treehugger Robot cf2da22a2a Merge "Re-apply ndk-docs rule." 2017-10-13 21:04:26 +00:00
Dan Albert 303e603fd2 Re-apply ndk-docs rule.
This time wrapped with a check for the docs directory for unbundled
branches.

Test: make checkbuild
Bug: None
Change-Id: If80f0a03850d6cad3eab6d759af02ff2a7870974
2017-10-13 11:56:17 -07:00
Elliott Hughes 05cf348722 Merge "Remove PicoTTS from the base." 2017-10-13 17:26:25 +00:00
Dan Willemsen 7ad03056dd Switch to $(file) instead of a rule to create soong.variables
Kati seems to have some problems updating this file, and the echo
syntax gets a bit messy. So define some macros and write out the file at
eval time using $(file).

Test: multiproduct_kati -only-config, check soong.variables (on AOSP and internal)
Change-Id: I74279b962918ca1f70fb1d0d25cbb30bd5675041
2017-10-12 21:10:39 -07:00
Treehugger Robot 5f958ccad5 Merge "Adding Android verified boot 1.0 metadata into ENG builds" 2017-10-13 03:39:21 +00:00
Mathieu Chartier 9e89b22ae7 Merge "Re-add missing log filter" 2017-10-13 02:25:08 +00:00
Treehugger Robot bd4239a1e1 Merge "Use CALLED_FROM_SETUP instead of KATI" 2017-10-13 01:46:11 +00:00
Treehugger Robot 60e1f8ee27 Merge "Support java compilation sharding for target side." 2017-10-13 00:40:37 +00:00
Dan Willemsen 20f99a60ea Use CALLED_FROM_SETUP instead of KATI
So that we can use kati instead of make for the initial config loading.

Test: m clean; m nothing
Test: USE_GOMA=true m nothing
Test: m PRODUCT-aosp_x86-sdk
Test: m APP-Calculator
Test: multiproduct_kati -only-config  (on AOSP and internal master)
Change-Id: I5d3af847607fa48868c2045401977eca37dc6ae1
2017-10-12 15:35:42 -07:00
Mathieu Chartier dcb1f24e9b Merge "Allow system server jars for WITH_DEXPREOPT_BOOT_IMG_AND_SYSTEM_SERVER_ONLY" 2017-10-12 22:32:43 +00:00
Mathieu Chartier f834eccd52 Allow system server jars for WITH_DEXPREOPT_BOOT_IMG_AND_SYSTEM_SERVER_ONLY
Renamed WITH_DEXPREOPT_BOOT_IMG_ONLY to
WITH_DEXPREOPT_BOOT_IMG_AND_SYSTEM_SERVER_ONLY and changed the
behavior accordingly.

Preopt system server jars since selinux prevents system server from
loading anything from /data. If we don't do this they will need to
be extracted which is not favorable for RAM usage or performance.

Test: make and flash and look at system server maps
Bug: 65122284
Bug: 62356545

(cherry picked from commit 418258cee9)

Merged-In: I2e70c80a86327b455450b95144f21020e7bf0c6f
Change-Id: I316e79c7c6d45e2ccbfff4065137cc3ef9d2738e
2017-10-12 20:40:41 +00:00
Nan Zhang ad818dcbea Support java compilation sharding for target side.
Add "LOCAL_JAVAC_SHARD_SIZE" to represent the number of java
source path entries in each shard.

Sharding is not allowed when "LOCAL_JAVAC_SHARD_SIZE" and
"LOCAL_JAR_PROCESSORS" are both enabled.

Limitation:
1. 0 <= LOCAL_JAVAC_SHARD_SIZE <= 8192
1. 0 < NUM(java_sources) <= 8192
2. 0 <= NUM(shards) <= 100

Performance Compare:
<Unsharded Build------------------------------------------------>
1. Build from clean state
rm -r -f out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/
&& time m
out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar

real	1m2.720s user	5m26.604s sys	0m39.552s

2. Incremental build
m out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar &&
touch frameworks/base/core/java/android/net/http/HttpResponseCache.java && time m
out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar

real	0m37.586s user	5m47.804s sys	0m50.388s

<Sharded Build-------------------------------------------------->
1. Build from clean state
rm -r -f out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/
&& time m
out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar

Javac Shard Size: 50----real	1m10.163s user	25m59.008s sys	1m58.460s
Javac Shard Size: 100---real	1m2.115s user	21m3.600s sys	1m15.964s
Javac Shard Size: 150---real	0m59.520s user	18m10.544s sys	1m12.628s
Javac Shard Size: 200---real	0m56.894s user	15m39.244s sys	1m11.608s
Javac Shard Size: 250---real	0m55.991s user	14m38.716s sys	1m2.292s
Javac Shard Size: 300---real	0m55.114s user	13m6.568s sys	1m8.200s
Javac Shard Size: 350---real	0m53.144s user	12m7.740s sys	1m3.836s
Javac Shard Size: 400---real	0m54.929s user	12m9.324s sys	1m4.340s
Javac Shard Size: 450---real	1m30.194s user	25m31.468s sys	1m52.416s
Javac Shard Size: 500---real	0m53.976s user	10m35.500s sys	0m55.160s

2. Incremental build
m out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar &&
touch frameworks/base/core/java/android/net/http/HttpResponseCache.java && time m
out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar

Javac Shard Size: 50-----real	0m16.322s user	1m8.648s sys	0m31.700s
Javac Shard Size: 100----real	0m16.163s user	1m22.932s sys	0m29.440s
Javac Shard Size: 150----real	0m16.611s user	1m37.828s sys	0m21.168s
Javac Shard Size: 200----real	0m16.936s user	1m49.248s sys	0m28.636s
Javac Shard Size: 250----real	0m17.509s user	1m54.944s sys	0m32.768s
Javac Shard Size: 300----real	0m18.868s user	1m54.088s sys	0m28.824s
Javac Shard Size: 350----real	0m17.629s user	1m54.108s sys	0m31.056s
Javac Shard Size: 400----real	0m18.658s user	2m7.712s sys	0m30.636s
Javac Shard Size: 450----real	0m18.874s user	2m8.808s sys	0m33.540s
Javac Shard Size: 500----real	0m19.432s user	2m24.400s sys	0m30.368s

time m nothing:
real 0m5.799s user 0m7.236s sys 0m3.068s

Test: m clean && m -j checkbuild
Bug: b/67424047
Change-Id: Id0766d2b7de7c4546d29bbc7f8a0dd0e4b9ad45b
2017-10-12 13:23:06 -07:00
Tom Cherry 47c4eb4680 Allow ro. properties to have arbitrary lengths
Bug: 23102347
Bug: 34954705
Test: provide a fingerprint >92 characters long and see that it
      successfully works

Change-Id: Ida9ffd5266acb3b432b208780804a08e8f7391b4
2017-10-12 09:21:48 -07:00
Elliott Hughes 61f7ee9386 Remove PicoTTS from the base.
Bug: N/A
Test: builds
Change-Id: I843383d8298ded51510f993168efd9189d4c176b
2017-10-12 09:11:29 -07:00
Yoshisato Yanagisawa 42b808b18e Make SOONG_JAVAC_WRAPPER use JAVAC_WRAPPER to call javac.
JAVAC_WRAPPER seems not be used when SOONG_JAVAC_WRAPPER is used.
Since JAVAC_WRAPPER just wraps actual javac call, the result must
be the same.

Bug: 67723445
Change-Id: I944f9534587c00ffd1bc79806c534432467ba4a5
2017-10-12 16:04:20 +09:00
Bowgo Tsai 6ceeb1a8bf Adding Android verified boot 1.0 metadata into ENG builds
Adding verified boot metadata with a "disable magic". The resulting
metadata at the end of each image (e.g., system.img, vendor.img) will
be the same as triggering an "adb disable-verity" on an USERDEBUG image.

This can help simplify the code on fs_mgr, which won't have to check if
current image is an ENG build or not.

Bug: 63056044
Test: boot sailfish eng/userdebug builds
Change-Id: I95d23ac7b76c04d6d4483c9c4dc1de16bf0d9c3a
2017-10-12 10:08:44 +08:00
Mathieu Chartier a996337658 Re-add missing log filter
Was accidentally deleted in a61acf62c9

Test: make
Change-Id: Ib25b5c5b54bfbfa1ec80dfc4a0ee5134c8be2631
2017-10-11 17:10:25 -07:00
Courtney Goeltzenleuchter 367da66255 Merge "Move include in prep of Android.bp support" 2017-10-11 14:14:05 +00:00
Courtney Goeltzenleuchter d0047d922e Move include in prep of Android.bp support
Test: make
Merged-In: I1d1a240b2557db3d9d0a0b2227f84eff09f8d4ca
Change-Id: I1d1a240b2557db3d9d0a0b2227f84eff09f8d4ca
(cherry picked from commit dd35495a77)
2017-10-11 03:29:53 +00:00
Treehugger Robot 492f67c0ac Merge "Prevent vendor libs from depending on private VNDK libraries" 2017-10-11 01:45:36 +00:00
Colin Cross 9f588bcd15 Merge "Add hostdex support for soong" 2017-10-10 21:52:17 +00:00
Jiyong Park a3fb1588f4 Prevent vendor libs from depending on private VNDK libraries
For module installed to /vendor partition, direct linking to the libs
marked as `vendor_available: false` is not allowed. The

Bug: 64730695
Test: Add vendor_available: false to libft2 and
libcompiler_rt. Add the two libs into LOCAL_SHARED_LIBRARIES of a vendor
lib (e.g. libdrm). Build fails with the link_type check error message.

Change-Id: Iaf23574ceddb0c087111e1d95997e9ddd60cdf87
2017-10-10 19:38:06 +09:00
Treehugger Robot f398cc78fe Merge changes Id1d8ffb8,I20a6396d
* changes:
  Fix broken foreach loop
  Reduce stack usage in ckati
2017-10-10 05:35:28 +00:00
Dan Willemsen 2d081b35d4 Fix broken foreach loop
This wasn't initializing the STATS.MODULE_TYPE.* variables, leading to
106kB of stack usage in ckati for sdk_x86_64-eng. With this fixed, the
next highest stack usage is 35kB.

Bug: 36182021
Test: Run $(flavor) over all the variables
Test: Check stack usage with custom ckati patches
Test: build-sdk_x86_64.ninja is identical before/after this change
Change-Id: Id1d8ffb8415ad6613d0093d31be5472d42d300fd
2017-10-10 03:36:42 +00:00
Dan Willemsen 44fd0f6b8e Reduce stack usage in ckati
When limiting ckati to 768kB of stack, it would crash with a segfault
when processing this recursive expanded variable. So always use := to
turn this into a simple variable.

I measured this using 772kB of stack in ckati on sdk_x86_64-eng. With
this change, the next highest stack user is 106kB.

Bug: 36182021
Test: ulimit -s 768; lunch sdk_x86_64-eng; m nothing
Test: build-sdk_x86_64.ninja is identical before/after this change
Change-Id: I20a6396db173decf178107e3aac6cf089e541e36
2017-10-10 03:36:21 +00:00
Colin Cross abfa40776f Add hostdex support for soong
When LOCAL_IS_HOST_MODULE is set LOCAL_BUILT_MODULE is the same
as common_javalib.jar, and dexpreopting shouldn't be enabled.

Bug: 67600882
Test: m -j checkbuild
Change-Id: I7397fa430ebc18d5cb6ba71b06e343180f09ae08
2017-10-09 17:37:49 -07:00
Alan Leung bc29d906bc Merge "Revert "Enable D8 by default."" 2017-10-09 21:28:25 +00:00
Alan Leung ba985e6efe Revert "Enable D8 by default."
Breaks EmulatorSmokeTests

Bug: 67569208

This reverts commit 4c1a09823c.

Change-Id: I85b28b9b59226f4ce6fda0ea2445933eade07d89
2017-10-09 21:24:52 +00:00
Treehugger Robot a867dfa71c Merge "Enable D8 by default." 2017-10-09 20:07:53 +00:00
Alan Leung 4c1a09823c Enable D8 by default.
Bug: 67569208
Test: m -j32

Change-Id: Ie80ce7ca0e6955ef6c1fbae1fa5b169374068cf4
2017-10-09 07:41:35 +00:00
Treehugger Robot 4d44c2dc12 Merge changes I06b8efa8,Id2fa2d18
* changes:
  Enable dexpreopt and dex stripping in soong jars
  Refactor removing classes.dex for dexpreopt
2017-10-06 23:14:19 +00:00
Colin Cross f229de4a4b Enable dexpreopt and dex stripping in soong jars
Dexpreopt jars from soong, which also strips classes.dex from
javalib.jar.

Test: m -j checkbuild
Test: unzip -l $OUT/system/framework/core-oj.jar
Change-Id: I06b8efa8993f6c44f0ad8c988e57a3cba08e17ad
2017-10-06 14:36:38 -07:00
Colin Cross ea1629e788 Refactor removing classes.dex for dexpreopt
Combining $(call dexpreopt-remove-classes.dex) into a
$(call copy-one-file) rule is hacky and error prone.  Make a helper
dexpreopt-copy-jar function that takes LOCAL_DEX_PREOPT and copies
the jar, removing the dex files if necessary.

Test: only description message changes to out/build-${TARGET_PRODUCT}.ninja
Change-Id: Id2fa2d18571ca01bc81302c912b8de25f3dde7a1
2017-10-06 14:26:25 -07:00
Tobias Thierer c6465f5d17 Merge "Cap LOCAL_JAVA_LANGUAGE_VERSION <= 1.8 for API versions <= 26." 2017-10-06 19:42:57 +00:00
Treehugger Robot dd68a6cb74 Merge "Don't reserve size for bad blocks when checking image size" 2017-10-06 19:40:29 +00:00