Commit Graph

38229 Commits

Author SHA1 Message Date
Nan Zhang ad818dcbea Support java compilation sharding for target side.
Add "LOCAL_JAVAC_SHARD_SIZE" to represent the number of java
source path entries in each shard.

Sharding is not allowed when "LOCAL_JAVAC_SHARD_SIZE" and
"LOCAL_JAR_PROCESSORS" are both enabled.

Limitation:
1. 0 <= LOCAL_JAVAC_SHARD_SIZE <= 8192
1. 0 < NUM(java_sources) <= 8192
2. 0 <= NUM(shards) <= 100

Performance Compare:
<Unsharded Build------------------------------------------------>
1. Build from clean state
rm -r -f out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/
&& time m
out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar

real	1m2.720s user	5m26.604s sys	0m39.552s

2. Incremental build
m out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar &&
touch frameworks/base/core/java/android/net/http/HttpResponseCache.java && time m
out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar

real	0m37.586s user	5m47.804s sys	0m50.388s

<Sharded Build-------------------------------------------------->
1. Build from clean state
rm -r -f out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/
&& time m
out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar

Javac Shard Size: 50----real	1m10.163s user	25m59.008s sys	1m58.460s
Javac Shard Size: 100---real	1m2.115s user	21m3.600s sys	1m15.964s
Javac Shard Size: 150---real	0m59.520s user	18m10.544s sys	1m12.628s
Javac Shard Size: 200---real	0m56.894s user	15m39.244s sys	1m11.608s
Javac Shard Size: 250---real	0m55.991s user	14m38.716s sys	1m2.292s
Javac Shard Size: 300---real	0m55.114s user	13m6.568s sys	1m8.200s
Javac Shard Size: 350---real	0m53.144s user	12m7.740s sys	1m3.836s
Javac Shard Size: 400---real	0m54.929s user	12m9.324s sys	1m4.340s
Javac Shard Size: 450---real	1m30.194s user	25m31.468s sys	1m52.416s
Javac Shard Size: 500---real	0m53.976s user	10m35.500s sys	0m55.160s

2. Incremental build
m out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar &&
touch frameworks/base/core/java/android/net/http/HttpResponseCache.java && time m
out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar

Javac Shard Size: 50-----real	0m16.322s user	1m8.648s sys	0m31.700s
Javac Shard Size: 100----real	0m16.163s user	1m22.932s sys	0m29.440s
Javac Shard Size: 150----real	0m16.611s user	1m37.828s sys	0m21.168s
Javac Shard Size: 200----real	0m16.936s user	1m49.248s sys	0m28.636s
Javac Shard Size: 250----real	0m17.509s user	1m54.944s sys	0m32.768s
Javac Shard Size: 300----real	0m18.868s user	1m54.088s sys	0m28.824s
Javac Shard Size: 350----real	0m17.629s user	1m54.108s sys	0m31.056s
Javac Shard Size: 400----real	0m18.658s user	2m7.712s sys	0m30.636s
Javac Shard Size: 450----real	0m18.874s user	2m8.808s sys	0m33.540s
Javac Shard Size: 500----real	0m19.432s user	2m24.400s sys	0m30.368s

time m nothing:
real 0m5.799s user 0m7.236s sys 0m3.068s

Test: m clean && m -j checkbuild
Bug: b/67424047
Change-Id: Id0766d2b7de7c4546d29bbc7f8a0dd0e4b9ad45b
2017-10-12 13:23:06 -07:00
Treehugger Robot f398cc78fe Merge changes Id1d8ffb8,I20a6396d
* changes:
  Fix broken foreach loop
  Reduce stack usage in ckati
2017-10-10 05:35:28 +00:00
Dan Willemsen 2d081b35d4 Fix broken foreach loop
This wasn't initializing the STATS.MODULE_TYPE.* variables, leading to
106kB of stack usage in ckati for sdk_x86_64-eng. With this fixed, the
next highest stack usage is 35kB.

Bug: 36182021
Test: Run $(flavor) over all the variables
Test: Check stack usage with custom ckati patches
Test: build-sdk_x86_64.ninja is identical before/after this change
Change-Id: Id1d8ffb8415ad6613d0093d31be5472d42d300fd
2017-10-10 03:36:42 +00:00
Dan Willemsen 44fd0f6b8e Reduce stack usage in ckati
When limiting ckati to 768kB of stack, it would crash with a segfault
when processing this recursive expanded variable. So always use := to
turn this into a simple variable.

I measured this using 772kB of stack in ckati on sdk_x86_64-eng. With
this change, the next highest stack user is 106kB.

Bug: 36182021
Test: ulimit -s 768; lunch sdk_x86_64-eng; m nothing
Test: build-sdk_x86_64.ninja is identical before/after this change
Change-Id: I20a6396db173decf178107e3aac6cf089e541e36
2017-10-10 03:36:21 +00:00
Alan Leung bc29d906bc Merge "Revert "Enable D8 by default."" 2017-10-09 21:28:25 +00:00
Alan Leung ba985e6efe Revert "Enable D8 by default."
Breaks EmulatorSmokeTests

Bug: 67569208

This reverts commit 4c1a09823c.

Change-Id: I85b28b9b59226f4ce6fda0ea2445933eade07d89
2017-10-09 21:24:52 +00:00
Treehugger Robot a867dfa71c Merge "Enable D8 by default." 2017-10-09 20:07:53 +00:00
Alan Leung 4c1a09823c Enable D8 by default.
Bug: 67569208
Test: m -j32

Change-Id: Ie80ce7ca0e6955ef6c1fbae1fa5b169374068cf4
2017-10-09 07:41:35 +00:00
Treehugger Robot 4d44c2dc12 Merge changes I06b8efa8,Id2fa2d18
* changes:
  Enable dexpreopt and dex stripping in soong jars
  Refactor removing classes.dex for dexpreopt
2017-10-06 23:14:19 +00:00
Colin Cross f229de4a4b Enable dexpreopt and dex stripping in soong jars
Dexpreopt jars from soong, which also strips classes.dex from
javalib.jar.

Test: m -j checkbuild
Test: unzip -l $OUT/system/framework/core-oj.jar
Change-Id: I06b8efa8993f6c44f0ad8c988e57a3cba08e17ad
2017-10-06 14:36:38 -07:00
Colin Cross ea1629e788 Refactor removing classes.dex for dexpreopt
Combining $(call dexpreopt-remove-classes.dex) into a
$(call copy-one-file) rule is hacky and error prone.  Make a helper
dexpreopt-copy-jar function that takes LOCAL_DEX_PREOPT and copies
the jar, removing the dex files if necessary.

Test: only description message changes to out/build-${TARGET_PRODUCT}.ninja
Change-Id: Id2fa2d18571ca01bc81302c912b8de25f3dde7a1
2017-10-06 14:26:25 -07:00
Tobias Thierer c6465f5d17 Merge "Cap LOCAL_JAVA_LANGUAGE_VERSION <= 1.8 for API versions <= 26." 2017-10-06 19:42:57 +00:00
Treehugger Robot dd68a6cb74 Merge "Don't reserve size for bad blocks when checking image size" 2017-10-06 19:40:29 +00:00
Tobias Thierer 745c6a55d3 Cap LOCAL_JAVA_LANGUAGE_VERSION <= 1.8 for API versions <= 26.
The default LOCAL_JAVA_LANGUAGE_VERSION is normally 1.8, or 1.9
when EXPERIMENTAL_USE_OPENJDK9 is explicitly set to the
nondefault value of "true".

Since old Android API levels do not support later versions,
LOCAL_JAVA_LANGUAGE_VERSIONs is already capped at 1.7 for
APIs <= 23.

This CL further caps it at 1.8 for all released APIs (currently
<= 26) even when EXPERIMENTAL_USE_OPENJDK9=true.
This may need to be updated in future.

This CL only affects builds where EXPERIMENTAL_USE_OPENJDK9 is
set; it does not affect the default build.

Bug: 67454477
Test: Treehugger

Change-Id: I54ca72bdb09aeb4184d5bb3c51d881dcfc943839
2017-10-06 15:57:33 +01:00
Treehugger Robot 0c0e2b0779 Merge "Use prebuilts/jdk/jdk9 if EXPERIMENTAL_USE_OPENJDK9 is set" 2017-10-06 04:34:52 +00:00
Treehugger Robot 750b8e84c1 Merge "sepolicy: add owners file" 2017-10-06 04:25:27 +00:00
Treehugger Robot 4ec7972785 Merge "Always pass --auto-add-overlay to aapt2 for aar prebuilts" 2017-10-06 00:02:35 +00:00
Colin Cross d7b99409ce Use prebuilts/jdk/jdk9 if EXPERIMENTAL_USE_OPENJDK9 is set
Bug: 62123342
Test: m -j EXPRIMENTAL_USE_OPENJDK9=1.8 checkbuild
Change-Id: I8be331c422412d6416d92184a2eba093f27fc0fb
2017-10-05 16:47:54 -07:00
Dan Willemsen e3c99016f9 Always pass --auto-add-overlay to aapt2 for aar prebuilts
Bug: 64723465
Test: Build all aar prebuilts in our internal tree
Change-Id: I82ef8d59d4f710b755226b2801dc65c2ccffe40c
2017-10-05 14:55:11 -07:00
Jeff Vander Stoep 381f4ea490 sepolicy: add owners file
Test: build
Change-Id: Ic11f283d3d1eeb2fec8b4745d7c0bc647cb7e717
2017-10-05 14:52:12 -07:00
Treehugger Robot f68eea331d Merge "Remove sysfs_writable type" 2017-10-05 17:36:29 +00:00
Treehugger Robot 4feec6ddb8 Merge "allow setting opengles property" 2017-10-05 17:36:25 +00:00
Treehugger Robot 03e8fb10d1 Merge "Add USE_D8_BY_DEFAULT in makefile." 2017-10-05 09:14:44 +00:00
Jeff Vander Stoep 0ce25ee468 Remove sysfs_writable type
It's not used.

Test: build and boot aosp_x86 emulator. No selinux denials.
Change-Id: Ibfc8bfc4f930bac006c86cc05009dc96e78bcf48
2017-10-04 16:43:13 -07:00
Jeff Vander Stoep 178d04e39c allow setting opengles property
Addresses:
avc:  denied  { set } for property=ro.opengles.version
scontext=u:r:qemu_props:s0 tcontext=u:object_r:opengles_prop:s0
tclass=property_service

Test: build
Change-Id: Ia7445344726b941ad153a126dfe66df4bc8536ab
2017-10-04 16:12:36 -07:00
Jiyong Park 03eb06150b Don't reserve size for bad blocks when checking image size
When checking *.img size with the partition size, the build system
reserves additional bits for spare bits and spare bad blocks.

However, for emmc and ufs, the space bits and blocks are entirely
managed by the underlying controller and thus not visible from outside
of the controller. In fact the check routine was made for legacy MTD
storages where raw flash blocks are directly exposed.

This makes the size checking a little bit conservative in modern devices.
Builds were failed even though the *.img can actually fit into the
partition. To handle this problem, the additional size is no longer
reserved when checking *.img size with the partition size.

This change also removes following build flags that are meaningful
only for devices having MTD storages:
BOARD_NAND_PAGE_SIZE
BOARD_NAND_SPARE_SIZE
Further use of them breaks the build

Bug: 35790399
Bug: 66399382
Test: build

Merged-In: I954bf261441b53844e75d05788866f1692a2ad43
Change-Id: I954bf261441b53844e75d05788866f1692a2ad43
2017-10-04 21:25:08 +09:00
Alan Leung 013a37b760 Add USE_D8_BY_DEFAULT in makefile.
Bug: 65854021
Test: m USE_D8=true  showcommands out/target/common/obj/JAVA_LIBRARIES/okhttp_intermediates/with-local/classes.dex
Change-Id: Iec1a05bf6f03cf856faf2e013fa169e1a0beaf7a
2017-10-03 17:15:01 -07:00
Colin Cross b33250278b Merge "Revert "Generate the NDK docs with the ndk-docs target."" 2017-10-04 00:13:54 +00:00
Colin Cross c294f55489 Revert "Generate the NDK docs with the ndk-docs target."
This reverts commit d51b6d0d75.

Reason for revert: Broke all unbundled builds

Change-Id: Id6722a076ee91ca150b1a53935632579c6f6f6dd
2017-10-04 00:13:00 +00:00
Treehugger Robot 2d1681c804 Merge "Use -Werror in build/make" 2017-10-03 22:38:24 +00:00
Dan Willemsen 75cc5bf9ae Merge "Ignore zip comments in dump-package-stats" 2017-10-03 22:29:18 +00:00
Treehugger Robot 0f47dd4c13 Merge "Generate the NDK docs with the ndk-docs target." 2017-10-03 22:11:17 +00:00
Chih-Hung Hsieh 2a5bc9dfbf Use -Werror in build/make
* Remove unused local variables.

Bug: 66996870
Test: build with WITH_TIDY=1
Change-Id: Iced5bb903902f49d7cd5cf5f572906a512c75e18
2017-10-03 13:53:13 -07:00
Dan Albert d51b6d0d75 Generate the NDK docs with the ndk-docs target.
Test: make ndk-docs
Bug: None
Change-Id: I846f18c2d466d18603bb3ba59aaed7a9c938b1e3
2017-10-03 13:08:25 -07:00
Dan Willemsen 093075304b Ignore zip comments in dump-package-stats
Use -q to silence zip comments, which also silences the first line.

Bug: 67345114
Test: find vendor -name '*.apk' | xargs build/tools/dump-package-stats
Change-Id: Icb319c603de7c29644c58520213940179f67b7e4
2017-10-03 12:56:09 -07:00
Treehugger Robot b039cf792f Merge "Delete jack support" 2017-10-03 01:17:25 +00:00
Treehugger Robot 26cf972f50 Merge "Fix incremental aapt builds" 2017-10-02 23:22:28 +00:00
Dan Willemsen f439c334da Fix incremental aapt builds
If a resource was moved / removed without any other timestamp updates,
ninja would not re-run aapt. To fix this, add a list of resource files
into the same command as aapt (echo ... >/dev/null). That way if the
list changes, ninja will recognize the command line changing and rerun
aapt.

Bug: 67315436
Test: mmma cts/tests/autofillservice
Test: mv cts/tests/autofillservice/res/layout/welcome_activity.xml \
      cts/tests/autofillservice/res/layout/welcome_activity_test.xml
Test: mmma cts/tests/autofillservice

Change-Id: Ia93ac4ff48910fed74a1f1539ce4e1f5a9db023f
2017-10-02 14:22:02 -07:00
Dan Willemsen ea655a138f Merge changes I7b2fc184,I71f6fcaf
* changes:
  Move generated aidl/logtags java classes to a subfolder
  Fix multiple output file generation for AAPT2
2017-10-02 19:54:15 +00:00
Colin Cross e673deb280 Delete jack support
Remove all support for running jack.

Bug: 65302138
Test: m -j checkbuild
Change-Id: I1ef87c88af3f2689f8b0dcf1a01c958b16159631
Merged-In: I1ef87c88af3f2689f8b0dcf1a01c958b16159631
(cherry picked from commit 5db5d31d73)
2017-10-02 11:15:19 -07:00
Dan Willemsen 4d99527dcd Move generated aidl/logtags java classes to a subfolder
So that they can't be generated into src/, which would be added to the
source list automatically.

Bug: 36698184
Test: m telephony-common without this change, pass
Test: m telephony-common without the cleanspec, fail with dup class
Test: m telephony-common with the cleanspec, pass
Change-Id: I7b2fc1847d52ff4a9ba8c69e76123fe152629ef8
2017-09-29 20:36:50 -07:00
Dan Willemsen ba9b0fbc05 Fix multiple output file generation for AAPT2
Make R.stamp actually part of the same rule that creates R.java.
Otherwise if the generated sources directory is removed, we don't
regenerate R.java / Manifest.java.

Also move the package-export.apk and proguard options file into the same
rule.

Test: m telephony-common; rm out/target/common/obj/*/*_intermediates/src; m telephony-common
Change-Id: I71f6fcaf20593ca23271d52d15537aaa9ef4cc77
2017-09-29 20:35:32 -07:00
Treehugger Robot a426e0e974 Merge "Respect LOCAL_MIN_SDK_VERSION for dx" 2017-09-29 20:13:08 +00:00
Treehugger Robot 4655de40c1 Merge "Use computed salt for AVB-signed images." 2017-09-29 19:44:03 +00:00
Tao Bao 2b6dfd6689 Use computed salt for AVB-signed images.
We want the generated images being identical for the same source files.
Currently the generated ext4 image (either from make_ext4fs or mke2fs)
is reproducible, but the AVB footer added by avbtool contain changes
because of the random salt being used.

This CL changes the avbtool invocation to specify "--salt <hexstring>"
(already supported by avbtool) to use reproducible salt that's computed
based on fingerprints (or thumbprints if applicable).

Bug: 67023482
Test: Regenerate images from the same source as follows:
  Use a target_files.zip from an AVB-enabled target.
    $ zip -d target_files.zip IMAGES/\*
    $ ./build/make/tools/releasetools/add_img_to_target_files.py \
          -v target_files.zip
  Repeat the above commands and compare the generated images.
Change-Id: Id9db17ae0132ca3a820b4be5a5ef06ca3fef71ed
(cherry picked from commit 8f05cca1d9)
2017-09-29 10:47:40 -07:00
Treehugger Robot 40ef25d7bd Merge "Apply vndk_package to generic system image" 2017-09-29 06:35:50 +00:00
Colin Cross 9deb331f20 Respect LOCAL_MIN_SDK_VERSION for dx
Follow Jack's behavior for now and pass LOCAL_MIN_SDK_VERSION to
dx.  Don't pass it to desugar for now, desugar with low
--min_sdk_version numbers enables features we haven't verified
yet, and existing apps won't be using the new language features
that require extra desugaring.

Test: examine classes.dex from an app with LOCAL_MIN_SDK_VERSION
Change-Id: Ic7d679b1b11fba98a33418a4f79d4762a91f1f9c
2017-09-28 21:40:35 -07:00
Mathieu Chartier e984c78faf Merge "Update paths for frameworks/base/config" 2017-09-28 23:43:46 +00:00
Mathieu Chartier d83a7f1fdb Update paths for frameworks/base/config
Bug: 64259751
Test: make

(cherry picked from commit 37ba2d97df)

Change-Id: Iac1f92d1e4126d088b691c46a0b858e9e9bfb1a4
Merged-In: Ie159ef9f3f96d7ecd22429327a64161bd63f525b
2017-09-28 17:54:20 +00:00
SzuWei Lin e5c62213c5 Apply vndk_package to generic system image
Add vndk_package into PRODUCT_PACKAGES, which includes
HAL interfaces, VNDK, VNDK-SP and LL-NDK.

Bug: 64654641
Bug: 65614701
Test: build pass
Change-Id: I8d8a02d88604d4287a519b698328a50dc11d819d
2017-09-28 15:26:44 +08:00