Commit Graph

34527 Commits

Author SHA1 Message Date
Tao Bao 4536e45f66 Merge "Support re-generating DTBO image from add_img_to_target_files.py." 2017-06-02 22:13:36 +00:00
Tianjie Xu 65b98cd3b1 Merge "Generate care_map.txt when AVB is enabled" 2017-06-02 18:09:58 +00:00
Tianjie Xu 6b2e155ee6 Generate care_map.txt when AVB is enabled
When AVB is enabled, generate care_map.txt and add it to the target
files. Also copy it into the OTA package where it will later be used
by the update_verifier.

Bug: 62208947
Test: \
1. Run add_img_to_target_files on the TF of a new pixel device,
and care_map.txt generates successfully.

2. Make dist in oc-dr1-release and find care_map.txt in the OTA package.

3. update_verifier succeeds in reading all the blocks on the care_map,
and fails to read out-of-bound blocks.

Change-Id: I2881711e6f87789cb7de150dbeca18b756fed68a
2017-06-01 21:36:43 -07:00
Tao Bao c633ed0230 Support re-generating DTBO image from add_img_to_target_files.py.
This is a step to enable signing a given target_files zip with release
keys.

When calling sign_target_files_apks.py, we will delete all the entries
under IMAGES/ in order to re-generate them (with the proper release
keys). In order to support that, we need to pack everything in need into
TF.zip.

Steps to test the CL.
a) Choose a target that has both AVB and DTBO enabled.
 $ m dist

b) Check IMAGES/dtbo.img and PREBUILT_IMAGES/dtbo.img both exist in the
   generated out/dist/TF.zip.

c) Remove the entries under IMAGES/ from the generated TF.zip.
 $ zip -d TF.zip IMAGES/\*

d) Re-generate the images with TF.zip.
 $ build/make/tools/releasetools/add_img_to_target_files.py TF.zip

e) Check that IMAGES/dtbo.img is re-generated, and it's identical to the
   image in b). Note that by default the re-generated image will carry a
   different footer, because of the random salt. This CL is verified by
   specifying the same salt.

Bug: 38315721
Test: see above.
Change-Id: I0bdc4e1cd4800962dc3902ca550dad6a8ca56c78
2017-06-01 20:07:33 -07:00
Treehugger Robot b5e90583ef Merge "Move math function definitions." 2017-06-02 00:11:04 +00:00
Treehugger Robot a46695c337 Merge "Fix make warning/error colors on Mac" 2017-06-01 08:24:49 +00:00
Treehugger Robot db5fa05429 Merge "Pack avbtool into otatools.zip." 2017-06-01 07:19:17 +00:00
Dan Willemsen 25f2d0ded7 Fix make warning/error colors on Mac
Apparently the default bash echo doesn't support \e even though the man
page says it does. So use \033 instead.

Test: m -j nothing on mac, get colors
Test: m -j nothing on linux, still get colors
Change-Id: I608fa87e3c28bf8f99264d39f9b250008a81235a
2017-05-31 19:20:09 -07:00
Tao Bao ba029141dc Merge "Remove the obsolete logic in img_from_target_files.py." 2017-05-31 23:18:15 +00:00
Steven Moreland a64f33668e Move math function definitions.
Created math.mk and moved all math function definitions there.
Then, included this file in config.mk. This allows the functions to be
used by whatever includes config.mk (envsetup and build).

Test: manually setting combinations of PRODUCT_SHIPPING_API_LEVEL
  and PRODUCT_FULL_TREBLE_OVERRIDE and looking at values of
  PRODUCT_FULL_TREBLE.
Bug: 62229856
Change-Id: Icdab4214a1e65ae202411613dbcb9c9ea5f43b09
2017-05-31 20:41:49 +00:00
Tao Bao 2bb109709a Remove the obsolete logic in img_from_target_files.py.
img_from_target_files.py used to handle the case that a given TF.zip not
containing the image entries under IMAGES/. That is only the case for
pre-Lollipop releases.

Also unzip the needed files only since we know that for sure now.

Test: img_from_target_files.py with an existing bullhead-TF.zip gives
      the same bullhead-img.zip.
Change-Id: I892379ba388df80ae63be9d3ce647fbb77fd4753
2017-05-31 11:17:56 -07:00
Tao Bao 3cba374da2 Pack avbtool into otatools.zip.
Also pack the test keys for easier testing.

Bug: 38315721
Test: m otatools-package and avbtool is present in otatools.zip.
Change-Id: Ieb63bf3f4bc211ef1f48ab278cb01b70845d06da
2017-05-31 10:20:25 -07:00
Treehugger Robot fe7aeb1de7 Merge "Support signing custom images with AVB HASH or AVB HASHTREE" 2017-05-31 03:24:25 +00:00
Tianjie Xu f7a76f9a30 Merge "Remove the obsolete UnpackPackageDir() in edify generator" 2017-05-31 01:08:15 +00:00
Treehugger Robot c6ad99c926 Merge "Makefile: Add the dependency for BUILT_OTATOOLS_PACKAGE." 2017-05-30 23:12:50 +00:00
Tianjie Xu 0996f815db Merge "Move recovery building from sign_target_files_apk to add_img_to_target_files" 2017-05-30 20:40:19 +00:00
Treehugger Robot 81787c1bbc Merge "Add blueprint_tools as default target" 2017-05-30 19:30:51 +00:00
Tianjie Xu 616fbeb144 Move recovery building from sign_target_files_apk to add_img_to_target_files
Currently we're building the boot/recovery image twice, which is
redundant. And b/38455129 shows a problematic case when the image
from two builds doesn't match. We should only build the recovery
image once in the add_img_to_target_files.

Bug: 62021378
Test: call sign_target_files_apk on an angler target file,
recovery-from-boot.p generates successfully; and SHA of recovery.img
matches the one in install-recovery.sh.

Change-Id: I01e033501d80c18a87cbb870300eee5c19a04441
2017-05-26 17:20:22 -07:00
Tianjie Xu ada0854169 Merge "Update the recovery files under SYSTEM/ if recovery patch is rebuilt" 2017-05-27 00:15:58 +00:00
Jeff Gaston 143f2b1fe5 Add blueprint_tools as default target
Bug: 38351765
Test: rm out -rf && m -j && stat out/soong/host/*/bin/bpfmt

Change-Id: I60ededa1b13fb575616162d6dc1b0b8159f97f20
2017-05-26 17:12:40 -07:00
Tianjie Xu 38af07ff20 Update the recovery files under SYSTEM/ if recovery patch is rebuilt
If we pass "rebuild_recovery" to add_img_to_target_files, the recovery
patch is rebuilt. But related files under SYSTEM/ (e.g.
SYSTEM/recovery-from-boot.p && SYSTEM/bin/install-recovery.sh) are not
updated.

This may cause a mismatch between system.img and SYSTEM/, and
may lead to a failure in validate_target_files.py.

Bug: 62096364
Test: Rebuild the system image in the TF and observe the recovery files
under SYSTEM/ get updated.

Change-Id: I7d679a612a86d02cf2eff81d1d120c0067138ed9
2017-05-26 22:40:23 +00:00
Treehugger Robot dddd7738d0 Merge "Update RenderScript build config and definitions." 2017-05-26 22:29:36 +00:00
Treehugger Robot 64caaf4679 Merge changes I73787aff,I0d149812
* changes:
  Move APP-% and PRODUCT-% implementation out of Kati
  Remove the rest of showcommands, dist goal
2017-05-26 22:08:49 +00:00
Miao Wang a9fed83fae Update RenderScript build config and definitions.
Bug: 38342163

  - obj/lib is deprecated and TARGET_OUT_INTERMEDIATE_LIBRARIES will be
  removed. We need to use per-module intermediates directory instead.
  - prebuilts/ndk/current folder is gone. For libm and libc, we just
  need ndk/r10 as stub libs for linking.

Test: mm and made sure this does not affect on device target.
Test: make -j50 FORCE_BUILD_RS_COMPAT=true RSTest_Compat works as
expected.

Change-Id: I8fbf5c10322707849a23c6b0dacc28b028db451e
2017-05-26 13:48:08 -07:00
Dan Willemsen 9bfcbc8a71 Move APP-% and PRODUCT-% implementation out of Kati
This way we'll share ninja files with the equivalent normal configs. We
were already parsing them in the make config step in order to fix
TARGET_PRODUCT / TARGET_BUILD_VARIANT / TARGET_BUILD_APPS, now they
replace themselves in MAKECMDGOALS with the appropriate goals.

If we're not going to pass any goals to ninja, pass the default goal.

Test: ALLOW_MISSING_DEPENDENCIES=true m -j APP-Calculator
Test: m -j PRODUCT-aosp_fugu-eng
Test: m -j PRODUCT-aosp_fugu-eng sdk
Test: m -j PRODUCT-aosp_fugu-sdk
Test: lunch aosp_arm64-userdebug; m -j
Test: lunch aosp_arm64-userdebug; m -j dist
Test: lunch aosp_arm64-userdebug; m -j adb
Change-Id: I73787aff9f74aed328e3fa75c571ae15a28851b0
2017-05-26 12:57:04 -07:00
Dan Willemsen 562baccae8 Remove the rest of showcommands, dist goal
showcommands is all handled in soong_ui, make/kati/ninja never see the
argument.

Remove the dist goal, since we'll never pass it to ninja, only to Kati
as a modifier.

Remove DUMP_%, since that's not used anywhere (it appears to be in our
code search via the NDK build system)

Test: m -j showcommands dist
Change-Id: I0d1498128caece685e98c2c2b5b0c3f545da0e11
2017-05-26 12:57:04 -07:00
Colin Cross 7bc6eb12f4 Merge "Pass TARGET_BUILD_PDK to soong" 2017-05-26 19:34:32 +00:00
Tobias Thierer bc8f82ff25 Merge changes from topic 'bug_38177295_checkbuild'
* changes:
  Let signapk access internal APIs under OpenJDK 9 toolchain
  Desugar: allow reflection over internal APIs.
2017-05-26 12:08:39 +00:00
Tobias Thierer 9cc3c76abd Let signapk access internal APIs under OpenJDK 9 toolchain
signapk relies on internal APIs sun.security.{pkcs,x509},
for example in com.android.apksig.internal.apk.v1.V1SchemeSigner.

This breaks at signapk runtime under OpenJDK 9 because those
packages are not exported by the java.base module.

This CL unbreaks signapk by allowing it to access these internal
packages. In the long term, signapk should migrate away from these
internal APIs (bug 37137869).

Test: make ANDROID_COMPILE_WITH_JACK=false checkbuild tests \
      && make checkbuild tests
      (with OpenJDK 8u45 toolchain on the PATH)
Test: make EXPERIMENTAL_USE_OPENJDK9=true \
      ANDROID_COMPILE_WITH_JACK=false checkbuild
      (with jdk 9-ea+170 toolchain on the PATH)

Bug: 37137869
Bug: 38177295
Change-Id: I64cab83e6eb7b135cf2ad7b523736cb409aaae02
2017-05-25 20:34:22 +01:00
Colin Cross 2ddbcdf3a7 Pass TARGET_BUILD_PDK to soong
Also enable ALLOW_MISSING_DEPENDENCIES when TARGET_BUILD_PDK is set
so that soong modules can reference modules that are disabled in the
PDK.

Test: builds
Bug: 62086238
Change-Id: Ic43e843a717b802ace0cee568b9e7e561a6c0868
2017-05-25 10:29:19 -07:00
Tobias Thierer 97479ccb7f Desugar: allow reflection over internal APIs.
Desugar reflects over internal APIs at runtime, using the
java.lang.invoke.MethodHandles.Lookup API.

On OpenJDK 9 toolchains, such reflection is only allowed
to packages to which the java.lang.invoke module is opened.

This CL adds an override to open the module to all unnamed
modules (i.e., to Desugar) when running Desugar.

Test: make checkbuild
      (with OpenJDK 8u45 toolchain on the PATH)
Test: make EXPERIMENTAL_USE_OPENJDK9=true checkbuild
      (with OpenJDK 9-ea toolchain on the PATH)

Bug: 38177295

Change-Id: I2cf74a96ea17366dd50b8d92af8e41e812247ef7
2017-05-25 18:14:27 +01:00
Tobias Thierer 5368131063 Merge "Drop build support for LEGACY_USE_JAVA7." 2017-05-24 10:24:53 +00:00
Bowgo Tsai 7ea994b21c Support signing custom images with AVB HASH or AVB HASHTREE
`make custom_images` supports to build different kinds of *non-droid* images,
e.g., odm.img. Adding the support of signing them with either AVB HASH footer
or AVB HASHTREE footer. The user can use HASH for small images and
HASHTREE for large images.

Sample signing configurations:
 * AVB HASH footer:
   - CUSTOM_IMAGE_AVB_HASH_ENABLE := true
   - CUSTOM_IMAGE_AVB_ADD_HASH_FOOTER_ARGS := --append_to_release_string my_odm_image

 * AVB HASHTREE footer:
   - CUSTOM_IMAGE_AVB_HASHTREE_ENABLE := true
   - CUSTOM_IMAGE_AVB_ADD_HASHTREE_FOOTER_ARGS := --fec_num_roots 8

 * Using custom signing key:
   - CUSTOM_IMAGE_AVB_ALGORITHM := SHA256_RSA2048
   - CUSTOM_IMAGE_AVB_KEY_PATH := external/avb/test/data/testkey_rsa2048.pem

Bug: 36701014
Test: `make custom_images` with AVB HASH footer
Test: `make custom_images` with AVB HASHTREE footer
Test: `make droid` to check system.img is still properly signed with AVB HASHTREE
Test: `make droid` to check vendor.img is still properly signed with AVB HASHTREE

Change-Id: I8dc420e12e37e9a631345c0cd883339db05d489f
2017-05-24 14:21:03 +08:00
Dan Willemsen 5f4274609f Merge "Fix override warnings when two test suites share an output" 2017-05-24 03:32:21 +00:00
Tao Bao e8ef8f70f8 Makefile: Add the dependency for BUILT_OTATOOLS_PACKAGE.
$ lunch aosp_marlin-userdebug
$ m otatools-package
$ echo "TEST" >> device/google/marlin/sepolicy/certs/app.x509.pem
$ m otatools-package

out/target/product/marlin/otatools.zip now contains an updated
app.x509.pem file.

Test: see above
Change-Id: I78a165e12a07c1b86a671a7ee1863a97fdea18c5
2017-05-23 18:25:29 -07:00
Dan Willemsen 988a08d009 Fix override warnings when two test suites share an output
Test: (internal master); m -j nothing, check warnings
Test: Spot check differences in build-*.ninja
Change-Id: I160583faa69d090f9b97a8a600598a39a8b1ed9c
2017-05-24 00:42:23 +00:00
Tianjie Xu 6a324ba588 Remove the obsolete UnpackPackageDir() in edify generator
package_extract_dir is used in file based OTA only and should be killed.

Bug: 37959785
Test: code search shows no usage of this function in aosp.
Change-Id: Id3719b969c24b7ecef0c7f0e4a3af09a72be54d4
2017-05-23 17:15:14 -07:00
Treehugger Robot 5d80b734f2 Merge "Move dataclean and installclean to soong_ui" 2017-05-23 22:32:34 +00:00
Treehugger Robot d5f8deb3ac Merge "Makefile: copy mke2fs.conf to ota root" 2017-05-23 19:45:37 +00:00
Tao Bao 78a41f6553 Merge "Fix the symlink detection when signing TF.zip." 2017-05-23 18:15:44 +00:00
Tobias Thierer c61bd998b5 Drop build support for LEGACY_USE_JAVA7.
This flag allowed for building with an OpenJDK 7 toolchain. It was
used for build bot builds that now work with an OpenJDK 8 toolchain.
Hence, this feature is no longer required. This CL drops it.

Bug: 27583810
Test: Treehugger build succeeds.
Change-Id: I7efca363d665292bfb88f4a4d6f3a120fc8f93f3
2017-05-23 14:17:48 +01:00
Tobias Thierer 48ee187e98 Merge "Fix fragile assumptions about build toolchain. (attempt #2)" 2017-05-23 12:14:54 +00:00
Tao Bao 406050bdb6 Fix the symlink detection when signing TF.zip.
We used to check for 'attr >> 16 == 0xa1ff' (i.e. 0o120777) to detect
symlinks in the input target_files zip (TF.zip). This becomes broken
after we switch to soong_zip, which packs symlinks with 0o120700.

This CL fixes the issue by using stat.S_ISLNK() instead.

Note that we don't need to stage the files with the exact permission
bits as in the input TF.zip. Because this part is covered by mkbootfs
by using the canned or the compiled-in fs_config - as long as the
files/directories are accessible and the symlinks are created.

Bug: 38455129
Test: sign_target_files_apks.py on bullhead TF.zip. Check the
      checksums in SYSTEM/bin/install-recovery.sh.
Change-Id: I51c1fc9a257fb3f18c16c2ed71528abaa6f7d9c9
2017-05-22 23:33:11 -07:00
Treehugger Robot 9f52f0d17d Merge "Add LOCAL_JAR_PROCESSOR" 2017-05-23 02:03:09 +00:00
Colin Cross f8a49c5a37 Add LOCAL_JAR_PROCESSOR
Allow using a host java library as an intermediate processor step
after javac but before desugar, proguard, or dx.

Test: manual
Change-Id: I7fb843e10d96c0167f6e4371582fabcc1454521f
2017-05-22 17:30:13 -07:00
Tobias Thierer 0ed47f7fd1 Fix fragile assumptions about build toolchain. (attempt #2)
This CL resubmits a fixed version of commit
e2a8da2683.

This CL applies the following changes:

- explicitly set the path where desugar will dump temporary class files.
  This ensures that the system property is set before it is read during
  InnerClassLambdaMetafactory.<clinit>. Before this CL, the system
  property was set by Desugar.createAndRegisterLambdaDumpDirectory(),
  which may run too late.

- explicitly specify -source 1.8 for droiddoc's javadoc run.
  Previously, the command used the language version of the build
  toolchain, which might fail due the backward-incompatible restrictions
  imposed by the proposed module system.

  Some Android build targets use LOCAL_JAVA_LANGUAGE_VERSION 1.7,
  but droiddoc combines sources from multiple build targets and there
  where no backwards incompatible changes in 1.8, so it should be fine
  to use 1.8 for code from either language level.

Bug: 38318052
Bug: 38225656
Bug: 38177295
Test: make clean && make ANDROID_COMPILE_WITH_JACK=false checkbuild tests \
      && make checkbuild tests
      (using OpenJDK 8 toolchain)

Change-Id: I2fffe6668747f48be44e34c67332af9b8a996d2a
2017-05-22 23:03:24 +01:00
Tobias Thierer 00e3bd93da Merge "Make: Don't rely on presence of tools.jar in toolchain." 2017-05-22 09:33:31 +00:00
Tobias Thierer 3c4c0b6ab3 Make: Don't rely on presence of tools.jar in toolchain.
Some cts, doclava and javassist code uses private APIs that require
tools.jar to be on the classpath when building with the standard
OpenJDK 8 toolchain. However, OpenJDK 9 toolchains do not have this
file nor require it to be on the classpath. The reliance on the
presence of tools.jar means that AOSP can currently not build on
such toolchains.

After this CL, when EXPERIMENTAL_USE_OPENJDK9 is set, the value of
HOST_JDK_TOOLS_JAR is empty.

That flag can be set via:

export EXPERIMENTAL_USE_OPENJDK9=true

and unset via:

unset EXPERIMENTAL_USE_OPENJDK9

Test: make checkbuild (using OpenJDK 8 toolchain)
Test: manually confirmed that device boots
Bug: 38177295
Change-Id: Icc5451b51e44a574fd21024d434ed1fa8711de17
2017-05-20 14:17:05 +01:00
Treehugger Robot ec0dec8599 Merge "Add soong variable for PRODUCT_FULL_TREBLE." 2017-05-20 00:05:51 +00:00
Treehugger Robot 1c7debc95c Merge "Use installed module stem for .so soname" 2017-05-19 23:34:58 +00:00