Commit Graph

6600 Commits

Author SHA1 Message Date
Tao Bao ae3a9a138d Merge "releasetools: Fix releasetools_py3_test." 2019-10-07 19:06:07 +00:00
Tao Bao 615b65d097 releasetools: Fix releasetools_py3_test.
Test: Use a lunch'd target;
      `atest --host releasetools_test releasetools_py3_test`
Change-Id: Ibcced83a7989ef8fc15464728ab1d5fa71ca988e
2019-10-06 23:29:34 -07:00
Tao Bao a264feffe8 releasetools: Update tests to match SparseImage change.
The previous change in commit 22632cc82c
changed the behavior in loading file map. It now always puts a copy of
the input text in `extra` field. Update the tests accordingly.

Bug: 79951650
Test: Use a lunch'd target; `atest --host releasetools_test`
Change-Id: Iccf06c817c1305bf9946d7759c6f6f6af21fe85e
2019-10-06 22:21:14 -07:00
Tao Bao 99d855db8f Merge "releasetools: Support verifying files with non-monotonic ranges." 2019-10-07 01:33:30 +00:00
Tao Bao 22632cc82c releasetools: Support verifying files with non-monotonic ranges.
Fixes: 79951650
Test: Run validate_target_files on target_files.zip with files in
      non-monotonic ranges.
Test: python -m unittest test_validate_target_files
Test: python3 -m unittest test_validate_target_files
Change-Id: I82571d3358598775de4cdeb5e64035689fea6487
2019-10-03 23:21:22 -07:00
Bill Peckham e868aec14b Moving recovery resources from /system to /vendor
This change is part of a topic that moves the recovery resources from the
system partition to the vendor partition, if it exists, or the vendor directory
on the system partition otherwise. The recovery resources are moving from the
system image to the vendor partition so that a single system image may be used
with either an A/B or a non-A/B vendor image. The topic removes a delta in the
system image that prevented such reuse in the past.

The recovery resources that are moving are involved with updating the recovery
partition after an update. In a non-A/B configuration, the system boots from
the recovery partition, updates the other partitions (system, vendor, etc.)
Then, the next time the system boots normally, a script updates the recovery
partition (if necessary). This script, the executables it invokes, and the data
files that it uses were previously on the system partition. The resources that
are moving include the following.

* install-recovery.sh
* applypatch
* recovery-resource.dat (if present)
* recovery-from-boot.p (if present)

This change includes the platform build system and release tools changes to
move the recovery resources from system to vendor (or /system/vendor). The
release tools need to know where to generate the recovery patch, and they
discover this from misc_info.txt variable board_uses_vendorimage, which the
platform build system generates.

We remove applypatch from PRODUCT_PACKAGES, but it is added back as a required
module in target/product/base_vendor.mk.

Several release tools rely on the misc_info.txt board_uses_vendorimage variable
to know how to generate and detect the recovery patch.

This change partially removes the --rebuild_recovery flag from the
merge_target_files.py script. The flag will be fully removed in a follow-on
change.

Bug: 68319577
Test: Ensure that recovery partition is updated correctly.
Change-Id: Ia4045bd67ffb3d899efa8d20dab4c4299b87ee5f
2019-10-04 00:04:34 +00:00
Treehugger Robot 1623ec97f1 Merge "add vendor_boot image support" 2019-09-27 21:07:24 +00:00
Steve Muckle e1b1086ef3 add vendor_boot image support
The vendor_boot partition is generated by mkbootimg and contains all the
device-specific information that used to reside in the boot partition.

Bug: 137297791
Change-Id: I5b005097b73f59857c3a2f92d693b3e67ee8424e
2019-09-27 10:26:00 -07:00
Chih-Hung Hsieh 83980501d9 Fix severity of analyzer warnings; add new warnings.
* The following were high severity warnings but mistakenly
  grouped into analyzer warnings in a recent change.
        clang-analyzer-unix.Malloc
        clang-analyzer-cplusplus.NewDeleteLeaks
        clang-analyzer-cplusplus.NewDelete
* Add new Protoc and Kotlin warnings about unused imports and variables.

Bug: 141493530
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: Ied1f9ace97f29fba58a40dff2346a0391ab22f06
2019-09-26 12:14:32 -07:00
Chih-Hung Hsieh 790020437d Factor out common code in Java warn patterns.
* Some comments are kept to reduce diff artifacts.
* Warning patterns will be separated by tools later,
  to make it easier to share code with Chromium and
  other build tools.

Bug: 141493530
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: I6b589478cf5ae73940b4712d9faabd0909546f02
2019-09-25 13:39:41 -07:00
Chih-hung Hsieh 415186a71c Merge "Add new compiler warning patterns" 2019-09-25 17:00:36 +00:00
Paul Duffin e19b7471ae Merge "Replace use of conscrypt-nojarjar with conscrypt-unbundled" 2019-09-25 09:58:56 +00:00
Chih-Hung Hsieh d591a5e012 Add new compiler warning patterns
* Add Kotlin, NDK, Protoc, Asm, and other new patterns.
* Put all clang-analyzer warnings to the ANALYZER severity group.
  Some security and null pointer warnings still have HIGH severity.
* Comment out old C/C++ warnings that overlap with analyzer
  warnings and do not appear in current build.log.
* Other old fixed warning patterns should be commented out later.

Bug: 141493530
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: Idc7b09e5e4eefa127d41e9f37da1f566ed1ede1f
2019-09-24 14:52:09 -07:00
Chih-hung Hsieh d5ea80b388 Merge "Handle rustc warning messages." 2019-09-23 16:53:38 +00:00
Tao Bao 08f9da1898 Merge "Don't generate hashtree when signing bundled APEXes." 2019-09-23 14:22:59 +00:00
Paul Duffin ebbcc1bb78 Replace use of conscrypt-nojarjar with conscrypt-unbundled
Bug: 141346785
Test: m checkbuild
Change-Id: Ibcd4aac938e0fdf5765ad8426fe64c6d053f3529
2019-09-23 14:30:07 +01:00
Chih-Hung Hsieh a1187072c5 Handle rustc warning messages.
* Combine the first two lines of rustc warning messages.
* Give some rustc warnings HIGH and MEDIUM severity.

Bug: 141311294
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: Ia3a206d68d9e92934296487b006a7b424044a822
2019-09-20 17:14:22 -07:00
Tao Bao 448004af9d Don't generate hashtree when signing bundled APEXes.
Bug: 139957269
Test: Sign a target_files.zip. Extract a re-signed APEX and check the
      hashtree size (being zero).
Test: Use sign_apex to sign an APEX file. Check the hashtree size (not
      being zero).
Test: python -m unittest test_apex_utils
Test: python -m unittest test_sign_apex
Change-Id: I927b7681d66920d7732b700ec3a8f7a65b4cb351
2019-09-19 14:41:34 -07:00
Treehugger Robot c4f6f36dc4 Merge "Creates img.zip from target files zip, not dir." 2019-09-19 01:03:33 +00:00
Daniel Norman 74eb74b146 Creates img.zip from target files zip, not dir.
Directory support for img_from_target_files was dropped in aosp/1095755.

Test: Create & boot a merged build.
Change-Id: Ib976ea60a24c2f172c597130d6bcce25199a93b3
2019-09-18 14:03:34 -07:00
Tao Bao 3e759462d9 releasetools: Remove more images out of secondary payload.
This CL additionally removes boot/dtbo/modem/vbmeta_* images out of
secondary payload. We essentially only keep system_other.img and
bootloader images there.

For Pixel devices, this additionally saves ~80MiB (mostly because of
the removal of boot and radio images).

Bug: 140771390
Test: ota_from_target_files \
          --include_secondary --skip_postinstall \
          -i input-target_files-1.zip \
          input-target_files-2.zip \
          output-ota.zip
Test: python -m unittest test_ota_from_target_files
Change-Id: If47b27c52b3547a4cc86223a988c53960bc6af40
2019-09-17 22:56:19 -07:00
Tao Bao d5b6094ed3 Merge "releasetools: Sanity check the build fingerprint." 2019-09-17 18:30:45 +00:00
Tao Bao e430f7652a Merge "Also install verity_key to ramdisk for non-system-as-root target." 2019-09-17 18:25:55 +00:00
Yifan Hong 7ca2c1539f Merge "extract-kernel: Fix indexing" 2019-09-17 16:50:36 +00:00
Tao Bao c99819311c Also install verity_key to ramdisk for non-system-as-root target.
The commit in d14b895665
(https://android-review.googlesource.com/c/platform/build/+/728287)
changed partition layout, to always build the root dir into system.img,
even for devices not using system-as-root (i.e. the ones with separate
boot ramdisk).

With the new layout, there will be two root dirs for non-system-as-root
targets during the boot. If such a device uses Verified Boot 1.0,
/verity_key needs to be available in both roots, to establish the chain
of trust.
 - bootloader uses the baked-in key to verify boot.img; it then loads
   the ramdisk from the verified boot.img
 - First stage init uses /verity_key (in ramdisk) to verify and mount
   system.img at /system, then chroot's to it
 - Second stage init uses /verity_key (in system.img) to verify and
   mount other partitions

This CL adds rules to additionally install verity_key into ramdisk for
such targets.

Bug: 139770257
Test: Set up a target to use non-system-as-root
      (BOARD_BUILD_SYSTEM_ROOT_IMAGE != true). `m dist`.
Test: Check that both ROOT/verity_key and BOOT/RAMDISK/verity_key exist
      in the built target_files.zip.
Test: Run validate_target_files to validate the above target_files.zip.
      $ validate_target_files \
          --verity_key_mincrypt /path/to/verity_key \
          target_files.zip
Test: Run sign_target_files_apks to sign the above target. Re-run
      validate_target_files on the signed target_files.zip.
Test: python -m unittest test_validate_target_files
Change-Id: Ibe7e771c8c376429add85851ac86055564765d3c
2019-09-17 08:53:16 -07:00
Tao Bao c4011cd719 releasetools: Sanity check the build fingerprint.
Bug: 140298338
Test: python -m unittest test_ota_from_target_files
Change-Id: I4add5f82d5b684c441b0484732da6991f8e7301a
2019-09-17 08:43:54 -07:00
Yifan Hong 8b72776511 extract-kernel: Fix indexing
The original script always tries to extract from index
0 even if header is found in positive indices. Fix that.

Also, continue to try other positions if previous run failed.

Test: builds
Bug: 139348603

Change-Id: Ia54fc709de9ae587fc64b59d94a1fa4ae669c14f
2019-09-16 16:30:35 -07:00
Tianjie Xu 43a049f814 Merge "Remove vendor & product in secondary payload" 2019-09-12 17:40:22 +00:00
Tianjie Xu 1c8080098a Remove vendor & product in secondary payload
We used to carry the full OTA for vendor and product partitions when
generating the secondary payload. Since they are not actually used, we
can remove them to reduce the size of factory OTA packages. For example,
the size for a sargo package reduces from 454M to 283M after the change.

Bug: 140771390
Test: generate and apply a second payload on Pixel
Change-Id: I091dc07db8c345b182d9f5dd8d48f5c209245641
2019-09-11 15:23:41 -07:00
dianlujitao 5dcbe701cf fs_config: Fix generation for devices without vendor partition
If the "--all-partitions" argument is empty, the system fs_config should
be generated with all partitions emitted.

Fixes: 8453f02c (fs_config: Fix cases without vendor/oem partition)
Change-Id: I058cfd1c8e665b40757998e773c35e26465bf58a
Signed-off-by: dianlujitao <dianlujitao@lineageos.org>
2019-09-10 16:17:47 +08:00
Yifan Hong 708cbdbd41 Merge "releasetools: add dep to checkvintf"
am: fd4a3de77d

Change-Id: Ie5ffcd9b96de64129555980d77d851f7672f2083
2019-09-04 17:53:30 -07:00
Yifan Hong 09298552e2 releasetools: add dep to checkvintf
'required' only works on python_binary_host.
Test: modify checkvintf source code, build ota_from_target_files

Change-Id: Ieac68b39b1ccabae679a1679f184cb3aeaaeefb6
2019-09-04 12:37:46 -07:00
Tao Bao bce0264348 Merge "releasetools: img_from_target_files uses zip2zip."
am: f833a75283

Change-Id: I1731934e5585185167db9638c7b6652871fbf913
2019-08-30 12:29:42 -07:00
Tao Bao f833a75283 Merge "releasetools: img_from_target_files uses zip2zip." 2019-08-30 19:13:59 +00:00
Tianjie Xu 8b106fd2bc Merge "Add a new flag "--disable_fec_computation" for OTA generation"
am: 9c9993c706

Change-Id: Ic077215542e20c2bc9fbaaaa2fa602f15d3f4a87
2019-08-29 13:27:20 -07:00
Tianjie Xu 9c9993c706 Merge "Add a new flag "--disable_fec_computation" for OTA generation" 2019-08-29 20:05:41 +00:00
Tianjie Xu 1b07983c6d Add a new flag "--disable_fec_computation" for OTA generation
We have added the flag to disable on device fec computation in
brillo_update_payload. Exposed it in ota_from_target_files as well.

Bug: 139723500
Test: generate an incremental package with the new flag
Change-Id: I683e3ec7791963a3c7e141ecb4e5a3fb53fe2108
2019-08-28 21:38:05 -07:00
Yifan Hong ee276acce8 Merge changes from topic "vintf_ota_delete"
am: bcba1cfdaf

Change-Id: If11df7c409b9d1087defc119bf3a132f5b7016c5
2019-08-28 17:21:36 -07:00
Yifan Hong a592de9c61 Check VINTF compatibility at OTA generation time.
am: 9276cf0226

Change-Id: Iee0efa785ccbf2aaadb1be7b3dc106f805242adb
2019-08-28 17:21:27 -07:00
Yifan Hong 96961bf94f releasetools: add tests for check_target_files_vintf
am: ccb86fe7d4

Change-Id: I3ecddd78ccce7f97bd02abea133ad84187b312ac
2019-08-28 17:21:07 -07:00
Yifan Hong d14a6110c5 Add a script to check VINTF compat of target files package.
am: e3ba82cff2

Change-Id: Iccf4b7afa93749f441f48a3270f3d4beac8a0f06
2019-08-28 17:20:52 -07:00
Tao Bao 57f8ed6b5e releasetools: img_from_target_files uses zip2zip.
Test: Build the following targets and check the built image archive.
      bonito-userdebug (contains flashable images);
      cf_x86_phone-userdebug (contains super.img);
      aosp_arm64-eng (contains VerifiedBootParams.textproto)
Test: m -j otatools-package; Check that zip2zip is included; Use
      bin/img_from_target_files to generate image archive.
Change-Id: I9f28765bd702849f0df309faebd6aa3072920536
2019-08-28 16:36:25 -07:00
Yifan Hong bcba1cfdaf Merge changes from topic "vintf_ota_delete"
* changes:
  merge_target_files: Checks VINTF metadata of resulting target files
  Check VINTF compatibility at OTA generation time.
  releasetools: add tests for check_target_files_vintf
  Add a script to check VINTF compat of target files package.
2019-08-28 23:19:21 +00:00
Yifan Hong 9276cf0226 Check VINTF compatibility at OTA generation time.
Instead of relying on META/{system,vendor}_{manifest,matrix}.xml
and blindly copy compatibility.zip from target files package to
OTA package, do a static check on the input target files package
before generating the OTA package.

META/{system,vendor}_{manifest,matrix} does not contain ODM manifest
fragments, which is incorrect.

Also, the on-device check of compatibility.zip uses a old libvintf
binary on the device, which may not understand the incoming VINTF
metadata. This change removes the on-device check. Hence, it removes
the requirement of forwards compatibility of libvintf.

This behavior can be skipped with --skip-compatibility-check.

Test: build OTA package
Bug: 139300422
Bug: 131425279
Change-Id: I7fb93be9eb73f578fc05a182c6c9d1f073db2800
2019-08-28 11:02:26 -07:00
Yifan Hong ccb86fe7d4 releasetools: add tests for check_target_files_vintf
Test: run it
Bug: 139300422
Change-Id: I265d1e6313177dc661bb5cdf1415b11684dec0a0
2019-08-28 11:02:26 -07:00
Yifan Hong e3ba82cff2 Add a script to check VINTF compat of target files package.
Instead of checking META/{system,vendor}_{manifest,matrix}.xml (which
is error-prone because ODM SKU-specific manifests are not checked),
this script read the target files package, remaps its directory
structure so that checkvintf understands it, and check VINTF
compatibility.

Also, put it in otatools.zip.

Test: run it on an extracted target files package
Bug: 131425279
Change-Id: I06036f9a8d7242d4bc11524028be40e780c508e8
2019-08-28 11:02:26 -07:00
Yifan Hong ade0d3f4a2 merge_target_files: Checks VINTF metadata of resulting target files
Bug: 139300422
Bug: 131425279
Test: test_merge_target_files
Change-Id: I972db2fc40555ab0fc38316f0a6e6f5e8f163371
2019-08-28 11:02:26 -07:00
Tao Bao 3a85b1a466 Merge "releasetools: Clean up img_from_target_files.py."
am: 1f8e8738c1

Change-Id: Ic5718fd194395f86f99da41791ed9c909f7638cd
2019-08-27 16:45:09 -07:00
Tao Bao ac63a9d728 releasetools: Clean up img_from_target_files.py.
Bug: N/A
Test: Run img_from_target_files to generate an image archive.
Change-Id: I802640b88fe0c7a6d1aab63411b8194d61bceaba
2019-08-26 20:33:11 -07:00
Daniel Mentz 1e8961033e Merge "Consider search_path to locate avb_*_key_path files"
am: 11dc247ba9

Change-Id: Ie0479fe88e37b330bc80fb6cb68d906546c70b0a
2019-08-26 12:21:24 -07:00