Commit Graph

6718 Commits

Author SHA1 Message Date
Steve Muckle f84668e1da Enable non-v3 recovery image with v3 boot image
Add support for building a non-v3 recovery image along with a
v3 boot.img and vendor_boot image.

Bug: 150825361
Change-Id: I80d4e71ff09b83dd1c7f9accb4de99a30f95f4bf
2020-04-14 13:03:58 -07:00
Jaewoong Jung e57fb1a3aa Merge "Add signing certificate lineage file support." 2020-04-07 22:33:25 +00:00
Jaewoong Jung 95445e6913 Add signing certificate lineage file support.
Also add multi-cert support to prebuilt apps so that they can benefit
from the new lineage feature.

(This is a cherry-pick change.)

Test: m GoogleServicesFramework w/ modified build rules
Test: m PrebuiltGmsCore w/ modified build rules
Test: apksigner lineage -v --print-certs -in <built_module_path>
Fixes: 152897457
Change-Id: If7d5d4bd308629c8340231520214c76c8a568a65
Merged-In: If7d5d4bd308629c8340231520214c76c8a568a65
2020-04-06 13:42:23 -07:00
Bill Peckham f6baf822ac Merge "Ensure that merge_target_files.py prints INFO level logs." 2020-04-06 19:00:56 +00:00
Tianjie Xu 0889aaeeae Merge "Change the parameter type of offset in read" 2020-04-04 20:00:20 +00:00
Bill Peckham cb84817fdf Ensure that merge_target_files.py prints INFO level logs.
Bug: 153166596
Test: Run a merge build, inspect logs.
Change-Id: Ic2795f26922db32030c7ab7eb4ec6f153c1e9654
2020-04-04 04:20:45 +00:00
Bill Peckham 5c7b034a5c Make the `partition=` tag optional.
Since we might use ToT release tools to sign a package
generated by an older build, we make the new
`partition=` tag optional. This also means we need to be
careful to use non-greedy regex matching.

Bug: 153133823
Test: python3 -m unittest
Test: input with and without the new `partition=` tag
Test: new test_ReadApkCerts_WithWithoutOptionalFields
Change-Id: Ic57efd34e745ad302ae17150c6f2318f0b4524cb
2020-04-03 17:09:37 -07:00
Tom Cherry 8c7a4f6fc1 Merge "Include private/fs_config.h directly when needed" 2020-04-03 14:36:58 +00:00
Tianjie Xu 2995096483 Merge "Add aftltool is a signing parameter" 2020-04-02 18:09:59 +00:00
Tianjie 2751d2b8f3 Change the parameter type of offset in read
As we support zip64 in libziparchive, we change the parameter of
the ReadAtOffset() in zip reader from uin32_t to off64_t. So the
derived class needs to be updated as well.

Bug: 150900468
Test: build
Change-Id: Icbfd2dd54b01ff62da988ba8598c1057f9bb6368
2020-04-02 06:16:43 +00:00
Kiyoung Kim 39197f846b Merge "Build required modules from default" 2020-04-02 06:14:43 +00:00
Tianjie 0f3074566c Add aftltool is a signing parameter
Right now we assert that the aftltool should exist since the one
in aosp won't work due to grpc dependencies.

Bug: 147870995
Test: build
Change-Id: Iabb2c375167572a965493a7648fdc1abe287af67
2020-04-01 12:20:21 -07:00
Kiyoung Kim 0a8d26210e Build required modules from default
Some required modules from host tool (such as imgdiff) are bound to
required from releasetools_common, but unfortunately required modules
from python_library_host does not work (b/140144201). To make this work
we should mark dependency from each host tool.

Bug: 152258157
Test: Build succeeded without system image
Change-Id: Iedfda9b48d78687d8f622b818eeb93896d4f119d
2020-04-01 14:21:01 +00:00
Tianjie Xu a3541df291 Merge "Add aftl inclusion proof" 2020-03-31 22:12:46 +00:00
Bill Peckham 19c3feb2d3 Build merged apexkeys.txt/apkcerts.txt by partition.
Propagate partition tag data to apexkeys.txt and
apkcerts.txt so that merge_target_files.py can build
merged versions of these files by filtering the
framework files for framework partitions and filtering
the vendor files for vendor partitions.

Bug: 138942268
Change-Id: Ic3226728e97dae84d38ec230ccc86d1b124bea94
2020-03-30 22:33:27 +00:00
Baligh Uddin 639b3b7a65 Continue to pass in --signing_helper for avbtool.
BUG: 152424931
Test: Tested Locally
Change-Id: I4e93a899b9055c937980a496f20b35c01ccbb4c1
2020-03-29 22:05:27 -07:00
Treehugger Robot 8b3c9a1b40 Merge "releasetools: Add missing log format specifier" 2020-03-27 22:26:42 +00:00
Luca Stefani 88e1a14523 releasetools: Add missing log format specifier
Test: m otapackage
Change-Id: I273cf315400dd2b624d2853ece0c30c6fb3d529d
2020-03-27 13:07:56 +00:00
Luca Stefani 3e4b44868b releasetools: Add signapk as required modules
* common.py requires the signapk jar

Test: m otapackage ( before it was broken )
Change-Id: Ic0cff0a21258cb1e7d1ced5553a940dcf5cd6d04
2020-03-27 13:05:21 +00:00
Baligh Uddin b1ad7c6dc7 Merge "Build the payload image only during apk-in-apex signing" 2020-03-24 04:54:29 +00:00
Tianjie c180a5d222 Build the payload image only during apk-in-apex signing
When doing apk-in-apex signing, the package name is not reserved
during the apex repacking. As a result, the name accidentally
reverts to 'com.android.wifi' from 'com.google.android.wifi'.

This cl changes the behavior to call 'apexer' by passing the
'--payload_only' argument. So we don't build the apex file from
scratch and the old AndroidManifest.xml will be reused.

Test: 152084536
Bug: unit tests pass
Change-Id: I8332b2ee84832fb196f2e1c4309abac5ab92e153
2020-03-23 18:15:34 -07:00
Tianjie Xu eaed60c1a1 Add aftl inclusion proof
The otatools should talk to the aftl server and append the inclusion
proofs when building the vbmeta image. We should only do this during the
signing process when the network is always available.

Also the inclusion proof doesn't impact the final vbmeta image size on
coral, where the final size is 8192 after padding. This is below the
physical image of 65536.

Bug: 147870995
Test: unit tests pass, run sign_target_file_apks
Change-Id: If84c6bf5198c9b05f5e0c16ae6335971915f47e3
2020-03-18 13:11:41 -07:00
Yifan Hong 69430e65fe Also check empty SKU if SKUS are not defined
In the case when both ODM_MANIFEST_SKUS and ODM_MANIFEST_FILE are
both empty, also checks VINTF compatibility for empty SKU case.

Same for device manifest. Although right now, DEVICE_MANIFEST_FILE and
DEVICE_MANIFEST_SKUS cannot be both empty (because libvintf requires
vendor manifest exists barring manifest fragments), let libvintf throws
an error for this case. In the build system, simply handle vendor manifest
the same way as ODM manifest.

Bug: 148601741
Test: manual
Change-Id: Ia0655fd4d37f6286b164a3221ec4b2716f29147d
2020-03-17 20:27:32 -07:00
Yifan Hong 28ffd73591 Proper checks for vendor and ODM SKUs
If DEVICE_MANIFEST_FILE / ODM_MANIFEST_FILES is set, also include check
for empty vendor / odm SKU, respectively. Then, do cross product of
vendor SKUs and ODM skus, and check each possible combination.

Test: build with and without vendor skus
Test: m check-vintf-all
Bug: 148601741

Change-Id: I23cc81998e7afc36c43540bf6d615b4d4b288447
2020-03-13 18:32:07 -07:00
Xindong Xu 2a7aaa69ae ota_tool: fix incremental ota error
Problem:
Logical partitions do not update right since commit
f67dd8059d

    Generate block based OTA for product partitions

due to the wrong order of parameters. This is logged in
/tmp/recovery.log:

    skipping 0 blocks already patched to 202

Bug: 149794632
Change-Id: I92f7a8a226a8a96b892254362d362c515bfb6dc6
2020-03-13 14:25:35 +01:00
Tom Cherry 7cff1fa1fc Include private/fs_config.h directly when needed
This is no longer a transitive include of android_filesystem_config.h

Bug: 149785767
Test: build
Change-Id: Ib546dbc3b6f546ba48009cc9d6d0f4564acf0872
2020-03-05 10:55:45 -08:00
Daniel Norman d5fe862628 Uses a per-partition fingerprint for building images and avb_salt.
This causes the output image files of a merged build to be identical
to the image files of the input partial builds, for each images in
PARTITIONS_WITH_CARE_MAP.

Test: python -m unittest test_common
Test: `m dist`; `unzip out/dist/target_files.zip IMAGES/\*`;
      `zip -d out/dist/target_files.zip IMAGES/\*`
      `add_img_to_target_files -a out/dist/target_files.zip`.
      Verify that the rebuilt images are identical to the deleted ones.
Test: Build a merged target (using merge_target_files.py). Verify that
      the partial target-files.zip IMAGES are identical to the merged
      target-files.zip IMAGES for PARTITIONS_WITH_CARE_MAP images.
Bug: 150405807
Change-Id: I5fdf5783c1aff9c14cf5408090389b1f65b69ca6
2020-03-02 19:05:20 +00:00
Chih-Hung Hsieh 3cce2bcc60 Separate html/csv output functions into html_writer.py
* also suppress some pylint warnings

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I497dbb7496ca21470ba33db03eedb27f5e8f1e96
2020-02-27 15:39:18 -08:00
Tianjie Xu 223d055768 Merge "Preserve the monotonic property when subtracting 0 block" 2020-02-27 21:22:23 +00:00
Tianjie Xu f02ececc7f Preserve the monotonic property when subtracting 0 block
The monotonic property of the range is lost after the recent change to
subtract 0th block from the file range. As a result, we read the block
in the wrong order when pull files from the raw image. This yields a
incorrect hash during validation.

Bug: 150334561
Test: run validate_target_file. unittests
Change-Id: Id4fedaaf273c42c8d21a4549fda4c9a45e22066a
2020-02-26 21:50:47 -08:00
Chih-Hung Hsieh 445ad810c8 Recognize more warning patterns.
* add more Java, Kotlin, and Rust warning patterns
* recognize more clang-tidy cert-* and bugprone-* warnings;
  still keep the catch-all 'cert' and 'bugprone' groups.

Test: warn.py build.log > warnings.html
Change-Id: I909ec09822f42f60493ebe943c055bf0d4b69938
2020-02-26 14:34:21 -08:00
Chih-Hung Hsieh 5ae5519661 Use same warn_common.py and warn.py for Android and Chrome
* add chrome_project_list.py
* use newer Chrome's warn_common.py and severity.py
* modify warn_common.py to work with both python2 and python3,
  to work with and without google3

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I9aeee7031c11988d8ead1733ff4846ae6cb1ee73
2020-02-25 19:47:35 -08:00
Bowgo Tsai 2fe786a528 Support removing AVB public keys
AVB public keys might be installed into the first-stage ramdisk.
This CL supports removing some AVB keys during the signing process,
which can be useful when a key needs to be revoked.

Also correcting the path of force_debuggable detection.

Bug: 150109393
Test: make dist
Test: ./build/tools/releasetools/sign_target_files_apks \
      --remove_avb_public_keys key1.avbpubkey,key2.avbpubkey \
      out/dist/*-target_files-*.zip signed-target_files.zip
Change-Id: I1af95adbbec655f40121f3392965dd1950aca4e5
2020-02-24 17:53:10 +08:00
David Anderson ab9c7e6d04 Exclude holes from the block map.
e2fsdroid can export holes as a "0" block, in order to make sure logical
and physical block assignments are maintained properly. Exclude these
hole blocks from range sets in LoadFileBlockMap.

Bug: 139201772
Test: m dist on non-A/B
Change-Id: Ic04e869572229678fdeaaa6fbf6f464d6c672051
2020-02-21 17:04:00 +00:00
P.Adarsh Reddy 8b2e7c9c51 Make oemaids_headers available to vendor.
This change makes the oemaids_headers header library
available to vendor side.

Bug: 149665231
Test: Tested compilation, oemaids_headers is available to
a sample vendor binary.

Change-Id: I87369f1db883e40e83d6076defdd4c0d84aac46f
2020-02-18 09:44:23 +00:00
Tamas Petz c0a8c63935 build_image: check value of "ext4_share_dup_blocks"
Regardless of the value of "ext4_share_dup_blocks" "-c" flag is passed
unconditionally. The correct behavior is to check if
"ext4_share_dup_blocks" is set to "true".

Test: 1. build bonito with "BOARD_EXT4_SHARE_DUP_BLOCKS := false"
      2. check for "shared_blocks" feature of system.img
         ./out/host/linux-x86/bin/tune2fs -l \
         ./out/target/product/bonito/system.img | \
         grep -o shared_blocks

Change-Id: I8c0e5606658dd0f47b71cfc6c453a77a57da4f42
2020-02-10 11:45:06 +01:00
Tianjie Xu 9bd832a012 Remove the key parameter when verifying avb images in validate_target_files
If a key is specified, the avbtool always use the input key to verify
all the chained images. And this will cause failures when the vbmeta &
system use different keys (e.g. RSA 4096 vs RSA2048). Because the public
key to vbmeta will always fail to verify the system image. Remove the
'--key' parameter in the verification command, so the avbtool will use
the embedded public in the image.

Test: validate target-file from sdk_gphone_x86_64
Bug: 148916990
Change-Id: I9d31be0f8c32af605af94fa73d07818f40f51ec4
2020-02-06 13:31:26 -08:00
Chih-Hung Hsieh ed74896600 More Java/Kotlin warning patterns
* Simplify java warning pattern functions.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I837fa9290acded3b936182877d9e0c00de4cdf4f
2020-02-04 15:34:12 -08:00
Chih-hung Hsieh 9f7a42025d Merge "Add new Java/Kotlin/Make warning patterns" 2020-02-04 18:41:57 +00:00
Chih-Hung Hsieh a7f5f3fcbb Add new Java/Kotlin/Make warning patterns
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I3195999d60f268f105ead022c193319de4af4e64
2020-01-31 16:14:04 -08:00
Tianjie Xu 0e38d64e49 Add quote to the signing arguments when passing to apexer
We should add quote when passing the signing arguments to apexer. So
the final argument whould look like
apexer ... --signing_args "--signing_helper_with_files=%path"

And the argument parser in avbtool will eventually parse the
"--signing_helper_with_files" correctly.

Bug: 148627666
Test: unittest pass, run a smoke sign_target_files_apk with signing arguments
Change-Id: Ie203d26a508cb98ca94e6c364a2640b681ad79f2
2020-01-31 20:19:18 +00:00
Tianjie Xu cea6ad1fb0 Pass signing_args to apexer to repack apex
In the build signing environment, we don't necessary have access to the
physical private key. And we work around it by passing an extra
signing_helper to the avbtool instead of a private key. Since we need
to perform signing to add the hashtree footer when rebuilding the apex
image. We should propogate the signing_helper to apexer and eventually
to avbtool.

Bug: 148627666
Test: unit tests pass. Smoke test by pass a public key to apexer and
hardcode the private key in signing helper script.
Change-Id: I56d57485493a5ba2cbe4c49e493ee9604f68088b
2020-01-31 01:37:36 +00:00
Tianjie Xu ba5de57120 Merge "Assets should be optional in apex repacking" 2020-01-29 22:59:30 +00:00
Tianjie Xu 83bd55c1db Assets should be optional in apex repacking
We should only provide the --assets_dir option when the assets directory
is available in the original apex.

Bug: 148452817
Test: unit tests pass
Change-Id: I44308b80a43ff3f441223f0ecc248991d1d83f31
2020-01-29 11:40:42 -08:00
Chih-Hung Hsieh b09530bd65 Add new C++ and Java warning patterns
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I11457fdab7abd475e835367582c0b9aab11cfe6f
2020-01-29 11:01:36 -08:00
Tianjie Xu 8fff584906 Merge "Make the error more instructive on missing android_jar_path" 2020-01-29 03:19:23 +00:00
Daniel Rosenberg dac25a96df Merge "Add build properties to enable casefold and prj_quota" 2020-01-28 23:53:25 +00:00
Tianjie Xu 61a792f323 Make the error more instructive on missing android_jar_path
So when $ANDROID_BUILD_TOP is missing, the error message print out the
path of the missing file; instead of a type error.

Bug: 148452817
Test: check result with missing ANDROID_BUILD_TOP
Change-Id: I5f841f6626eb3843364f20e78b278d3244505396
2020-01-28 10:58:14 -08:00
Daniel Rosenberg 6cc2c81f4d Add build properties to enable casefold and prj_quota
This adds the ability to format f2fs and ext4 in a way that supports
casefolding and project quotas. Project quotas require wider inodes, and
thus cannot be enable apart from at build time. Casefolding can be
configured with f2fs.fsck/tune2fs later.

Both features are enabled via inheriting emulated_storage.mk in the
device.mk

Bug: 138322712
Bug: 138321217
Test: Enable on device with related patches, view fs configuration on /data
Change-Id: Ib1ab0589006ff1b3fb7478b7e9b0a8aad71e4b09
2020-01-27 12:45:47 -08:00
Tianjie Xu 88a759d651 Resign apks contained in apex
Some apex payload images contain apk files. And these apks need to be
signed during the signing processed when sign_target_files_apks is
called. To support the signing, we can extract the payload and repack
the apex file with the (de)apexer tool. Add the signing support in the
apex_util.

Bug: 146508800
Test: unit tests pass, run sign_apex, sign_target_files_apks
Change-Id: If6d58975248709a144b07dbabf47c27916e5695e
2020-01-27 19:48:39 +00:00