Commit Graph

6795 Commits

Author SHA1 Message Date
Yifan Hong 28ffd73591 Proper checks for vendor and ODM SKUs
If DEVICE_MANIFEST_FILE / ODM_MANIFEST_FILES is set, also include check
for empty vendor / odm SKU, respectively. Then, do cross product of
vendor SKUs and ODM skus, and check each possible combination.

Test: build with and without vendor skus
Test: m check-vintf-all
Bug: 148601741

Change-Id: I23cc81998e7afc36c43540bf6d615b4d4b288447
2020-03-13 18:32:07 -07:00
Xindong Xu 2a7aaa69ae ota_tool: fix incremental ota error
Problem:
Logical partitions do not update right since commit
f67dd8059d

    Generate block based OTA for product partitions

due to the wrong order of parameters. This is logged in
/tmp/recovery.log:

    skipping 0 blocks already patched to 202

Bug: 149794632
Change-Id: I92f7a8a226a8a96b892254362d362c515bfb6dc6
2020-03-13 14:25:35 +01:00
Tom Cherry 7cff1fa1fc Include private/fs_config.h directly when needed
This is no longer a transitive include of android_filesystem_config.h

Bug: 149785767
Test: build
Change-Id: Ib546dbc3b6f546ba48009cc9d6d0f4564acf0872
2020-03-05 10:55:45 -08:00
Daniel Norman d5fe862628 Uses a per-partition fingerprint for building images and avb_salt.
This causes the output image files of a merged build to be identical
to the image files of the input partial builds, for each images in
PARTITIONS_WITH_CARE_MAP.

Test: python -m unittest test_common
Test: `m dist`; `unzip out/dist/target_files.zip IMAGES/\*`;
      `zip -d out/dist/target_files.zip IMAGES/\*`
      `add_img_to_target_files -a out/dist/target_files.zip`.
      Verify that the rebuilt images are identical to the deleted ones.
Test: Build a merged target (using merge_target_files.py). Verify that
      the partial target-files.zip IMAGES are identical to the merged
      target-files.zip IMAGES for PARTITIONS_WITH_CARE_MAP images.
Bug: 150405807
Change-Id: I5fdf5783c1aff9c14cf5408090389b1f65b69ca6
2020-03-02 19:05:20 +00:00
Chih-Hung Hsieh 3cce2bcc60 Separate html/csv output functions into html_writer.py
* also suppress some pylint warnings

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I497dbb7496ca21470ba33db03eedb27f5e8f1e96
2020-02-27 15:39:18 -08:00
Tianjie Xu 223d055768 Merge "Preserve the monotonic property when subtracting 0 block" 2020-02-27 21:22:23 +00:00
Tianjie Xu f02ececc7f Preserve the monotonic property when subtracting 0 block
The monotonic property of the range is lost after the recent change to
subtract 0th block from the file range. As a result, we read the block
in the wrong order when pull files from the raw image. This yields a
incorrect hash during validation.

Bug: 150334561
Test: run validate_target_file. unittests
Change-Id: Id4fedaaf273c42c8d21a4549fda4c9a45e22066a
2020-02-26 21:50:47 -08:00
Chih-Hung Hsieh 445ad810c8 Recognize more warning patterns.
* add more Java, Kotlin, and Rust warning patterns
* recognize more clang-tidy cert-* and bugprone-* warnings;
  still keep the catch-all 'cert' and 'bugprone' groups.

Test: warn.py build.log > warnings.html
Change-Id: I909ec09822f42f60493ebe943c055bf0d4b69938
2020-02-26 14:34:21 -08:00
Chih-Hung Hsieh 5ae5519661 Use same warn_common.py and warn.py for Android and Chrome
* add chrome_project_list.py
* use newer Chrome's warn_common.py and severity.py
* modify warn_common.py to work with both python2 and python3,
  to work with and without google3

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I9aeee7031c11988d8ead1733ff4846ae6cb1ee73
2020-02-25 19:47:35 -08:00
Bowgo Tsai 2fe786a528 Support removing AVB public keys
AVB public keys might be installed into the first-stage ramdisk.
This CL supports removing some AVB keys during the signing process,
which can be useful when a key needs to be revoked.

Also correcting the path of force_debuggable detection.

Bug: 150109393
Test: make dist
Test: ./build/tools/releasetools/sign_target_files_apks \
      --remove_avb_public_keys key1.avbpubkey,key2.avbpubkey \
      out/dist/*-target_files-*.zip signed-target_files.zip
Change-Id: I1af95adbbec655f40121f3392965dd1950aca4e5
2020-02-24 17:53:10 +08:00
David Anderson ab9c7e6d04 Exclude holes from the block map.
e2fsdroid can export holes as a "0" block, in order to make sure logical
and physical block assignments are maintained properly. Exclude these
hole blocks from range sets in LoadFileBlockMap.

Bug: 139201772
Test: m dist on non-A/B
Change-Id: Ic04e869572229678fdeaaa6fbf6f464d6c672051
2020-02-21 17:04:00 +00:00
P.Adarsh Reddy 8b2e7c9c51 Make oemaids_headers available to vendor.
This change makes the oemaids_headers header library
available to vendor side.

Bug: 149665231
Test: Tested compilation, oemaids_headers is available to
a sample vendor binary.

Change-Id: I87369f1db883e40e83d6076defdd4c0d84aac46f
2020-02-18 09:44:23 +00:00
Tamas Petz c0a8c63935 build_image: check value of "ext4_share_dup_blocks"
Regardless of the value of "ext4_share_dup_blocks" "-c" flag is passed
unconditionally. The correct behavior is to check if
"ext4_share_dup_blocks" is set to "true".

Test: 1. build bonito with "BOARD_EXT4_SHARE_DUP_BLOCKS := false"
      2. check for "shared_blocks" feature of system.img
         ./out/host/linux-x86/bin/tune2fs -l \
         ./out/target/product/bonito/system.img | \
         grep -o shared_blocks

Change-Id: I8c0e5606658dd0f47b71cfc6c453a77a57da4f42
2020-02-10 11:45:06 +01:00
Tianjie Xu 9bd832a012 Remove the key parameter when verifying avb images in validate_target_files
If a key is specified, the avbtool always use the input key to verify
all the chained images. And this will cause failures when the vbmeta &
system use different keys (e.g. RSA 4096 vs RSA2048). Because the public
key to vbmeta will always fail to verify the system image. Remove the
'--key' parameter in the verification command, so the avbtool will use
the embedded public in the image.

Test: validate target-file from sdk_gphone_x86_64
Bug: 148916990
Change-Id: I9d31be0f8c32af605af94fa73d07818f40f51ec4
2020-02-06 13:31:26 -08:00
Chih-Hung Hsieh ed74896600 More Java/Kotlin warning patterns
* Simplify java warning pattern functions.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I837fa9290acded3b936182877d9e0c00de4cdf4f
2020-02-04 15:34:12 -08:00
Chih-hung Hsieh 9f7a42025d Merge "Add new Java/Kotlin/Make warning patterns" 2020-02-04 18:41:57 +00:00
Chih-Hung Hsieh a7f5f3fcbb Add new Java/Kotlin/Make warning patterns
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I3195999d60f268f105ead022c193319de4af4e64
2020-01-31 16:14:04 -08:00
Tianjie Xu 0e38d64e49 Add quote to the signing arguments when passing to apexer
We should add quote when passing the signing arguments to apexer. So
the final argument whould look like
apexer ... --signing_args "--signing_helper_with_files=%path"

And the argument parser in avbtool will eventually parse the
"--signing_helper_with_files" correctly.

Bug: 148627666
Test: unittest pass, run a smoke sign_target_files_apk with signing arguments
Change-Id: Ie203d26a508cb98ca94e6c364a2640b681ad79f2
2020-01-31 20:19:18 +00:00
Tianjie Xu cea6ad1fb0 Pass signing_args to apexer to repack apex
In the build signing environment, we don't necessary have access to the
physical private key. And we work around it by passing an extra
signing_helper to the avbtool instead of a private key. Since we need
to perform signing to add the hashtree footer when rebuilding the apex
image. We should propogate the signing_helper to apexer and eventually
to avbtool.

Bug: 148627666
Test: unit tests pass. Smoke test by pass a public key to apexer and
hardcode the private key in signing helper script.
Change-Id: I56d57485493a5ba2cbe4c49e493ee9604f68088b
2020-01-31 01:37:36 +00:00
Tianjie Xu ba5de57120 Merge "Assets should be optional in apex repacking" 2020-01-29 22:59:30 +00:00
Tianjie Xu 83bd55c1db Assets should be optional in apex repacking
We should only provide the --assets_dir option when the assets directory
is available in the original apex.

Bug: 148452817
Test: unit tests pass
Change-Id: I44308b80a43ff3f441223f0ecc248991d1d83f31
2020-01-29 11:40:42 -08:00
Chih-Hung Hsieh b09530bd65 Add new C++ and Java warning patterns
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I11457fdab7abd475e835367582c0b9aab11cfe6f
2020-01-29 11:01:36 -08:00
Tianjie Xu 8fff584906 Merge "Make the error more instructive on missing android_jar_path" 2020-01-29 03:19:23 +00:00
Daniel Rosenberg dac25a96df Merge "Add build properties to enable casefold and prj_quota" 2020-01-28 23:53:25 +00:00
Tianjie Xu 61a792f323 Make the error more instructive on missing android_jar_path
So when $ANDROID_BUILD_TOP is missing, the error message print out the
path of the missing file; instead of a type error.

Bug: 148452817
Test: check result with missing ANDROID_BUILD_TOP
Change-Id: I5f841f6626eb3843364f20e78b278d3244505396
2020-01-28 10:58:14 -08:00
Daniel Rosenberg 6cc2c81f4d Add build properties to enable casefold and prj_quota
This adds the ability to format f2fs and ext4 in a way that supports
casefolding and project quotas. Project quotas require wider inodes, and
thus cannot be enable apart from at build time. Casefolding can be
configured with f2fs.fsck/tune2fs later.

Both features are enabled via inheriting emulated_storage.mk in the
device.mk

Bug: 138322712
Bug: 138321217
Test: Enable on device with related patches, view fs configuration on /data
Change-Id: Ib1ab0589006ff1b3fb7478b7e9b0a8aad71e4b09
2020-01-27 12:45:47 -08:00
Tianjie Xu 88a759d651 Resign apks contained in apex
Some apex payload images contain apk files. And these apks need to be
signed during the signing processed when sign_target_files_apks is
called. To support the signing, we can extract the payload and repack
the apex file with the (de)apexer tool. Add the signing support in the
apex_util.

Bug: 146508800
Test: unit tests pass, run sign_apex, sign_target_files_apks
Change-Id: If6d58975248709a144b07dbabf47c27916e5695e
2020-01-27 19:48:39 +00:00
Treehugger Robot 5023711e9d Merge "Always check VNDK variant identicalness" 2020-01-17 18:45:41 +00:00
Vic Yang 4873e65c8d Always check VNDK variant identicalness
Previously, we only check VNDK core and vendor variants are identical
when a VNDK library is not declared to have different variants AND the
target has TARGET_VNDK_USE_CORE_VARIANT set.  Therefore, it is fairly
easily to break a TARGET_VNDK_USE_CORE_VARIANT target as it needs to be
tested explicitly.

This change uses the new LOCAL_CHECK_SAME_VNDK_VARIANTS and expands the
check to run regardless of TARGET_VNDK_USE_CORE_VARIANT.  Also adds
support for VNDK-in-product.

Bug: 145157349
Test: Build success for targets with and without
      TARGET_VNDK_USE_CORE_VARIANT.
Test: With the corresponding change in build/make, remove libbinder
      from build/soong/cc/config/vndk.go and check build fails even
      when TARGET_VNDK_USE_CORE_VARIANT is not set.

Change-Id: Iec708b971072e6580f77a03e243b30b89b3b054d
2020-01-14 20:05:49 -08:00
Chih-Hung Hsieh 5392cdbeb1 Call a writer to dump html; new warning patterns.
* HTML emit functions now take a writer parameter.
  This makes warn_common.py one step closer to the ChromeOS version.
* Add new found warning patterns from java and yacc.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I5c446ca767746598f07603591fdf98f7d82cae17
2020-01-14 11:57:12 -08:00
Chih-Hung Hsieh 8724ff72ef Clean up cpp warning patterns
* Remove the useless 'option' key.
  It is only used in some C/C++ warning patterns
  to give a hint of options to turn to -Werror.
  Now the global default is -Werror.
* Factor out common code patterns into high/medium/low functions.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: Ibd3f768b1552ada925eb5afb0f01ab674c968a87
2020-01-13 11:10:54 -08:00
Chih-Hung Hsieh 949205a661 Use new Severity class and update *_warn_patterns
* This new class definition and patterns are
  shared between Android and ChromeOS compiler tools.
* Suppress hard to fix and false positive linter warnings.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: Icb47809100ad30796cb1da82610e989d450194fa
2020-01-10 15:21:20 -08:00
Treehugger Robot b4484dc33b Merge "Rebuild the vendor install-recovery if necessary." 2020-01-10 13:49:55 +00:00
Treehugger Robot 88ab531831 Merge "Go one directory up looking for releasetools path" 2020-01-10 12:31:32 +00:00
Robin Lee da427de124 Rebuild the vendor install-recovery if necessary.
Test: Manual on a device with VENDOR/bin/install-recovery.sh
Bug: 146504238
Bug: 68319577
Change-Id: I33c253716bda5e7e655dac5c0636e9995e7b64c1
2020-01-09 00:29:38 +01:00
Treehugger Robot 620a2f2366 Merge "Add system_ext to VINTF search paths." 2020-01-07 23:04:54 +00:00
Chih-Hung Hsieh a9f7746f29 Add new compiler warning patterns
* Mostly new Java and Kotlin warnings found in nightly builds.
* More clean up of Java and C++ warning patterns will follow.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: I8776401d15c3a59535722d52a5eec03f954d3b15
2020-01-06 12:11:44 -08:00
Robin Lee 34ea73940e Go one directory up looking for releasetools path
Change I366099c3dfd5fa4282745ef258a8cf35338e1e42 introduced a more
restricted search where we only look in out/host/x/bin instead of
out/host/x when looking for files.

Because signapk is in out/host/x/framework, and lib64 is in out/host/x,
we need to go one level up for this to work properly.

Fix: 147096688
Test: atest --host releasetools_test releasetools_py3_test
Change-Id: I597e1d95c09eb0e27b2faebb107218761901d9ac
2020-01-02 19:25:20 +00:00
P Adarsh Reddy 7e9b5c4204 Handle virtual A/B defines during dynamic_partitions_info merge.
While merging the Dynamic Partition info Dicts in the case of
split builds, pick the virtual-a/b flags like virtual_ab,
virtual_ab_retrofit from the vendor half.

Bug: 146811936
Test: Merged build contains the virtual a/b defines.
Change-Id: I0171b074094e47cfd12dc6af2565acdcd1b65163
2019-12-24 06:32:03 +00:00
Yifan Hong dafae5228c Merge "Add /system/vendor/odm for VINTF search path" 2019-12-23 22:26:55 +00:00
Yifan Hong 9cbb624abd Add system_ext to VINTF search paths.
Bug: 140280874
Test: pass

Change-Id: Ifd5b784fc3c4553284f74a1f06cbac0a9bef4c75
2019-12-19 14:00:10 -08:00
Yifan Hong 2870d1e3c1 Add /system/vendor/odm for VINTF search path
Compared with board_config.mk and found this one is missing.

Also add docs to explain where are these values from.

Test: pass

Change-Id: Ic22d75a538657bd286edcae6d95f9ee462911975
2019-12-19 14:00:03 -08:00
Chih-hung Hsieh 9b88d8f8b1 Merge "Split warn.py into multiple files" 2019-12-19 04:30:08 +00:00
Chih-Hung Hsieh 888d143e4c Split warn.py into multiple files
* Split warning patterns into *_warn_patterns.py
* Split project list into android_project_list.py
* Split out the Severity class, to be changed later
* Split core of warn.py into warn_common.py and
  leave only platform dependent code in warn.py.
  This allows the core logic be used with different
  parallel processing libraries.
* Old warn.py just calls -m warn.warn.

Test: path_to_build/tools/warn.py build.log
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: I6734e4472a21018cd5ce06d549f6dbca24f4de54
2019-12-17 15:12:31 -08:00
David Anderson 117b52f74e releasetools: Pass --virtual-ab to lpmake.
Note that we are currently excluding this flag for retrofit devices,
since it necessitates support for a newer super.img format, which
retrofit devices may not fully support.

Bug: 134949511
Test: m -j, lpdump super_empty.img
Change-Id: Id69a8d755b9f4c4bb5fbd456a416ced1a5f1d135
2019-12-16 22:17:28 +00:00
Treehugger Robot d6aa90ce9a Merge "sign_target_apks: Add networkstack to default key_map" 2019-12-13 22:22:08 +00:00
Dan Austin 5290364e9b Add AFTL relevant options to OPTIONS in common.py.
Adds AFTL relevant options in preparation for avbtool changes. The
options are:
aftl_server: A string representing the URL of the transparency log.
aftl_key_path: The path to the public key associated with the
transparency log (in PEM format).
aftl_manufacturer_key_path: The path to the key associated with the
manufacturer for AFTL message signing.
aftl_signing_helper: The signing helper for use with AFTL message
signing.

Change-Id: I73d466f24b171c16dbdaa40c589779bed52be551
Test: Ran add_img_to_target_files with all AFTL options.
Bug: 138779249
2019-12-12 15:48:52 -08:00
David Anderson 0ec64ac0aa releasetools: Generalize base_fs_file handling.
Share the code to handle base_fs files rather than duplicate it for
system/vendor. At the same time, add support for system_ext and product
partitions.

Bug: 139201772
Test: manual test
Change-Id: Ib1b7c018c828901e24eefc01e3faead03e77bc2d
2019-12-11 22:32:00 +00:00
Oleh Cherpak 982e608085 sign_target_apks: Add networkstack to default key_map
Test: run cts -m CtsSecurityTestCases -t \
	android.security.cts.PackageSignatureTest#testPackageSignatures
Bug: 145955635
Change-Id: I1a1498562e2b5983010cb98e3edcd03ceb2cce19
Signed-off-by: Oleh Cherpak <oleh.cherpak@globallogic.com>
2019-12-10 15:03:57 +00:00
Mitchell Wills a428b092e6 Fix comment to reflect that the value passed to head is actually the size
There is no functional change

Bug: 130257221
Test: generated an archive using the script and ran the result
Change-Id: I946f2827a9519873be2173fc50d9bc1cc46a326f
2019-12-06 19:13:21 -08:00
Mitchell Wills 032de6765b Use bash instead of sh in self extracting archive
`echo -n` and `read` seems to not work the same on mac

Bug: 130257221
Test: generated an archive using the script and ran the result on linux and mac
Change-Id: I129090ccf50781b748bff9a5c7890824d2670ad4
2019-12-06 16:55:24 -08:00
Treehugger Robot cfd25567e6 Merge "logtags: Support # line comments." 2019-12-06 23:10:08 +00:00
David Anderson c55b2f516c Merge "build_image: Do not use base_fs file for system_other." 2019-12-04 21:28:53 +00:00
David Anderson 3269f25476 build_image: Do not use base_fs file for system_other.
system_other is not OTA'd (outside of factory OTA), and its block map differs greatly from
system.img, so do not attempt to re-use it.

Bug: 139201772
Test: make with base_fs file for system
Change-Id: I1ed2091da381ebdb93522a081e5d2f7c18c9d069
2019-12-04 21:28:42 +00:00
Baligh Uddin ac936fd573 Fix syntax error for SignApexFile
BUG: 145662097
TEST: Local smoke test which invokes sign_apex for local apex files.
2019-12-04 08:30:32 -08:00
Alistair Delva 1cfe41269f Merge "Allow f2fs sparse mode to be disabled" 2019-12-02 21:54:20 +00:00
Tobias Thierer 07f622b078 logtags: Support # line comments.
Allow # line comments to start anywhere in a line, not just in column 0.

Test: Added trailing # comments to SecurityLogTags.logtags and checked
      that 'make droid' still worked.
Test: Piped a few example lines through a small test script to verify
      that the following line behaved as expected:
      line = re.sub('#.*$', '', line)

Change-Id: Ib7904620adf8da7d11511ef6d8300e5fb4c73c29
2019-11-29 15:28:48 +00:00
Treehugger Robot e7c1f6314c Merge "Removes custom prefix/suffix from MergeDynamicPartitionInfoDicts." 2019-11-27 20:30:34 +00:00
Daniel Norman 55417148f9 Removes custom prefix/suffix from MergeDynamicPartitionInfoDicts.
All callers of this function now always pass the same values, so this
change hardcodes those values within the function body.

Fix: 145008064
Test: python -m unittest test_common
Test: build & boot a merged target that uses DAP

Change-Id: I0051c5ba507983231825edfcaf349e574efa451a
2019-11-26 11:09:17 -08:00
Yifan Hong 874a5aa95e Merge "Move check partition size logs to a file" 2019-11-26 00:32:52 +00:00
Steven Laver 615385ee8b releasetools: merge_target_files.py should handle new contents of dynamic_partitions_info.txt
With a recent change (cb8e2755ff) to the
generation of dynamic_partitions_info.txt,
process_dynamic_partitions_info_txt() needs a corresponding update.

Change-Id: I30ec92176fd9cd737160b579214eb5b3d6a44c10
2019-11-22 09:34:27 -08:00
Donghoon Yu 92420db7dd releasetools: fix avbtool arguments.
The following error occurred when building dist.
Separate these arguments

error log:
avbtool: error: unrecognized arguments: --partition_name vendor_boot

Bug: 144820089

Change-Id: I3891ef3bd9a2f7d2cd1377228b9894b599dc1cc3
Signed-off-by: Donghoon Yu <hoony.yu@samsung.com>
2019-11-21 06:15:02 +00:00
Alistair Delva 91238cce14 Allow f2fs sparse mode to be disabled
Bug: 142424832
Bug: 142803849
Change-Id: Ic112128f15e3f29208332afceea4fbc3fca40b36
2019-11-15 19:05:41 +00:00
Treehugger Robot e24ef86473 Merge "releasetools: build_super_image.py should unpack */build.prop" 2019-11-15 04:34:02 +00:00
Steven Laver f43fa9dd99 releasetools: build_super_image.py should unpack */build.prop
build_super_image.py attempts to retrieve the "ro.product.device"
property, which requires parsing the */build.prop files from the target
files package. This fixes the following error:
"ExternalError: couldn't resolve ro.product.device"

Test: ./otatools/releasetools/build_super_image.py -p ./otatools
aosp_arm64-target_files.zip super.img

Change-Id: I592524a27b1a4e7544644b1a344071cff9718942
2019-11-14 23:47:31 +00:00
Mitchell Wills 855bf6a85b [generate-self-extracting-archive] Cleanup and improve error handling
Increase the max extract offset and add some input sanity checks
Make the output file executable

Test: Ran manually with various license inputs
Bug: 125451157
Change-Id: Id76d55479366f1d9b8906e6d04c1a6db8d4d8285
2019-11-11 11:18:26 -08:00
Ian Kasprzak 9601e6e0f8 Add "with-license" rule to generate public AOSP phone image.
Bug: 125451157
Test: Verified output in out and dist folders

Additional verification:
- Without the VENDOR_BLOBS_NOTICE build fails
- Output shell artifact displays license as expected
- Shell artifact permissions are correct (ug executable)
- Shell artifact unzips inner zip after license agreement
- Unzipping shell artifact only unzips license.txt file

Change-Id: If1004bd61d873c592c8541a13724eb575f59cddf
2019-11-09 14:21:45 -08:00
Yifan Hong 309109351e Move check partition size logs to a file
Don't contaminate the build log.
Test: m check-all-partition-sizes -j (verbose logs stored to file)
Test: m check-all-partition-sizes-nodeps -j (see verbose logs)
Test: manually run the script with[out] -v and with[out] --logfile

Change-Id: I345a340deab3e29bb9cb05d4970a55d8758607a7
2019-11-05 14:06:27 -08:00
Yifan Hong 96527f375f Fix Virtual A/B size checks
Test: build
Test: test_check_partition_sizes
Bug: 143111912
Change-Id: I4e056c25948e4169a0b5b098168141e27c31a0d4
2019-11-05 14:06:23 -08:00
Treehugger Robot f5ad6e5443 Merge "Support Factory OTA for Virtual A/B devices" 2019-11-05 18:47:01 +00:00
Yifan Hong fe07343fed Support Factory OTA for Virtual A/B devices
In Factory OTA, secondary payload should be generated without
virtual_ab flag set, so that update package generator generates
a regular A/B OTA (with snapshot_enabled set to false), and
OTA client writes system_b without snapshots.

Test: apply secondary OTA
Fixes: 138733621
Change-Id: I05d534a12d4a7891ce5597f3c29e55f5185e5a8d
2019-11-04 15:14:26 -08:00
Yifan Hong 3a7c2ef7cc Reland "Move partition size checks to python script"
This reverts commit 0141e45b96.

Reason for revert: Fixed in follow-up CL
Test: forrest
Bug: 143734706

Change-Id: I007acf228d4fb4d6a16ae9089e3f04cf33a567bb
2019-11-01 11:41:26 -07:00
Andrei-Valentin Onea 15e75e8807 Merge "Revert "Move partition size checks to python script"" 2019-11-01 15:36:10 +00:00
Raff Tsai 0141e45b96 Revert "Move partition size checks to python script"
This reverts commit a4906cd306.

Reason for revert: b/143734706

Change-Id: I905bd9c0bfcbd123c8f9e78986bfcb5600a9eca9
2019-11-01 05:36:20 +00:00
Yifan Hong 8244f27722 Merge "Move partition size checks to python script" 2019-11-01 01:52:11 +00:00
Yifan Hong a4906cd306 Move partition size checks to python script
Easier to read and modify its logic.

Test: builds
Test: test_check_partition_sizes
Test: atest releasetools_test
Test: atest releasetools_py3_test
Change-Id: I73204664b64a0bcee7375fa6fca46758d5a2bb72
2019-10-31 11:22:20 -07:00
Stephen Hines aa94811b2a Merge "Guess source tree root from common file prefix." 2019-10-18 17:59:23 +00:00
Tianjie Xu 526b8563a5 Merge "Stop creating update_engine_payload_key" 2019-10-17 20:59:51 +00:00
Treehugger Robot 323a460d5b Merge "releasetools: Fix an issue in common.GetAvbPartitionArg." 2019-10-16 20:52:19 +00:00
Tianjie Xu 2df23d7f41 Stop creating update_engine_payload_key
Now the update_engine is able to read public keysfrom otacerts directly.
So the update_engine_payload_key is no longer needed.

Also remove the key replace in sign_target_files_apks.py. So we should
not use the new script to sign the old target files.

Bug: 116660991
Test: build the system image, unit tests pass
Change-Id: I9dae1f8b397f2b5efafed66a8faac1cb9087c741
2019-10-16 18:27:22 +00:00
Tianjie Xu 3654beb83d Merge "Call delta_generator to get the signature size" 2019-10-15 16:49:15 +00:00
Tao Bao 3612c88ed7 releasetools: Fix an issue in common.GetAvbPartitionArg.
It's a bug introduced in commit 1aeef725a7, which affects unittest only.

Bug: 130351427
Test: Use a lunch'd target. Run
      `atest --host releasetools_test releasetools_py3_test`.
Change-Id: I7ff01a6af47d002e1203bd376d477b60d769cbd1
2019-10-14 17:53:21 -07:00
Tao Bao 21fb765515 Merge "releasetools: Set the search path based on the current executable." 2019-10-14 22:20:02 +00:00
Tao Bao dc2f0b0abd Merge "Include per-partition fingerprint as AVB prop." 2019-10-14 14:53:46 +00:00
Tianjie Xu 21e6deb647 Call delta_generator to get the signature size
The signature size is needed during payload hashing and signing.
We used to sign the update_engine's payload with RSA keys only. In
this case, the signature size always equals the key size. But the
assumption is no longer true for EC keys, whose DER-encoded signature
size is a variant with a maximum size.

Therefore, we always give the maximum signature size to the delta
generator, who then add paddings to the real signature if necessary.
The maximum signature size is calculated by calling the delta_generator
with the new option '--calculate_signature_size'. For custom payload
signers, we also deprecate the '--payload_signer_key_size' and replace
it with '--payload_signer_maximum_signature_size'.

The EC key in the test is generated with:
The EC key in the unittest is generated with the command:
openssl ecparam -name prime256v1 -genkey -noout -out prime256v1-key.pem
openssl pkey -in prime256v1-key.pem -out testkey_EC.key

Bug: 141244025
Test: sign and verify a payload
Change-Id: Ife6e269d8aa3d870405aca20086330f1795e167f
2019-10-12 00:36:54 +00:00
Tao Bao afd92a8689 releasetools: Set the search path based on the current executable.
We used to mimic the behavior of build system, to find the default
search path based on OUT_DIR_COMMON_BASE or OUT_DIR. These variables
should be internal to build system.

Since we've switched releasetools script to hermetic Python executables
(e.g. `m -j ota_from_target_files`, then run the binary at
`out/host/linux-x86/bin/ota_from_target_files`), we can set the search
path in relative to the path of the current executable.

Bug: 133126366
Test: TreeHugger
Test: 1. Build aosp_x86, by "lunch aosp_x86; m -j"
      2. Inject errors to the executables under out/host/linux-x86/bin,
         e.g.  to `lpmake`.
      3. Set up OUT_DIR (e.g., to /tmp/out) and build the same product
         again by "export OUT_DIR=/tmp/out; lunch aosp_x86; m -j". Check
         that the second run finishes successfully (with the binaries at
         /tmp/out as opposed to out/; otherwise it would fail the build
         due to the invalid binaries from step 2).
Test: lunch a target;
      `atest --host releasetools_test releasetools_py3_test`
Change-Id: I366099c3dfd5fa4282745ef258a8cf35338e1e42
2019-10-11 08:53:41 -07:00
cfig 1aeef725a7 AVB: decouple vbmeta.img from recovery.img for non-A/B devices
For following cases:
Case 1: A/B devices: no change
Case 2: non-A/B devices, with unsigned recovery image:
    not allowed anymore by mandating BOARD_AVB_RECOVERY_KEY_PATH
Case 3: non-A/B devices, with signed recovery image:
    vbmeta.img should not include ChainPartitionDescriptor of recovery.img,
    otherwise device can not even boot into normal mode if recovery partition
    is damaged

This CL will cause a build break if BOARD_AVB_RECOVERY_KEY_PATH
is not set for non-A/B targets with recovery.img
The following is an example to fix the build break by specifying
AVB signing configs for the recovery.img.

BOARD_AVB_RECOVERY_KEY_PATH := external/avb/test/data/testkey_rsa2048.pem
BOARD_AVB_RECOVERY_ALGORITHM := SHA256_RSA2048
BOARD_AVB_RECOVERY_ROLLBACK_INDEX := $(PLATFORM_SECURITY_PATCH_TIMESTAMP)
BOARD_AVB_RECOVERY_ROLLBACK_INDEX_LOCATION := 2

Also note that libavb in bootloader needs an update to include this
commit Iaa886037edb18c2ff6c60fa2a7f883ab7303ba1a, to support verifying
recovery.img independently (not through vbmeta.img).

Bug: 130351427
Test (Case 3):
  normal   mode: avb_slot_verify(flags=AVB_SLOT_VERIFY_FLAGS_NONE)
  recovery mode: avb_slot_verify(flags=AVB_SLOT_VERIFY_FLAGS_NO_VBMETA_PARTITION)
Test: PYTHONPATH=build/make/tools/releasetools \
    python -m unittest test_validate_target_files
Test: Use a lunch'd target. `atest --host releasetools_test releasetools_py3_test`
Test: validate_target_files.py with Case-3 target files
Change-Id: I2a73252b385fa463b4abd444923a8acc473df0b4
2019-10-10 14:28:34 +00:00
Tao Bao 19b02fe8e5 Include per-partition fingerprint as AVB prop.
This allows querying per-partition fingerprint via libavb (in
particular, avb_property_lookup).

Bug: 80097573
Test: `m dist`; `avbtool info_image --image /path/to/image` to check the
      written prop.
Test: `atest --host releasetools_test releasetools_py3_test`
Test: Run sign_target_files_apks to sign a target_files.zip that's built
      with the change. Check the AVB prop in the signed images.
Change-Id: Id4f06df82c29e77aca128439c8c11f0367fe587e
2019-10-09 21:25:19 -07:00
Tao Bao 3ed35d3d25 releasetools: Fix the support for getting reproducible images.
We used to support getting idempotent images when given the same inputs,
by specifying UUID and AVB salt to the image builders. The support was
broken by Q changes that moved away from build-time fingerprints [1].

This CL addresses the issue by updating the logic in computing UUID and
AVB salt to be based on BuildInfo.fingerprint, as opposed to the
deprecated static build fingerprint.

[1] https://android-review.googlesource.com/c/platform/build/+/892933

Bug: 134525174
Test: TreeHugger
Test: lunch a target; `atest --host releasetools_test releasetools_py3_test`
Test: `m dist`; `unzip out/dist/target_files.zip IMAGES/\*`;
      `zip -d out/dist/target_files.zip IMAGES/\*`
Test: `add_img_to_target_files -a out/dist/target_files.zip`. Verify that
      the rebuilt images are identical to the ones deleted.
Change-Id: I8844fc7f044e9abfcd96a00e8958fa0583c70eaf
2019-10-07 23:32:07 -07:00
Tao Bao 1c320f8573 releasetools: Move BuildInfo into common.
There is no change to module functionalities. Testcases are moved around
accordingly.

Bug: 134525174
Test: TreeHugger
Test: lunch a target; atest --host releasetools_test releasetools_py3_test
Change-Id: I7bc8f49cc239e7c6655fe5e375508f01c1743b94
2019-10-07 20:13:59 -07:00
Tao Bao e114804150 releasetools: Move MockScriptWriter into test_utils.
Bug: 134525174
Test: TreeHugger
Test: lunch a target; atest --host releasetools_test releasetools_py3_test
Change-Id: I6d30f4d153d59d65227275e1d3285e30dfafd90e
2019-10-07 20:00:34 -07:00
Tao Bao 60f5d67677 releasetools: Update OWNERS.
Test: N/A
Change-Id: Ie476ff08a360f15f769e54dca7428a2d055f1a37
2019-10-07 12:39:55 -07:00
Tao Bao 7b1c49a70d Merge "releasetools: Update the handling of recovery fstab." 2019-10-07 19:33:21 +00:00
Tao Bao ae3a9a138d Merge "releasetools: Fix releasetools_py3_test." 2019-10-07 19:06:07 +00:00
Tao Bao 615b65d097 releasetools: Fix releasetools_py3_test.
Test: Use a lunch'd target;
      `atest --host releasetools_test releasetools_py3_test`
Change-Id: Ibcced83a7989ef8fc15464728ab1d5fa71ca988e
2019-10-06 23:29:34 -07:00
Tao Bao a264feffe8 releasetools: Update tests to match SparseImage change.
The previous change in commit 22632cc82c
changed the behavior in loading file map. It now always puts a copy of
the input text in `extra` field. Update the tests accordingly.

Bug: 79951650
Test: Use a lunch'd target; `atest --host releasetools_test`
Change-Id: Iccf06c817c1305bf9946d7759c6f6f6af21fe85e
2019-10-06 22:21:14 -07:00
Tao Bao 99d855db8f Merge "releasetools: Support verifying files with non-monotonic ranges." 2019-10-07 01:33:30 +00:00
Tao Bao 765668fff7 releasetools: Update the handling of recovery fstab.
First, remove the use of info_dict['fstab'] in add_img_to_target_files.
 - info_dict['fstab'] corresponds to recovery fstab
   (`/etc/recovery.fstab`), which may differ from the one used for
   normal boot.
 - When calling build_image.ImagePropFromGlobalDict, we already have the
   desired info from global dict (`META/info_dict.txt`).

Second, common.LoadInfoDict now loads recovery fstab only for non-A/B
devices. Because the info is only meaningful for installing non-A/B OTAs
(under recovery mode).

Fixes: 132458722
Test: TreeHugger
Test: Build non-A/B incremental OTAs.
Change-Id: Id23e7b17264c123319fe00b1663d52bfd9b4a5e2
2019-10-04 22:39:32 -07:00
Tao Bao 22632cc82c releasetools: Support verifying files with non-monotonic ranges.
Fixes: 79951650
Test: Run validate_target_files on target_files.zip with files in
      non-monotonic ranges.
Test: python -m unittest test_validate_target_files
Test: python3 -m unittest test_validate_target_files
Change-Id: I82571d3358598775de4cdeb5e64035689fea6487
2019-10-03 23:21:22 -07:00
Bill Peckham e868aec14b Moving recovery resources from /system to /vendor
This change is part of a topic that moves the recovery resources from the
system partition to the vendor partition, if it exists, or the vendor directory
on the system partition otherwise. The recovery resources are moving from the
system image to the vendor partition so that a single system image may be used
with either an A/B or a non-A/B vendor image. The topic removes a delta in the
system image that prevented such reuse in the past.

The recovery resources that are moving are involved with updating the recovery
partition after an update. In a non-A/B configuration, the system boots from
the recovery partition, updates the other partitions (system, vendor, etc.)
Then, the next time the system boots normally, a script updates the recovery
partition (if necessary). This script, the executables it invokes, and the data
files that it uses were previously on the system partition. The resources that
are moving include the following.

* install-recovery.sh
* applypatch
* recovery-resource.dat (if present)
* recovery-from-boot.p (if present)

This change includes the platform build system and release tools changes to
move the recovery resources from system to vendor (or /system/vendor). The
release tools need to know where to generate the recovery patch, and they
discover this from misc_info.txt variable board_uses_vendorimage, which the
platform build system generates.

We remove applypatch from PRODUCT_PACKAGES, but it is added back as a required
module in target/product/base_vendor.mk.

Several release tools rely on the misc_info.txt board_uses_vendorimage variable
to know how to generate and detect the recovery patch.

This change partially removes the --rebuild_recovery flag from the
merge_target_files.py script. The flag will be fully removed in a follow-on
change.

Bug: 68319577
Test: Ensure that recovery partition is updated correctly.
Change-Id: Ia4045bd67ffb3d899efa8d20dab4c4299b87ee5f
2019-10-04 00:04:34 +00:00
Treehugger Robot 1623ec97f1 Merge "add vendor_boot image support" 2019-09-27 21:07:24 +00:00
Steve Muckle e1b1086ef3 add vendor_boot image support
The vendor_boot partition is generated by mkbootimg and contains all the
device-specific information that used to reside in the boot partition.

Bug: 137297791
Change-Id: I5b005097b73f59857c3a2f92d693b3e67ee8424e
2019-09-27 10:26:00 -07:00
Chih-Hung Hsieh 83980501d9 Fix severity of analyzer warnings; add new warnings.
* The following were high severity warnings but mistakenly
  grouped into analyzer warnings in a recent change.
        clang-analyzer-unix.Malloc
        clang-analyzer-cplusplus.NewDeleteLeaks
        clang-analyzer-cplusplus.NewDelete
* Add new Protoc and Kotlin warnings about unused imports and variables.

Bug: 141493530
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: Ied1f9ace97f29fba58a40dff2346a0391ab22f06
2019-09-26 12:14:32 -07:00
Chih-Hung Hsieh 790020437d Factor out common code in Java warn patterns.
* Some comments are kept to reduce diff artifacts.
* Warning patterns will be separated by tools later,
  to make it easier to share code with Chromium and
  other build tools.

Bug: 141493530
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: I6b589478cf5ae73940b4712d9faabd0909546f02
2019-09-25 13:39:41 -07:00
Chih-hung Hsieh 415186a71c Merge "Add new compiler warning patterns" 2019-09-25 17:00:36 +00:00
Paul Duffin e19b7471ae Merge "Replace use of conscrypt-nojarjar with conscrypt-unbundled" 2019-09-25 09:58:56 +00:00
Chih-Hung Hsieh d591a5e012 Add new compiler warning patterns
* Add Kotlin, NDK, Protoc, Asm, and other new patterns.
* Put all clang-analyzer warnings to the ANALYZER severity group.
  Some security and null pointer warnings still have HIGH severity.
* Comment out old C/C++ warnings that overlap with analyzer
  warnings and do not appear in current build.log.
* Other old fixed warning patterns should be commented out later.

Bug: 141493530
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: Idc7b09e5e4eefa127d41e9f37da1f566ed1ede1f
2019-09-24 14:52:09 -07:00
Chih-Hung Hsieh f5db8985f5 Guess source tree root from common file prefix.
* When processing a downloaded build.log file,
  find_android_root is very slow because the
  android source tree is not found.
  Change it to look for warn.py only up to 100 times,
  and then set android_root to the longest common path prefix.

Bug: 141524232
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: I8cc2e208d439267313aaa6cdbc00f40320409c63
2019-09-24 18:19:17 +00:00
Chih-hung Hsieh d5ea80b388 Merge "Handle rustc warning messages." 2019-09-23 16:53:38 +00:00
Tao Bao 08f9da1898 Merge "Don't generate hashtree when signing bundled APEXes." 2019-09-23 14:22:59 +00:00
Paul Duffin ebbcc1bb78 Replace use of conscrypt-nojarjar with conscrypt-unbundled
Bug: 141346785
Test: m checkbuild
Change-Id: Ibcd4aac938e0fdf5765ad8426fe64c6d053f3529
2019-09-23 14:30:07 +01:00
Chih-Hung Hsieh a1187072c5 Handle rustc warning messages.
* Combine the first two lines of rustc warning messages.
* Give some rustc warnings HIGH and MEDIUM severity.

Bug: 141311294
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: Ia3a206d68d9e92934296487b006a7b424044a822
2019-09-20 17:14:22 -07:00
Tao Bao 448004af9d Don't generate hashtree when signing bundled APEXes.
Bug: 139957269
Test: Sign a target_files.zip. Extract a re-signed APEX and check the
      hashtree size (being zero).
Test: Use sign_apex to sign an APEX file. Check the hashtree size (not
      being zero).
Test: python -m unittest test_apex_utils
Test: python -m unittest test_sign_apex
Change-Id: I927b7681d66920d7732b700ec3a8f7a65b4cb351
2019-09-19 14:41:34 -07:00
Treehugger Robot c4f6f36dc4 Merge "Creates img.zip from target files zip, not dir." 2019-09-19 01:03:33 +00:00
Daniel Norman 74eb74b146 Creates img.zip from target files zip, not dir.
Directory support for img_from_target_files was dropped in aosp/1095755.

Test: Create & boot a merged build.
Change-Id: Ib976ea60a24c2f172c597130d6bcce25199a93b3
2019-09-18 14:03:34 -07:00
Tao Bao 3e759462d9 releasetools: Remove more images out of secondary payload.
This CL additionally removes boot/dtbo/modem/vbmeta_* images out of
secondary payload. We essentially only keep system_other.img and
bootloader images there.

For Pixel devices, this additionally saves ~80MiB (mostly because of
the removal of boot and radio images).

Bug: 140771390
Test: ota_from_target_files \
          --include_secondary --skip_postinstall \
          -i input-target_files-1.zip \
          input-target_files-2.zip \
          output-ota.zip
Test: python -m unittest test_ota_from_target_files
Change-Id: If47b27c52b3547a4cc86223a988c53960bc6af40
2019-09-17 22:56:19 -07:00
Tao Bao d5b6094ed3 Merge "releasetools: Sanity check the build fingerprint." 2019-09-17 18:30:45 +00:00
Tao Bao e430f7652a Merge "Also install verity_key to ramdisk for non-system-as-root target." 2019-09-17 18:25:55 +00:00
Yifan Hong 7ca2c1539f Merge "extract-kernel: Fix indexing" 2019-09-17 16:50:36 +00:00
Tao Bao c99819311c Also install verity_key to ramdisk for non-system-as-root target.
The commit in d14b895665
(https://android-review.googlesource.com/c/platform/build/+/728287)
changed partition layout, to always build the root dir into system.img,
even for devices not using system-as-root (i.e. the ones with separate
boot ramdisk).

With the new layout, there will be two root dirs for non-system-as-root
targets during the boot. If such a device uses Verified Boot 1.0,
/verity_key needs to be available in both roots, to establish the chain
of trust.
 - bootloader uses the baked-in key to verify boot.img; it then loads
   the ramdisk from the verified boot.img
 - First stage init uses /verity_key (in ramdisk) to verify and mount
   system.img at /system, then chroot's to it
 - Second stage init uses /verity_key (in system.img) to verify and
   mount other partitions

This CL adds rules to additionally install verity_key into ramdisk for
such targets.

Bug: 139770257
Test: Set up a target to use non-system-as-root
      (BOARD_BUILD_SYSTEM_ROOT_IMAGE != true). `m dist`.
Test: Check that both ROOT/verity_key and BOOT/RAMDISK/verity_key exist
      in the built target_files.zip.
Test: Run validate_target_files to validate the above target_files.zip.
      $ validate_target_files \
          --verity_key_mincrypt /path/to/verity_key \
          target_files.zip
Test: Run sign_target_files_apks to sign the above target. Re-run
      validate_target_files on the signed target_files.zip.
Test: python -m unittest test_validate_target_files
Change-Id: Ibe7e771c8c376429add85851ac86055564765d3c
2019-09-17 08:53:16 -07:00
Tao Bao c4011cd719 releasetools: Sanity check the build fingerprint.
Bug: 140298338
Test: python -m unittest test_ota_from_target_files
Change-Id: I4add5f82d5b684c441b0484732da6991f8e7301a
2019-09-17 08:43:54 -07:00
Yifan Hong 8b72776511 extract-kernel: Fix indexing
The original script always tries to extract from index
0 even if header is found in positive indices. Fix that.

Also, continue to try other positions if previous run failed.

Test: builds
Bug: 139348603

Change-Id: Ia54fc709de9ae587fc64b59d94a1fa4ae669c14f
2019-09-16 16:30:35 -07:00
Tianjie Xu 43a049f814 Merge "Remove vendor & product in secondary payload" 2019-09-12 17:40:22 +00:00
Tianjie Xu 1c8080098a Remove vendor & product in secondary payload
We used to carry the full OTA for vendor and product partitions when
generating the secondary payload. Since they are not actually used, we
can remove them to reduce the size of factory OTA packages. For example,
the size for a sargo package reduces from 454M to 283M after the change.

Bug: 140771390
Test: generate and apply a second payload on Pixel
Change-Id: I091dc07db8c345b182d9f5dd8d48f5c209245641
2019-09-11 15:23:41 -07:00
dianlujitao 5dcbe701cf fs_config: Fix generation for devices without vendor partition
If the "--all-partitions" argument is empty, the system fs_config should
be generated with all partitions emitted.

Fixes: 8453f02c (fs_config: Fix cases without vendor/oem partition)
Change-Id: I058cfd1c8e665b40757998e773c35e26465bf58a
Signed-off-by: dianlujitao <dianlujitao@lineageos.org>
2019-09-10 16:17:47 +08:00
Yifan Hong 708cbdbd41 Merge "releasetools: add dep to checkvintf"
am: fd4a3de77d

Change-Id: Ie5ffcd9b96de64129555980d77d851f7672f2083
2019-09-04 17:53:30 -07:00
Yifan Hong 09298552e2 releasetools: add dep to checkvintf
'required' only works on python_binary_host.
Test: modify checkvintf source code, build ota_from_target_files

Change-Id: Ieac68b39b1ccabae679a1679f184cb3aeaaeefb6
2019-09-04 12:37:46 -07:00
Tao Bao bce0264348 Merge "releasetools: img_from_target_files uses zip2zip."
am: f833a75283

Change-Id: I1731934e5585185167db9638c7b6652871fbf913
2019-08-30 12:29:42 -07:00
Tao Bao f833a75283 Merge "releasetools: img_from_target_files uses zip2zip." 2019-08-30 19:13:59 +00:00
Tianjie Xu 8b106fd2bc Merge "Add a new flag "--disable_fec_computation" for OTA generation"
am: 9c9993c706

Change-Id: Ic077215542e20c2bc9fbaaaa2fa602f15d3f4a87
2019-08-29 13:27:20 -07:00
Tianjie Xu 9c9993c706 Merge "Add a new flag "--disable_fec_computation" for OTA generation" 2019-08-29 20:05:41 +00:00
Tianjie Xu 1b07983c6d Add a new flag "--disable_fec_computation" for OTA generation
We have added the flag to disable on device fec computation in
brillo_update_payload. Exposed it in ota_from_target_files as well.

Bug: 139723500
Test: generate an incremental package with the new flag
Change-Id: I683e3ec7791963a3c7e141ecb4e5a3fb53fe2108
2019-08-28 21:38:05 -07:00
Yifan Hong ee276acce8 Merge changes from topic "vintf_ota_delete"
am: bcba1cfdaf

Change-Id: If11df7c409b9d1087defc119bf3a132f5b7016c5
2019-08-28 17:21:36 -07:00
Yifan Hong a592de9c61 Check VINTF compatibility at OTA generation time.
am: 9276cf0226

Change-Id: Iee0efa785ccbf2aaadb1be7b3dc106f805242adb
2019-08-28 17:21:27 -07:00
Yifan Hong 96961bf94f releasetools: add tests for check_target_files_vintf
am: ccb86fe7d4

Change-Id: I3ecddd78ccce7f97bd02abea133ad84187b312ac
2019-08-28 17:21:07 -07:00
Yifan Hong d14a6110c5 Add a script to check VINTF compat of target files package.
am: e3ba82cff2

Change-Id: Iccf4b7afa93749f441f48a3270f3d4beac8a0f06
2019-08-28 17:20:52 -07:00
Tao Bao 57f8ed6b5e releasetools: img_from_target_files uses zip2zip.
Test: Build the following targets and check the built image archive.
      bonito-userdebug (contains flashable images);
      cf_x86_phone-userdebug (contains super.img);
      aosp_arm64-eng (contains VerifiedBootParams.textproto)
Test: m -j otatools-package; Check that zip2zip is included; Use
      bin/img_from_target_files to generate image archive.
Change-Id: I9f28765bd702849f0df309faebd6aa3072920536
2019-08-28 16:36:25 -07:00
Yifan Hong bcba1cfdaf Merge changes from topic "vintf_ota_delete"
* changes:
  merge_target_files: Checks VINTF metadata of resulting target files
  Check VINTF compatibility at OTA generation time.
  releasetools: add tests for check_target_files_vintf
  Add a script to check VINTF compat of target files package.
2019-08-28 23:19:21 +00:00
Yifan Hong 9276cf0226 Check VINTF compatibility at OTA generation time.
Instead of relying on META/{system,vendor}_{manifest,matrix}.xml
and blindly copy compatibility.zip from target files package to
OTA package, do a static check on the input target files package
before generating the OTA package.

META/{system,vendor}_{manifest,matrix} does not contain ODM manifest
fragments, which is incorrect.

Also, the on-device check of compatibility.zip uses a old libvintf
binary on the device, which may not understand the incoming VINTF
metadata. This change removes the on-device check. Hence, it removes
the requirement of forwards compatibility of libvintf.

This behavior can be skipped with --skip-compatibility-check.

Test: build OTA package
Bug: 139300422
Bug: 131425279
Change-Id: I7fb93be9eb73f578fc05a182c6c9d1f073db2800
2019-08-28 11:02:26 -07:00
Yifan Hong ccb86fe7d4 releasetools: add tests for check_target_files_vintf
Test: run it
Bug: 139300422
Change-Id: I265d1e6313177dc661bb5cdf1415b11684dec0a0
2019-08-28 11:02:26 -07:00
Yifan Hong e3ba82cff2 Add a script to check VINTF compat of target files package.
Instead of checking META/{system,vendor}_{manifest,matrix}.xml (which
is error-prone because ODM SKU-specific manifests are not checked),
this script read the target files package, remaps its directory
structure so that checkvintf understands it, and check VINTF
compatibility.

Also, put it in otatools.zip.

Test: run it on an extracted target files package
Bug: 131425279
Change-Id: I06036f9a8d7242d4bc11524028be40e780c508e8
2019-08-28 11:02:26 -07:00
Yifan Hong ade0d3f4a2 merge_target_files: Checks VINTF metadata of resulting target files
Bug: 139300422
Bug: 131425279
Test: test_merge_target_files
Change-Id: I972db2fc40555ab0fc38316f0a6e6f5e8f163371
2019-08-28 11:02:26 -07:00
Tao Bao 3a85b1a466 Merge "releasetools: Clean up img_from_target_files.py."
am: 1f8e8738c1

Change-Id: Ic5718fd194395f86f99da41791ed9c909f7638cd
2019-08-27 16:45:09 -07:00
Tao Bao ac63a9d728 releasetools: Clean up img_from_target_files.py.
Bug: N/A
Test: Run img_from_target_files to generate an image archive.
Change-Id: I802640b88fe0c7a6d1aab63411b8194d61bceaba
2019-08-26 20:33:11 -07:00
Daniel Mentz 1e8961033e Merge "Consider search_path to locate avb_*_key_path files"
am: 11dc247ba9

Change-Id: Ie0479fe88e37b330bc80fb6cb68d906546c70b0a
2019-08-26 12:21:24 -07:00
Daniel Mentz 25478181b9 Consider search_path to locate avb_*_key_path files
avb_*_key_path are usually relative file paths. If these files cannot be
located from the current directory, try prepending OPTIONS.search_path.

Bug: 139836256
Test: python -m unittest test_common
Test: python -m unittest test_add_img_to_target_files
Change-Id: I020b257f458a1f5762691d0499feaf4027e37126
2019-08-26 18:17:14 +00:00
Baligh Uddin 6fac8bab91 Merge "Add a guard against deleting AVB_PUBKey."
am: 667c1aa83d

Change-Id: I255c746b3f6c8b98787f157800d143523c82161f
2019-08-25 22:16:58 -07:00
Baligh Uddin 15881282d7 Add a guard against deleting AVB_PUBKey.
The avb_pubkey may not be present, if the apex was initially
unsigned or generated from a bundle.  In this case, running
sign_apex to generate a signed apex binary would result in an error.
This fix checks for presense of avbpubkey before attempting the
deletion

BUG: 139994107
Change-Id: I3cb2e88a11ad8797e38ba5fb98c96a4ec4135fc8
2019-08-25 22:01:45 -07:00
Dan Willemsen 86be07669f Merge "Remove package-stats.txt"
am: 1211130c92

Change-Id: I6be259692a3a25934f30595b7f23177c4f6b62bb
2019-08-21 23:51:21 -07:00
Dan Willemsen d7994816c1 Remove package-stats.txt
I can't find any remaining users of this functionality.

Test: codesearch; treehugger
Change-Id: I63b079b381211b743baa293267c992d9b2eadcf9
2019-08-21 12:36:14 -07:00