In BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS, if we have defined
"--include_descriptors_from_image" with an image file whose path points
to source tree, add_img_to_target_files.py or sign_target_files_apks.py
may fail to find the file. Because these scripts may run without a
source tree, by taking target_files.zip as the only input.
This CL scans additional locations in the input target_files.zip to find
those missing files in avb_vbmeta_args. As long as the files are included
in the target_files.zip, they get a second chance to be found.
Bug: 63910867
Test: As follows:
1. Setup BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS with a local file path;
2. Remove the local file;
3. sign_target_files_apks.py fails without this CL;
4. sign_target_files_apks.py works.
Change-Id: I3c58f80a5535db02b74cfe40d0c0beff72587cf8
Brotli has a better compression ratio than deflation. So for non-AB
full OTA, we can compress the new.dat with brotli first and store it
in the zip package.
This increase the OTA generation time by ~5 minutes for a full OTA
(measured based on 2.0G system.new.dat)
Bug: 34220646
Test: Generate a full OTA for bullhead
Change-Id: I9c0550af8eafcfa68711f74290c8e2d05a96648f
Accidentally broken by the cherry-pick in commit
f829b40c48 - the original CL in oc-dev
doesn't require the 'import stat' line.
Bug: 63629728
Test: `pylint --rcfile=pylintrc sign_target_files_apks.py`
Test: Successfully sign a build with sign_target_files_apks.py.
Change-Id: I94be613fb2219597148c4339ac33fc93d0cdae47
system_avb_hashtree_enable has been renamed to
avb_system_hashtree_enable in commit
3e599ead66. 'care_map.txt' is missing due
to the change.
Bug: 63142730
Test: `m dist` and check IMAGES/care_map.txt exists in target_files zip.
Change-Id: I60c269b41df844b50353d357bf67c20f15548642
(cherry picked from commit 3f72176da0)
system_avb_hashtree_enable has been renamed to
avb_system_hashtree_enable in commit
3e599ead66. 'care_map.txt' is missing due
to the change.
Bug: 63142730
Test: `m dist` and check IMAGES/care_map.txt exists in target_files zip.
Change-Id: I60c269b41df844b50353d357bf67c20f15548642
Currently we're writing META/misc_info.txt to the new TF.zip during
ReplaceVerityPrivateKey(). We should delay that until we have replaced
everything in need. Otherwise we won't be able to replace/overwrite
that zip entry (unless `zip -d` first).
This CL also cleans up the return value of ReplaceVerityPublicKey() and
ReplaceVerityKeyId(), since the caller no longer needs the values.
Test: sign_target_files_apks.py and check the generated signed TF.zip.
Change-Id: I9fbd7182247728281519e5e3971557f6b018ad65
(cherry picked from commit 46a5999a02)
Radio images are added to INSTALLED_RADIOIMAGE_TARGET, which by default
will be packed under RADIO/ in a target_files zip. This CL introduces
BOARD_PACK_RADIOIMAGES that allows additionally copying them into
IMAGES/, which will then be included into <product>-img.zip.
Bug: 62195105
Test: Define BOARD_PACK_RADIOIMAGES and `m dist`. Check the generated
target_files zip and <product>-img.zip.
Change-Id: I3deafd2dfecd1d4dbfdfc2d002fc40ef22fb60ea
This patch tries to fix the problem where the default properties need
to go with the system image especially on non-AB devices where
/default.prop is on the ramdisk image. A symlink is created at
/default.prop for backward compatibility.
Bug: 37815285
Test: Tested with ag/2416542. Booted pixel phones, checked the location
of prop.default, verified the symlink, checked a few properties
(via adb shell getprop) and manually tested a few apps (Camera,
Maps etc).
sign_target_files_apks.py was tested with:
sign_target_files_apks -o -e DynamiteLoader.apk= -e DynamiteModulesA.apk= \
-e DynamiteModulesB.apk= -e DynamiteModulesC.apk= -e DynamiteModulesD.apk= \
-e GoogleCertificates.apk= out/dist/*-target_files-*.zip signed-target_files.zip
Booted to recovery and ran 'adb sideload' successfully.
Change-Id: I1a9a2ba49c8252afc13ced3dea71253afbd3091e
Merged-In: I1a9a2ba49c8252afc13ced3dea71253afbd3091e
(cherry-picked from 4fbbe4578bb10d54292d9b243edf4999fddf1c93)
This patch tries to fix the problem where the default properties need
to go with the system image especially on non-AB devices where
/default.prop is on the ramdisk image. A symlink is created at
/default.prop for backward compatibility.
Bug: 37815285
Test: Tested with ag/2416542. Booted pixel phones, checked the location
of prop.default, verified the symlink, checked a few properties
(via adb shell getprop) and manually tested a few apps (Camera,
Maps etc).
sign_target_files_apks.py was tested with:
sign_target_files_apks -o -e DynamiteLoader.apk= -e DynamiteModulesA.apk= \
-e DynamiteModulesB.apk= -e DynamiteModulesC.apk= -e DynamiteModulesD.apk= \
-e GoogleCertificates.apk= out/dist/*-target_files-*.zip signed-target_files.zip
Booted to recovery and ran 'adb sideload' successfully.
Change-Id: I1a9a2ba49c8252afc13ced3dea71253afbd3091e
* Owners are selected from top CL approvals or owners.
They will be suggested to review/approve future CLs.
* OWNERS files are recognized by the new find-owners plugin, see .md files in
https://gerrit.googlesource.com/plugins/find-owners/+/master/src/main/resources/Documentation/
Test: build/make/tools/checkowners.py -c -v OWNERS
Change-Id: Ibe6765e5448c471ca413413e3d052507ccee4ea6
Current build system will include AVB metadata from each partition and
store them into /vbmeta partiton when BOARD_AVB_ENABLE is set, which makes
each partition tightly-coupled.
Add the support for 'chain partition':
- The vbmeta of each partition is stored on the same partition itself.
- The public key used to verify each partition is stored in /vbmeta.
For example, the following build variables are required to enable chain
partition for system partition:
- BOARD_AVB_SYSTEM_KEY_PATH := path/to/system_private_key
- BOARD_AVB_SYSTEM_ALGORITHM := SHA512_RSA8192
- BOARD_AVB_SYSTEM_ROLLBACK_INDEX := 1
- BOARD_AVB_SYSTEM_ROLLBACK_INDEX_LOCATION := 2
The corresponding settings will be added into META/misc_info.txt for
build_image.py and/or add_img_to_target_files.py:
- avb_system_key_path=path/to/system_private_key
- avb_system_algorithm=SHA512_RSA8192
- avb_system_add_hashtree_footer_args=--rollback_index 1
- avb_system_rollback_index_location=2
To enable chain partition for other partitions, just replace SYSTEM with
BOOT, VENDOR and/or DTBO in the build variables.
Also switch from `avbtool make_vbmeta_image --setup_rootfs_from_kernel system.img ...`
to `avbtool add_hashtree_footer --image system.img --setup_as_rootfs_from_kernel...`
when BOARD_BUILD_SYSTEM_ROOT_IMAGE is true. This works for both chained
and no-chained:
- chained: `avbtool add_hashtree_footer --setup_as_rootfs_from_kernel` will
add dm-verity kernel cmdline descriptor to system.img
- no-chained: `avbtool make_vbmeta_image --include_descriptors_from_image
system.img` will include the kernel cmdline descriptor from system.img into
vbmeta.img
Bug: 38399657
Test: `make` pass, flash images from $OUT and boot device without chain partitions
Test: `make` pass, flash images from $OUT and boot device with chain partitions
Test: `make dist` pass, flash images from TF.zip and boot device without chain partitions
Test: `make dist` pass, flash images from TF.zip and boot device with chain partitions
Test: follow the same steps in
https://android-review.googlesource.com/#/c/407572/
Change-Id: I344f79290743d7d47b5e7441b3a21df812a69099
Merged-In: I344f79290743d7d47b5e7441b3a21df812a69099
(cherry picked from commit 3e599ead66)
Check the SHA1 of recovery.img and boot.img embedded in the
install-recovery.sh.
Bug: 35411009
Test: validation script detects mismatch for both full recovery and
recovery-from-boot.
Change-Id: I5f07a869d9fa17fad26a22ef9ca3ecb06b1b28e3
The script has been moved to /system/bin since L.
Bug: 35411009
Test: Generate a exact same OTA for bullhead.
Change-Id: Iaecb86e3dabc5b659605837bb06f16d20fa80334
We should only disallow zip64 for the image and OTA zips (because we
don't have zip64 support in libziparchive yet). But target_files zips
are fine to use zip64 with host tools (and we already do that in
add_img_to_target_files.py).
This CL also sets the default compression method to DEFLATED when
creating the signed TF.zip.
Test: sign_target_files.apks.py signing a large TF.zip passes.
Change-Id: I8043739860604134fa1166e920c95c28797bbcc1
In case the system/vendor image is updated for an A/B target file, we'll
delay the write of META/care_map.txt. Specifically, we'll run "zip -d"
to remove the old entry and then zip in the new one from tmp dir.
Bug: 62345693
Test: Run add_img_to_target_files.py on sailfish target files, and
care_map.txt is updated.
Change-Id: Id2f9997aac50c0740e6944d0d0d43d98f5fc6c29
Current build system will include AVB metadata from each partition and
store them into /vbmeta partiton when BOARD_AVB_ENABLE is set, which makes
each partition tightly-coupled.
Add the support for 'chain partition':
- The vbmeta of each partition is stored on the same partition itself.
- The public key used to verify each partition is stored in /vbmeta.
For example, the following build variables are required to enable chain
partition for system partition:
- BOARD_AVB_SYSTEM_KEY_PATH := path/to/system_private_key
- BOARD_AVB_SYSTEM_ALGORITHM := SHA512_RSA8192
- BOARD_AVB_SYSTEM_ROLLBACK_INDEX := 1
- BOARD_AVB_SYSTEM_ROLLBACK_INDEX_LOCATION := 2
The corresponding settings will be added into META/misc_info.txt for
build_image.py and/or add_img_to_target_files.py:
- avb_system_key_path=path/to/system_private_key
- avb_system_algorithm=SHA512_RSA8192
- avb_system_add_hashtree_footer_args=--rollback_index 1
- avb_system_rollback_index_location=2
To enable chain partition for other partitions, just replace SYSTEM with
BOOT, VENDOR and/or DTBO in the build variables.
Also switch from `avbtool make_vbmeta_image --setup_rootfs_from_kernel system.img ...`
to `avbtool add_hashtree_footer --image system.img --setup_as_rootfs_from_kernel...`
when BOARD_BUILD_SYSTEM_ROOT_IMAGE is true. This works for both chained
and no-chained:
- chained: `avbtool add_hashtree_footer --setup_as_rootfs_from_kernel` will
add dm-verity kernel cmdline descriptor to system.img
- no-chained: `avbtool make_vbmeta_image --include_descriptors_from_image
system.img` will include the kernel cmdline descriptor from system.img into
vbmeta.img
Bug: 38399657
Test: `make` pass, flash images from $OUT and boot device without chain partitions
Test: `make` pass, flash images from $OUT and boot device with chain partitions
Test: `make dist` pass, flash images from TF.zip and boot device without chain partitions
Test: `make dist` pass, flash images from TF.zip and boot device with chain partitions
Test: follow the same steps in
https://android-review.googlesource.com/#/c/407572/
Change-Id: I344f79290743d7d47b5e7441b3a21df812a69099
Currently we're writing META/misc_info.txt to the new TF.zip during
ReplaceVerityPrivateKey(). We should delay that until we have replaced
everything in need. Otherwise we won't be able to replace/overwrite
that zip entry (unless `zip -d` first).
This CL also cleans up the return value of ReplaceVerityPublicKey() and
ReplaceVerityKeyId(), since the caller no longer needs the values.
Test: sign_target_files_apks.py and check the generated signed TF.zip.
Change-Id: I9fbd7182247728281519e5e3971557f6b018ad65
Bug: 62252466
Test: Re-generate an incremental OTA that goes from the fingerprint to
thumbprint, and check the updater-script.
Change-Id: I6e2cbf68cbd22fbcf0d200fc3fdc8a33da510a53
board_avb_algorithm and board_avb_key_path are overlapping with
avb_signing_args. In core/Makefile, only avb_signing_args (i.e.
INTERNAL_AVB_SIGNING_ARGS) will be used in the AVB-signing command. It
covers the contents in board_avb_{algorithm,key_path}. We should do the
same thing in tools/releasetools to avoid potential inconsistency.
This CL cleans up the logic in tools/releasetools, by always using
avb_signing_args. This also allows easier signing key replacement (so we
can replace the key/algorithm/signer in 'avb_signing_args').
board_avb_system_add_hashtree_footer_args is unused in releasetools
script, and the same information has been covered by
system_avb_add_hashtree_footer_args. This CL removes this arg as well.
Test: `m dist`. Then a) check the removed three args no longer exist in
META/misc_info.txt; b) check that rebuilding images with
add_img_to_target_files.py uses the same parameters.
Change-Id: I7db890b5c942de5b6868d8d1ebf937586d4729c0
When AVB is enabled, generate care_map.txt and add it to the target
files. Also copy it into the OTA package where it will later be used
by the update_verifier.
Bug: 62208947
Test: \
1. Run add_img_to_target_files on the TF of a new pixel device,
and care_map.txt generates successfully.
2. Make dist in oc-dr1-release and find care_map.txt in the OTA package.
3. update_verifier succeeds in reading all the blocks on the care_map,
and fails to read out-of-bound blocks.
Change-Id: I2881711e6f87789cb7de150dbeca18b756fed68a
This is a step to enable signing a given target_files zip with release
keys.
When calling sign_target_files_apks.py, we will delete all the entries
under IMAGES/ in order to re-generate them (with the proper release
keys). In order to support that, we need to pack everything in need into
TF.zip.
Steps to test the CL.
a) Choose a target that has both AVB and DTBO enabled.
$ m dist
b) Check IMAGES/dtbo.img and PREBUILT_IMAGES/dtbo.img both exist in the
generated out/dist/TF.zip.
c) Remove the entries under IMAGES/ from the generated TF.zip.
$ zip -d TF.zip IMAGES/\*
d) Re-generate the images with TF.zip.
$ build/make/tools/releasetools/add_img_to_target_files.py TF.zip
e) Check that IMAGES/dtbo.img is re-generated, and it's identical to the
image in b). Note that by default the re-generated image will carry a
different footer, because of the random salt. This CL is verified by
specifying the same salt.
Bug: 38315721
Test: see above.
Change-Id: I0bdc4e1cd4800962dc3902ca550dad6a8ca56c78
img_from_target_files.py used to handle the case that a given TF.zip not
containing the image entries under IMAGES/. That is only the case for
pre-Lollipop releases.
Also unzip the needed files only since we know that for sure now.
Test: img_from_target_files.py with an existing bullhead-TF.zip gives
the same bullhead-img.zip.
Change-Id: I892379ba388df80ae63be9d3ce647fbb77fd4753
Also pack the test keys for easier testing.
Bug: 38315721
Test: m otatools-package and avbtool is present in otatools.zip.
Change-Id: Ieb63bf3f4bc211ef1f48ab278cb01b70845d06da
Currently we're building the boot/recovery image twice, which is
redundant. And b/38455129 shows a problematic case when the image
from two builds doesn't match. We should only build the recovery
image once in the add_img_to_target_files.
Bug: 62021378
Test: call sign_target_files_apk on an angler target file,
recovery-from-boot.p generates successfully; and SHA of recovery.img
matches the one in install-recovery.sh.
Change-Id: I01e033501d80c18a87cbb870300eee5c19a04441
If we pass "rebuild_recovery" to add_img_to_target_files, the recovery
patch is rebuilt. But related files under SYSTEM/ (e.g.
SYSTEM/recovery-from-boot.p && SYSTEM/bin/install-recovery.sh) are not
updated.
This may cause a mismatch between system.img and SYSTEM/, and
may lead to a failure in validate_target_files.py.
Bug: 62096364
Test: Rebuild the system image in the TF and observe the recovery files
under SYSTEM/ get updated.
Change-Id: I7d679a612a86d02cf2eff81d1d120c0067138ed9
`make custom_images` supports to build different kinds of *non-droid* images,
e.g., odm.img. Adding the support of signing them with either AVB HASH footer
or AVB HASHTREE footer. The user can use HASH for small images and
HASHTREE for large images.
Sample signing configurations:
* AVB HASH footer:
- CUSTOM_IMAGE_AVB_HASH_ENABLE := true
- CUSTOM_IMAGE_AVB_ADD_HASH_FOOTER_ARGS := --append_to_release_string my_odm_image
* AVB HASHTREE footer:
- CUSTOM_IMAGE_AVB_HASHTREE_ENABLE := true
- CUSTOM_IMAGE_AVB_ADD_HASHTREE_FOOTER_ARGS := --fec_num_roots 8
* Using custom signing key:
- CUSTOM_IMAGE_AVB_ALGORITHM := SHA256_RSA2048
- CUSTOM_IMAGE_AVB_KEY_PATH := external/avb/test/data/testkey_rsa2048.pem
Bug: 36701014
Test: `make custom_images` with AVB HASH footer
Test: `make custom_images` with AVB HASHTREE footer
Test: `make droid` to check system.img is still properly signed with AVB HASHTREE
Test: `make droid` to check vendor.img is still properly signed with AVB HASHTREE
Change-Id: I8dc420e12e37e9a631345c0cd883339db05d489f
package_extract_dir is used in file based OTA only and should be killed.
Bug: 37959785
Test: code search shows no usage of this function in aosp.
Change-Id: Id3719b969c24b7ecef0c7f0e4a3af09a72be54d4
We used to check for 'attr >> 16 == 0xa1ff' (i.e. 0o120777) to detect
symlinks in the input target_files zip (TF.zip). This becomes broken
after we switch to soong_zip, which packs symlinks with 0o120700.
This CL fixes the issue by using stat.S_ISLNK() instead.
Note that we don't need to stage the files with the exact permission
bits as in the input TF.zip. Because this part is covered by mkbootfs
by using the canned or the compiled-in fs_config - as long as the
files/directories are accessible and the symlinks are created.
Bug: 38455129
Test: sign_target_files_apks.py on bullhead TF.zip. Check the
checksums in SYSTEM/bin/install-recovery.sh.
Change-Id: I51c1fc9a257fb3f18c16c2ed71528abaa6f7d9c9
(cherry picked from commit 406050bdb6)
We used to check for 'attr >> 16 == 0xa1ff' (i.e. 0o120777) to detect
symlinks in the input target_files zip (TF.zip). This becomes broken
after we switch to soong_zip, which packs symlinks with 0o120700.
This CL fixes the issue by using stat.S_ISLNK() instead.
Note that we don't need to stage the files with the exact permission
bits as in the input TF.zip. Because this part is covered by mkbootfs
by using the canned or the compiled-in fs_config - as long as the
files/directories are accessible and the symlinks are created.
Bug: 38455129
Test: sign_target_files_apks.py on bullhead TF.zip. Check the
checksums in SYSTEM/bin/install-recovery.sh.
Change-Id: I51c1fc9a257fb3f18c16c2ed71528abaa6f7d9c9
Clear fs_config_generate_extra_partition_list after use.
Test: build fs_config_files and fs_config_dirs targets.
Change-Id: Ie18bcd2df2ff1becb178e9ba45d226e2e9a8faab
Signed-off-by: William Roberts <william.c.roberts@intel.com>
* Add logic to handle decrypted keys from common.GetKeyPassword in
WriteABOTAPackageWithBrilloScript.
* Get the keys passwords in main and store them in OPTIONS.key_passwords.
This allows accessing them in WriteABOTAPackageWithBrilloScript and SignOutput
so it's only required to ask for the password once, while allowing to use
decrypted signing keys.
Test: ota_from_target_files.py on marlin and angler respectively.
Change-Id: I7c9b0198855a4b630c52b8552e904f312f09c4ce
The script prints the meta info, new/patch data size. It also simulates
the transfer commands and calculates amount of I/O as well as cache
needed.
We can add the more options to parse system/vendor.map so that we can
analyze the location change of files.
Bug: 31514709
Test: parse a bullhead incremental/full ota.
Change-Id: I70ed4f4c15e595b1851109e9799d44ac4e815c16
This is useful for devices with low disk space with different
build variants.
Bug: 37469715
Test: Regular image builds successfully, errors occur when
the headroom size is greater than available partition space.
Change-Id: I526cdd0f84981bbd16e3afcfe1cd7fc43dce98ef
Caller can optionally specify the verbose flag which overrides
OPTIONS.verbose. The command line won't be outputed with verbose=False.
This is useful for cases that a) those command lines are less useful
(but will spam the output otherwise); b) sensitive info is part of the
invocation.
'verbose=False' will be consumed by common.Run() only, instead of being
passed to subprocess.Popen().
Test: ota_from_target_files.py on a block based OTA.
Change-Id: I7d5b4094d756a60f84f89c6a965e7ccc68e435f8
Xml files will be generated instead of html files, and then Settings
will generate html file on runtime from xml files of partitions.
Test: build succeeded and tested on sailfish
Bug: 37099941
Merged-In: Id7899381a537b1947cbb6164ed2b4a98492334af
Change-Id: Id7899381a537b1947cbb6164ed2b4a98492334af
(cherry picked from commit 3b7560bf88)
Xml files will be generated instead of html files, and then Settings
will generate html file on runtime from xml files of partitions.
Test: build succeeded and tested on sailfish
Bug: 37099941
Change-Id: Id7899381a537b1947cbb6164ed2b4a98492334af
find_android_root works only when warn.py is found
so it works for build bot, but not when warn.py is
run against a stand alone build.log file.
Now get the TOP directory path from build.log
if it is found.
Bug: 37725427
Test: warn.py --gencsv build.log
Change-Id: I7b776c2e71a8d8d769f0600b08abe37acbd6ce7c
Misspelled fs_config_generate_extra_partition_list when used to
calculate LOCAL_REQUIRED_MODULES dependencies. This resulted in
fs_config_dirs and fs_config_files failing to generate the vendor,
oem or odm variants (ie: fs_config_files_vendor)
Test: build generates correct fs_config override files.
Bug: 37721445
Bug: 36071012
Change-Id: I053dd5cdb357c7cea05a5ff63b0686bd0bc99df6
This reverts commit b1e59b8a73.
This CL enables generating compatibility zip entry in an A/B OTA
package. We're not enabling the support for non-A/B OTAs for now until
they have the necessary support (e.g. /proc/config.gz exists).
Bug: 36810913
Test: Generate full and incremental packages for marlin and applly under
normal boot and sideload respectively.
Change-Id: I69f0a3feb7a0efc7f727f92f31c9e63f47ee6530
(cherry picked from commit b63c952cc7)
Add build targets to generate the vendor, oem and odm slices of the
fs_config_dirs and fs_config_files targets for each of the respective
filesystems, and automatically trigger the individual and enabled
partition build objects based as dependents of fs_config_dirs and
fs_config_files.
Sensitive to the following board definition pairs:
BOARD_VENDORIMAGE_FILE_SYSTEM_TYPE BOARD_USES_VENDORIMAGE
BOARD_OEMIMAGE_FILE_SYSTEM_TYPE BOARD_USES_OEMIMAGE
BOARD_ODMIMAGE_FILE_SYSTEM_TYPE BOARD_USES_ODMIMAGE
Which when defined and set means the images are generated for vendor,
oem or odm respectively.
Test: manually build w/ m 'fs_config_(dirs|files)_(vendor|oem|odm)'
and check the expected products, or fails if not pertinent.
Test: build for product that produces split to system and vendor and
subsequently boots with vendor products correctly configured.
Bug: 36071012
Change-Id: I44289a3dd50b7657367c826ad813c671a3de6afc
Add a -P option to fs_config_generate that allows us to filter on
a comma separated partition whitelist or minus prefixed -blacklist.
A partition is defined as accessible as <partition>/ or
system/<partition>/.
No change for legacy, where no -P flag is specified.
For system.img, but not vendor, oem or odm file references:
fs_config_generate -P -vendor,-oem,-odm
For vendor.img file references:
fs_config_generate -P vendor
For oem.img file references:
fs_config_generate -P oem
For odm.img file references:
fs_config_generate -P odm
Add and fortify tests for each of these cases.
SideEffects: none
Test: gTest host fs_config-unit-test
Bug: 36071012
Change-Id: I0537453bb289d41ef98ad76e69f78ae49358a1b3
This reverts commit b1e59b8a73.
This CL enables generating compatibility zip entry in an A/B OTA
package. We're not enabling the support for non-A/B OTAs for now until
they have the necessary support (e.g. /proc/config.gz exists).
Bug: 36810913
Test: Generate full and incremental packages for marlin and applly under
normal boot and sideload respectively.
Change-Id: I69f0a3feb7a0efc7f727f92f31c9e63f47ee6530
Bug: None
Test: Ran warn.py on a build log. Code links in resultant HTML open in
new tabs when clicked.
Change-Id: Ia3def5ea14fe216700cb2ffabe4a39b98e69d55a
This reverts commit 853cd306b0.
Temporarily reverting the CL to work around the bug in recovery code.
Bug: 37413730
Change-Id: I8e77112ffafcf32332794739ee1c855cc83b32b0
Uses avbtool to sign vendor.img if BOARD_AVB_ENABLE is set.
It also allows appending additional arguments to avbtool via
BOARD_AVB_VENDOR_ADD_HASHTREE_FOOTER_ARGS.
e.g.,
BOARD_AVB_ENABLE := true
BOARD_AVB_VENDOR_ADD_HASHTREE_FOOTER_ARGS := --generate_fec
Bug: 35415839
Test: "make" with the above variables and use avbtool to check vbmeta is
appended to vendor.img
Test: "make dist" with the above variables
Change-Id: I8ada38dff3def6d34613e77c67944def8a49f464
Merged-In: I8ada38dff3def6d34613e77c67944def8a49f464
(cherry picked from commit 8ee4a3db8c)
Uses avbtool to sign vendor.img if BOARD_AVB_ENABLE is set.
It also allows appending additional arguments to avbtool via
BOARD_AVB_VENDOR_ADD_HASHTREE_FOOTER_ARGS.
e.g.,
BOARD_AVB_ENABLE := true
BOARD_AVB_VENDOR_ADD_HASHTREE_FOOTER_ARGS := --generate_fec
Bug: 35415839
Test: "make" with the above variables and use avbtool to check vbmeta is
appended to vendor.img
Test: "make dist" with the above variables
Change-Id: I8ada38dff3def6d34613e77c67944def8a49f464
Store the metadata in a archive within the OTA package so the collection
can be retrieved as a single unit by the device.
Bug: 36810913
Test: `make dist` marlin (ab) and angler (non-AB) for incremental and full OTA.
Change-Id: I94118e48a3c6ed5ff890b67d7dafbca02346aeea
While we are generating the html file also expose a way for a user to
save off a csv file. This prevents having to run the tool twice.
* Add a new flag called csvpath that takes a file path for the csv dump.
* Use csv.writer to simply print formats.
* Modify the dump_csv and count_severity to take a csv writer.
Test: run with a build.log file, compare before and after html/csv dumps.
Change-Id: I27ed730625a352fdc3fb2928433eb5e8b2ea229c
This patch updates the logic to use the PRODUCT_IOT variable instead of
BRILLO.
Bug: 36702887
Test: `make`; Image doesn't have the dev key.
Change-Id: I1751e97d8cdfeba83c7e4720a017a5f4dcfd49da
This CL is to generate every static RRO package for its target package
automatically at build-time.
BOARD_ENFORCE_RRO build variable is added to specify whether enforcing
RRO is required or not.
BOARD_ENFORCE_RRO_EXEMPT_SOURCES build variable is added to specify
the module list of which item should be exempt from enforcing RRO.
Test: tested on bullhead and sailfish
Bug: 34097942
Change-Id: I455b2ce34e66c57a540c299b5e378b7c4e78d5b8
(cherry picked from commit 3070610b72)
fileslist.go is still disabled by default. To enable, use:
USE_FILESLIST_GO=true m -j
Bug: 36274890
Test: Manual, with m -j
Change-Id: I722e17fa7fb6ba91e2b2c44cd7524d974cbe4a56
(cherry picked from commit 9fe97e231b)
The test broke due to api change in constructor of Transfer.
Bug: 36456381
Test: blockimgdiff test passes
Change-Id: Ic73b66f8fb6141ff6f50c113b0d8cc55d34e0b5d
fileslist.go is still disabled by default. To enable, use:
USE_FILESLIST_GO=true m -j
Bug: 36274890
Test: Manual, with m -j
Change-Id: I722e17fa7fb6ba91e2b2c44cd7524d974cbe4a56
We've just created the target files package and still have the extracted
directory, so pass that to ota_from_target_files so that it doesn't have
to re-extract what we just compressed.
This saves a little bit of time -- for bullhead-userdebug on internal
master, this brings the time to build an OTA from ~340s -> ~310s. Much
of the time is still spent generating and signing the OTA.
Test: lunch bullhead-userdebug; m -j otapackage
Test: bullhead-ota-*.zip is identical before/after
Change-Id: Ib51cf6b02123e9c49280e9957c6590645470a52c
system_root_image has been dead since commit
2ce63edab7.
Test: pylint --rcfile=pylintrc add_img_to_target_files.py
Change-Id: Id791747b7313923b82279a21d264a998455a92d4
When bsdiff/imgdiff fails, dump the output along with the src/tgt
filenames and block ranges. Also, exit the script gracefully instead
of raising an exception about iterating over 'NoneType'.
Bug: 31381530
Test: Package generates successfully for angler, and error outputs
correctly with error injection.
Change-Id: I06a2dfe545fbdff7043de05fee34b378453a9291
This CL is to generate every static RRO package for its target package
automatically at build-time.
BOARD_ENFORCE_RRO build variable is added to specify whether enforcing
RRO is required or not.
BOARD_ENFORCE_RRO_EXEMPT_SOURCES build variable is added to specify
the module list of which item should be exempt from enforcing RRO.
Test: tested on bullhead and sailfish
Bug: 34097942
Change-Id: I455b2ce34e66c57a540c299b5e378b7c4e78d5b8
The line was unintentionally removed in commit 7f804ba.
Test: ota_from_target_files.py generates a full OTA. Check the package
metadata.
Change-Id: Icae88e2a9bb2bfc450a3d0d7ab524d6a6eac9df5
BBOTA v1 and v2 (introduced in L and L MR1 respectively) don't support
resumable OTA. We shouldn't generate packages using v1/v2 at the risk of
bricking devices.
BBOTA v3 (since M) and v4 (since N) both support resumable OTAs. BBOTA
v4 additionally supports using FEC to possibly recover a corrupted
image.
Bug: 33694730
Test: Generate full and incremental OTAs w/ and w/o the CL. They should
give identical packages (in v4).
Change-Id: Ib89d9cd63ba08e8e9aa4131bed18876b89d244c0
Remove the following functions that are needed for file-based OTAs only:
- SetPermissions()
- SetPermissionsRecursive()
- MakeSymlinks()
- DeleteFiles()
- DeleteFilesIfNotMatching()
- RenameFiles()
- SkipNextActionIfTargetExists()
Bug: 35853185
Test: Verified there's no reference to these functions.
Change-Id: Iff24a9d705476211effaef28eed2a147fa5fcbce
In addition to the current behavior of add_img_to_target_files working
on an existing zip file, allow passing in a directory where the target
files have already been extracted. When in this mode, it writes the
images out to that directory instead of the zip file.
This allows us to call add_img_to_target_files on the temp directory
used during the build to create the target files package, saving the
time and space of unzipping what we just zipped. This also allows us to
use the parallel soong_zip, which compresses the images much faster.
Test: aosp_marlin target_files zip is the same before/after this change
Test: marlin target_files zip is the same before/after this change
Test: bullhead target_files zip is the same before/after this change
Change-Id: I155654cdc7ad7754ba4ef76ec69c31f504a58198
We have stopped shipping devices with file-based OTAs, and are not
actively maintaining the support. Devices using file-based OTAs
should be moved to block-based, if not A/B OTAs.
We will also need to clean up EdifyGenerator class, which will be
handled in follow-up CLs.
Bug: 35853185
Test: Generate full and incremental OTAs w/ and w/o the CL, and they
give identical packages.
Test: Not specifying --block also generates block-base OTAs.
Change-Id: I3b0fc8ce5600e109f3251fe41f655534aaa298c7
We must have created the images (system.img, system.map etc) prior to
calling ota_from_target_files.py (added by commit
2e0d8fcf08, into Lollipop).
Remove the obsolete suppport for handling "old" target_files zips that
don't have such images. This simplies the logic for BuildSystem() and
BuildVendor(), which now would only be called by
add_img_to_target_files.py itself.
Test: Generating full and incremental OTAs give the same results, w/ and
w/o this CL.
Change-Id: I0ea243d09d3378249d7982701ae4ec339b04b7b8
The first one in WriteVerifyPackage() is mismatching function parameters
that can be trivially fixed.
The other one is in WriteABOTAPackageWithBrilloScript(), where we don't
have edify script instance.
Test: `pylint --rcfile=pylintrc ota_from_target_files.py`.
Change-Id: Ie238ef5b296dfe9e725b61376992447b662d2376
The major issue with the existing implementation is unnecessarily
holding too much data in memory, such as HashBlocks() which first reads
in *all* the data to a list before hashing. We can leverage generator
functions to stream such operations.
This CL makes the following changes to reduce the peak memory use.
- Adding RangeSha1() and WriteRangeDataToFd() to Image classes. These
functions perform the operations on-the-fly.
- Caching the computed SHA-1 values for a Transfer instance.
As a result, this CL reduces the peak memory use by ~80% (e.g. reducing
from 5.85GB to 1.16GB for the same incremental, as shown by "Maximum
resident set size" from `/usr/bin/time -v`). It also effectively
improves the (package generation) performance by ~30%.
Bug: 35768998
Bug: 32312123
Test: Generating the same incremental w/ and w/o the CL give identical
output packages.
Change-Id: Ia5c6314b41da73dd6fe1dbe2ca81bbd89b517cec
This reverts commit a7316ce094.
This CL differs from the original CL by not unzipping RADIO/*. This is
because: a) AOSP targets don't have RADIO/ entries in the TF.zip; b)
we're not using the unzipped RADIO files (but reading them from the zip
files directly) - checked all the device-specific releasetools for
angler, bullhead, ryu, shamu, volantis, fugu, marlin and sailfish.
Test: `m dist` with AOSP targets (tested fugu and bullhead).
Change-Id: I4d0c67214ddd6202fc27c71bb79f52b5f4d40c64
This reverts commit aa3a04f19d.
Reason for revert: Some AOSP targets don't include RADIO/ in the TF.zip. We may possibly skip unzipping RADIO/, or by always creating a dummy RADIO folder in TF.zip. Revert this CL for now.
Change-Id: I8e90d322706a4fe82129bdfab5ffa1eab989c648
When building BBOTAs, it only needs *some* unzipped entries in the given
target_files zip(s). In particular, it needs 'IMAGES/*', 'META/*',
'RADIO/*'. (It also reads 'SYSTEM/build.prop' and 'OTA/bin/updater', but
directly from the zip file.)
This CL specifies the entries to unzip. It saves the I/O cost, as well as
the temporary storage.
Test: ota_from_target_files.py gives the same package w/ and w/o the CL.
Test: check_target_files_signatures.py still works.
Change-Id: I728428aa0e138879e49f9efbdb46a85892fc7038
external/dbus and external/dbus-binding-generator was removed from
the manifest.
Bug: 31602715
Test: make checkbuild
Change-Id: Iea0277720acad8ac0fa630b8745f90fb3c3b5f00
This remove the fstab dependency when building the OTA package for
marlin/sailfish.
Bug: 35811655
Test: OTA package builds successfully for sailfish.
Change-Id: If223d11dddca396c47262042c576f9e7d0cb5b33
(cherry picked from commit 7d051adc3b)
fstab_version is defined by RECOVERY_FSTAB_VERSION in
bootable/recovery. We have moved to fstab_version 2 since commit
f35d1cef7c19db975a1295e8c23c7fb8bd2489f9 (landed into JB MR2).
Drop the support for fstab_version 1, since we won't run the latest OTA
script over a JB target_files zip.
Test: No impact on building full/incremental OTAs.
Change-Id: Ia87c4e7da6c5e71ce0908fca2e4f1ad1c06ba869
This remove the fstab dependency when building the OTA package for
marlin/sailfish.
Bug: 35811655
Test: OTA package builds successfully for sailfish.
Change-Id: If223d11dddca396c47262042c576f9e7d0cb5b33
The merged two branches have become identical since commit
fc3422ad36 (landed into Nougat).
Test: Get identical incremental packages w/ and w/o the CL.
Change-Id: Id1183f8ed83f684a0dac1a4af87b6e075b08aabc
We use the timestamps in builds to determine a downgrade, which might
not be always the truth. For examples, two builds cut from different
branches may carry timestamps in a reverse order. An incremental package
won't be able to be pushed nor applied, based on the timestamp
comparison.
We used to handle such a case with manual work, by setting the
post-timestamp to (pre-timestamp + 1) in the package metadata. This CL
automates the process by adding a new flag --override_timestamp.
Note that it doesn't change anything in the installed image, but only
affects the assertions for pushing / installing the package.
With the change in this CL:
- If it's a downgrade without any extra flag, fail the package
generation (we only print warnings prior to this CL);
- If it's a downgrade with --downgrade flag, generate a downgrade
package with forced data wipe (same as before);
- If it's a downgrade with --override_timestamp, generate a normal
incremental with hacked timestamp (pre-timestamp + 1) (new in this CL
to avoid the manual change);
- If it's not a downgrade but with any of the above two flags specified,
fail the package generation.
Bug: 33744169
Test: Generate an incremental from builds with reversed timestamps.
Change-Id: I8b187d32708b4a7c3e20f8c6adb8f9527b73b965
We introduced META/misc_info.txt to hold the misc info since Gingerbread
(commit 37974731fc). Remove the backwards
compatibility support for building pre-G TF zips.
Test: `m dist` works.
Change-Id: Ibff7aaf69cc7e460634c049d11a004f7196f8f73
Otherwise the comparison is inconsistent between ReviseStashSize() and
WriteTransfers().
Bug: 35775675
Test: Successfully generate a previously failed incremental.
Change-Id: I554a51a210bf322cb5c79e28cf85607a417b094a
all_named_products was giving the file path instead of product name for
the name:path format of PRODUCT_MAKEFILES.
kati_all_products has been replaced with multiproduct_kati
Test: get_build_var all_named_products
Test: multiproduct_kati
Change-Id: I24015ef0778ac7cd45201aa55c1737b0553f09fe
This reverts commit d6145b6a16.
The cause of the earlier breakage has been identified and fixed by
commit 4b577413ff.
Bug: 35467608
Test: lunch ryu-userdebug; m dist otatools-package
This is a partial revert to the last makeparallel change, since now we
were being killed by the signal before waiting for our child to exit. So
instead of not installing the handlers, only pass the signal along if
it's a SIGTERM.
Bug: 35214134
Test: Ensure that we're still only getting one signal for SIGINT
Test: Ctrl-C, ensure that all the soong_ui lines are before the make
error line.
Change-Id: I26fff9483a3abfd79ceb5a9ea47e3f7572d9e923
As part of the soong_ui effort, we noticed that we'd sometimes see
multiple SIGINTs from a single ctrl-c from a user. ctrl-c sends a SIGINT
to the entire process group, so make, makeparallel, soong_ui, and all of
its children would get a signal. Since makeparallel was passing it along
to it's child, soong_ui would get two signals.
So instead, follow what Make does and only pass along SIGTERM. Assume
that all other signals went to the entire process group.
Bug: 35214134
Test: Send SIGINT to process group, check in makeparallel's child for
more than one signal received.
Change-Id: I5b2a77ad0fcebbaa5087439948e71bf3b541061a
This CL changes the --oem_settings flag to allow a comma seperated list of
property files. All property values will be used when asserting properties such
as ro.product.name.
For example, if two property files are provided with ro.product.name values of
"sprout" and "sprout_a", the resulting otapackage will check that the device's
ro.product.name property matches at least one of them.
Bug: 34191373
Test: manual
Change-Id: I954673511be8f0929982235cc9cbfbd85a9ee1f4
Commit e98fb7a8d3 switched to using
futility-host instead of the prebuilt futility. This CL adds support to
handle signing old TF.zip that still says "futility=prebuilt/..." in
META/misc_info.txt.
Bug: 35467608
Test: Generate otatools.zip and sign an old ryu TF.zip.
Change-Id: I48a9cc918c7afce361e1ec9bc4f85f74fa92566e
(cherry picked from commit 4f104d1b94)
Commit e98fb7a8d3 switched to using
futility-host instead of the prebuilt futility. This CL adds support to
handle signing old TF.zip that still says "futility=prebuilt/..." in
META/misc_info.txt.
Bug: 35467608
Test: Generate otatools.zip and sign an old ryu TF.zip.
Change-Id: I48a9cc918c7afce361e1ec9bc4f85f74fa92566e
Tao Bao