Bug: 68224784
Test: Enable AVB on aosp_bullhead-userdebug. `m dist`. Note that the
prebuilt vendor.img needs to be AVB-signed first.
Test: `m dist` with aosp_walleye-userdebug.
Change-Id: I18235e4d0dde6af71a96e46ec434480cc9a22bef
AddImagesToTargetFiles() takes either a zip file, or a zip root as
input. We used to create IMAGES/ directory only when working with zip
root input. Commit 262bf3f0b5 has changed
to also stage boot / recovery images there when working with a zip file.
This CL makes sure the directory is always available under both modes.
Bug: 63456822
Test: zip -d target_files.zip IMAGES/\*;
add_img_to_target_files.py target_files.zip
Test: sign_target_files_apks.py target_files.zip signed-target_files.zip
Change-Id: Iea91d0403cdec1b16bb93bb71d3ed06856b8f7c3
Most logtags files only have entries with hardcoded numbers.
Allow these files to be converted to java without needing the
merged logtags file.
Test: m checkbuild
Change-Id: I4cd38aa502fc9b8e50af92ff8296d3ad6abfb56e
If only vbmeta.img is missing from the target_files.zip,
'add_img_to_target_files.py -a' will fail to re-create vbmeta.img.
++++ vbmeta ++++
Traceback (most recent call last):
File "./build/make/tools/releasetools/add_img_to_target_files.py", line 693, in <module>
main(sys.argv[1:])
File "./build/make/tools/releasetools/add_img_to_target_files.py", line 687, in main
AddImagesToTargetFiles(args[0])
File "./build/make/tools/releasetools/add_img_to_target_files.py", line 584, in AddImagesToTargetFiles
boot_contents = boot_image.WriteToTemp()
AttributeError: 'NoneType' object has no attribute 'WriteToTemp'
Because it has skipped loading the boot.img, which leads to the above error.
Bug: 63456822
Test: As follows:
$ zip -d target_files.zip vbmeta.img
$ add_img_to_target_files.py -a target_files.zip
Test: `m dist` on aosp_arm64-userdebug
Change-Id: Ieba5f0b6848c3ec5d8bcfd24d48cc5af8b39f06c
When generating full OTAs for AVB-enabled targets, it fails the size
assertion for boot.img.
ERROR: boot size (31457280) is 100.00% of limit (31457280)
The size assumption is no longer true for AVB-signed images, because
the actual image size should be always identical to the limit.
Bug: 67841177
Test: Enable AVB on bullhead. `m dist`.
Change-Id: I9bd432ccd88094628eb71308554839549eccb6d9
With the new implementation of handling large apks, we need to call
imgdiff with block-limit to split the apk and generate the patch at
the same time. The call to imgdiff would significantly increase the
time consumption of the "FindTransfers" function which we used to
execute sequentially. This cl addresses this issue and speeds up the
process by making the imgdiff call parallel.
Bug: 34220646
Test: Create and sideload an incremental package for angler
Change-Id: Id62e348418fc1d22e32ea6c8ac16d9ab3ec92d7b
With the update of brotli to version 1.0.1 the new binary tool is now
called "brotli". This patch updates the scripts to use the new name.
Bug: 34220646
Test: `make dist`; ota_from_target_files on non-A/B device.
Change-Id: Ie8da4333388d029d5960a723a55efd875d8ee820
When signing a target_files.zip, the OTA certificate specified by
default_system_dev_certificate could be replaced with a mapped key. When
that happens, we must explicitly specify --package_key when generating
OTA packages with ota_from_target_files.py. Otherwise the OTA package
will be signed with the wrong key, which leads to verification failures.
This CL updates the default_system_dev_certificate value in
misc_info.txt accordingly.
Test: Sign a target_files.zip and replace the OTA key. Check
META/misc_info.txt in the generated target_files.zip.
$ ./build/make/tools/releasetools/sign_target_files_apks.py -v \
--replace_ota_keys \
-k build/target/product/security/testkey=build/target/product/security/platform \
out/dist/aosp_marlin-target_files-eng.tbao.zip \
signed-marlin-target_files-test.zip
Change-Id: I093234b5add3e27c5b3887cefeffd74e6f0a3e98
test_common constructs a few 2GiB strings in memory, which leads to huge
memory footprint (18GiB). This CL moves away from in-memory strings to
generators, which reduces the memory use down to 41MiB. It also reduces
the time cost from 294s to 139s as an extra benefit for free.
The CL addresses some trivial pylint warnings as well.
* Before
$ /usr/bin/time -v python -m unittest -v test_common
...
----------------------------------------------------------------------
Ran 11 tests in 294.986s
OK
Command being timed: "python -m unittest -v test_common"
User time (seconds): 110.51
System time (seconds): 109.34
Percent of CPU this job got: 74%
Elapsed (wall clock) time (h:mm:ss or m:ss): 4:55.06
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 18894172
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 1
Minor (reclaiming a frame) page faults: 20774908
Voluntary context switches: 48
Involuntary context switches: 3241
Swaps: 0
File system inputs: 184
File system outputs: 8406424
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
* After
$ /usr/bin/time -v python -m unittest -v test_common
...
----------------------------------------------------------------------
Ran 11 tests in 139.100s
OK
Command being timed: "python -m unittest -v test_common"
User time (seconds): 59.00
System time (seconds): 4.73
Percent of CPU this job got: 45%
Elapsed (wall clock) time (h:mm:ss or m:ss): 2:19.17
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 41252
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 106569
Voluntary context switches: 44
Involuntary context switches: 103
Swaps: 0
File system inputs: 8
File system outputs: 8422808
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
Fixes: 68988396
Test: See above.
Change-Id: I00f16603a4ee59fb085b189c6f5b5ee9d2378690
imgdiff fails on plain strings after we put more checks the input
data. This leads to failure when running unittest
"test_recovery_from_boot". The CL fixes the issue by serving gzipped
dummy images as the test input.
Bug: 68988329
Test: InstallRecoveryScriptFormatTest passes
Change-Id: I1952b311681ac4f39c0e51aef1de1177aebeac43
$ python rangelib.py
**********************************************************************
File "rangelib.py", line 323, in __main__.RangeSet.next_item
Failed example:
list(rangelib.RangeSet("10-19 3 5 7").next_item())
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.7/doctest.py", line 1315, in __run
compileflags, 1) in test.globs
File "<doctest __main__.RangeSet.next_item[2]>", line 1, in <module>
list(rangelib.RangeSet("10-19 3 5 7").next_item())
NameError: name 'rangelib' is not defined
**********************************************************************
1 items had failures:
1 of 3 in __main__.RangeSet.next_item
***Test Failed*** 1 failures.
Test: python rangelib.py
Change-Id: I6c791ff4453f40d90b34f97b881460da9878fe64
"ro.vendor." prefix will be used only for vendor-specific properties
except for ro.vendor.build.date, ro.vendor.build.date.utc and
ro.vendor.build.fingerprint.
Those exceptions have been used since Android L. So we can't rename them
to support the system-only upgrade from old Android.
Bug: 36796459
Test: confirmed that ro.vendor.product.* were changed to
ro.product.vendor.* in vendor/build.prop
Merged-In: Ibc107b00264a2dc194261d1968b27a9b17b5d798
Change-Id: Ibc107b00264a2dc194261d1968b27a9b17b5d798
(cherry picked from commit 8cf6443264)
It has been long deprecated since commit
fdd8e69c42 (Donut).
Test: `m dist` with aosp_marlin-userdebug and aosp_angler-userdebug.
Test: No user of "--board_config" in code search.
Change-Id: Ic0336c22cac32de5de88161748d09b579f892e99
Commit b937ead5d9 added the fallback to
bsdiff on imgdiff failures. However, it missed setting the transfer
style accordingly, which led to patch header mismatch.
Bug: 68659848
Test: Generate an incremental that has the fallback from imgdiff to
bsdiff. Examine the generated transfer list and verify that it has
"bsdiff" for the fallback command.
Change-Id: I55e46879d590a8af82ea796b9d98ffdb30360408
Following the new API contract, this effectively add extra padding
before central dir to make it 4KB aligned.
Test: build succeeded
Bug: 30972906
Change-Id: I7cac9d2c4371b473c88df867b3b2ae906443db10
Use zip_archive directly. Note that this codepath is used only
when recompressing archives with zopfli during the alignment step.
It's unclear whether this is in use at all, but I verified that the
results are identical (note the usage of the "-z" flag in the test
below).
Test: make && out/host/linux-x86/bin/zipalign -v -f -z 8 \
out/target/product/marlin/system/app/Email/Email.apk ./out.zip
Bug: 35246701
Change-Id: I641cdb6d409cc07974d49d42c9f9e6d4f905e472
Duplicate essential product sysprops for manufacturer, model, brand, name, and device
from system on vendor so that mixed builds (vendor.img + GSI system.img) have correct
product information in their CTS/VTS reports.
Test: make
Bug: 64458205
Merged-In: Ib63b37772be493b9e035d9b7e8c5e2cf66f2fa8c
Change-Id: Ib63b37772be493b9e035d9b7e8c5e2cf66f2fa8c
(cherry picked from commit aea0f92ff3)
When generating block based OTAs, we read files from the sparse image
directly with the help of block map file. However, the block map info
might not be accurate if the image is created with mke2fs. Because
mke2fs may skip allocating actual blocks if they contain all zeros.
ota_from_target_files.py consequently passes incomplete APK files to
imgdiff, which fails to generate patches.
This CL works around the issue by falling back from imgdiff to bsdiff on
failures. We should figure out a better way in b/68016761 to remove the
workaround, which would otherwise hide other issues in imgdiff.
Bug: 67824829
Bug: 68016761
Test: ota_from_target_files.py passes on previously failing TF zips.
Change-Id: Ib24c5b5f89812b97a0c87c6bf0dc147ae39bc92f
Test: `m dist` on Linux and macOS.
Test: Use an existing target_files.zip for bullhead. Remove IMAGES/*
and rebuild the images with 'add_img_to_target_files.py -a'. It
generates idential images as in the original TF.zip.
Change-Id: I803656bf2c924dce53f2271dcb967a94fff48440
We have added support in mkuserimg_mke2fs.sh that allows specifying
filesystem UUID and hash seed UUID. This CL generates and passes the
UUIDs based on the image name and build fingerprint. This way we can
rebuild and get identical images.
Note that this only applies to images generated with `m dist` and with
signing tools (sign_target_files_apks.py and
add_img_to_target_files.py). Images under $OUT (built with `make`) are
not affected.
Bug: 64994964
Test: lunch marlin-userdebug; run `m dist` twice (after deleting the
intermediate files under $OUT/obj/PACKAGING), and compare the
generated images.
Change-Id: I41cf4e5869582bb930af2f35a8e9c79bff43b2a2
(cherry picked from commit 3aa21e6bb9)
Duplicate essential product sysprops for manufacturer, model, brand, name, and device
from system on vendor so that mixed builds (vendor.img + GSI system.img) have correct
product information in their CTS/VTS reports.
Test: make
Bug: 64458205
Change-Id: Ib63b37772be493b9e035d9b7e8c5e2cf66f2fa8c
Bug: 23102347
Bug: 34954705
Test: provide a fingerprint >92 characters long and see that it
successfully works
Change-Id: Ida9ffd5266acb3b432b208780804a08e8f7391b4
Adding verified boot metadata with a "disable magic". The resulting
metadata at the end of each image (e.g., system.img, vendor.img) will
be the same as triggering an "adb disable-verity" on an USERDEBUG image.
This can help simplify the code on fs_mgr, which won't have to check if
current image is an ENG build or not.
Bug: 63056044
Test: boot sailfish eng/userdebug builds
Change-Id: I95d23ac7b76c04d6d4483c9c4dc1de16bf0d9c3a
Use -q to silence zip comments, which also silences the first line.
Bug: 67345114
Test: find vendor -name '*.apk' | xargs build/tools/dump-package-stats
Change-Id: Icb319c603de7c29644c58520213940179f67b7e4
Use -q to silence zip comments, which also silences the first line.
Bug: 67345114
Test: find vendor -name '*.apk' | xargs build/tools/dump-package-stats
Change-Id: Icb319c603de7c29644c58520213940179f67b7e4
We want the generated images being identical for the same source files.
Currently the generated ext4 image (either from make_ext4fs or mke2fs)
is reproducible, but the AVB footer added by avbtool contain changes
because of the random salt being used.
This CL changes the avbtool invocation to specify "--salt <hexstring>"
(already supported by avbtool) to use reproducible salt that's computed
based on fingerprints (or thumbprints if applicable).
Bug: 67023482
Test: Regenerate images from the same source as follows:
Use a target_files.zip from an AVB-enabled target.
$ zip -d target_files.zip IMAGES/\*
$ ./build/make/tools/releasetools/add_img_to_target_files.py \
-v target_files.zip
Repeat the above commands and compare the generated images.
Change-Id: Id9db17ae0132ca3a820b4be5a5ef06ca3fef71ed
(cherry picked from commit 8f05cca1d9)
We want the generated images being identical for the same source files.
Currently the generated ext4 image (either from make_ext4fs or mke2fs)
is reproducible, but the AVB footer added by avbtool contain changes
because of the random salt being used.
This CL changes the avbtool invocation to specify "--salt <hexstring>"
(already supported by avbtool) to use reproducible salt that's computed
based on fingerprints (or thumbprints if applicable).
Bug: 67023482
Test: Regenerate images from the same source as follows:
Use a target_files.zip from an AVB-enabled target.
$ zip -d target_files.zip IMAGES/\*
$ ./build/make/tools/releasetools/add_img_to_target_files.py \
-v target_files.zip
Repeat the above commands and compare the generated images.
Change-Id: Id9db17ae0132ca3a820b4be5a5ef06ca3fef71ed
Merged-In: Id9db17ae0132ca3a820b4be5a5ef06ca3fef71ed
(cherry picked from commit 8f05cca1d9)
We have added support in mkuserimg_mke2fs.sh that allows specifying
filesystem UUID and hash seed UUID. This CL generates and passes the
UUIDs based on the image name and build fingerprint. This way we can
rebuild and get identical images.
Note that this only applies to images generated with `m dist` and with
signing tools (sign_target_files_apks.py and
add_img_to_target_files.py). Images under $OUT (built with `make`) are
not affected.
Bug: 64994964
Test: lunch marlin-userdebug; run `m dist` twice (after deleting the
intermediate files under $OUT/obj/PACKAGING), and compare the
generated images.
Change-Id: I41cf4e5869582bb930af2f35a8e9c79bff43b2a2
(cherry picked from commit 3aa21e6bb9)
libm is a default library for device builds, so default it for host
builds as well.
Also removes duplicate additions of -ldl, -lpthread, -lm and -lrt.
Test: m host
Change-Id: I6a07e12053090eb6997b79d4091c28ac9a9022de
vendor.img could be built from the source - in which case we have
unpacked files under VENDOR/ - or dropped in as a prebuilt binary blob.
We should consider either of them as target having a vendor partition.
Because we need to add its AVB hashtree info into vbmeta.img if target
is using AVB. Otherwise libfs_mgr would refuse to mount this
"AVB-enabled" vendor.img.
For targets not using AVB, this change is no-op.
Bug: 65462819
Test: Having vendor.img as prebuilt, `make dist`. Check that the
generated vbmeta.img contains the info from vendor.img.
Test: Build, flash and boot the above image.zip.
Change-Id: Iaeb30e2059cb33fb39f23e5ffd28f338d00ccbfc
Merged-In: Iaeb30e2059cb33fb39f23e5ffd28f338d00ccbfc
(cherry picked from commit b22afea0f0)
vendor.img could be built from the source - in which case we have
unpacked files under VENDOR/ - or dropped in as a prebuilt binary blob.
We should consider either of them as target having a vendor partition.
Because we need to add its AVB hashtree info into vbmeta.img if target
is using AVB. Otherwise libfs_mgr would refuse to mount this
"AVB-enabled" vendor.img.
For targets not using AVB, this change is no-op.
Bug: 65462819
Test: Having vendor.img as prebuilt, `make dist`. Check that the
generated vbmeta.img contains the info from vendor.img.
Test: Build, flash and boot the above image.zip.
Change-Id: Iaeb30e2059cb33fb39f23e5ffd28f338d00ccbfc
Make the build log less verbose; also print the result for
AdjustPartitionSizeForVerity().
Bug: 65292710
Test: Run add_img_to_target_files with and without "-v" and check
the logs.
Change-Id: Icdc86197ce69db036658ea9e77cc4ebe1750463b
We pack _all_ the APK certs info into META/apkcerts.txt at build time,
including the ones that are not installed for the target. When
detecting for compressed APKs, we should only count the ones that are
actually installed.
Bug: 65498015
Test: `check_target_files_signatures.py bullhead-target_files.zip`
Change-Id: I2f32d4667be0efbfd5f9365dde819f009572533d
When creating ext4 images with mke2fs, it may skip allocating some
blocks if they contain all zeros. As a result, there could be less
blocks listed in the block map than the actual file length.
For example, for a file with a length of 112200-byte (27+ blocks),
the listed blocks in block.map could be '43665-43688' (24 blocks).
Because some all-zero blocks are not taking actual space.
The generated ext4 images are perfectly valid - kernel will figure out
that data block is not allocated and writes all zeros into user buffer.
However, we can't fully reconstruct a file from its block list in our
Python script. Ideally this can be avoided by mounting or parsing an
ext4 image directly, which is yet to be supported in our script.
This CL skips checking for such files to avoid failing
validate_target_files.py.
Bug: 65213616
Test: validate_target_files.py passes on targets with mke2fs generated
images (e.g. marlin).
Change-Id: Id9cc59e345b9283844044ef94ceb5702f0ca0526
(cherry picked from commit b418c30e3a)
Some OWNERS files use secondary email addresses, so we only
check existence of _account_id and email attributes.
Test: run against existing OWNERS files
Change-Id: I6f0804a8a90b7a23783c3fe45686d4d973876ed1
When creating ext4 images with mke2fs, it may skip allocating some
blocks if they contain all zeros. As a result, there could be less
blocks listed in the block map than the actual file length.
For example, for a file with a length of 112200-byte (27+ blocks),
the listed blocks in block.map could be '43665-43688' (24 blocks).
Because some all-zero blocks are not taking actual space.
The generated ext4 images are perfectly valid - kernel will figure out
that data block is not allocated and writes all zeros into user buffer.
However, we can't fully reconstruct a file from its block list in our
Python script. Ideally this can be avoided by mounting or parsing an
ext4 image directly, which is yet to be supported in our script.
This CL skips checking for such files to avoid failing
validate_target_files.py.
Bug: 65213616
Test: validate_target_files.py passes on targets with mke2fs generated
images (e.g. marlin).
Change-Id: Id9cc59e345b9283844044ef94ceb5702f0ca0526
$ PYTHONPATH=$PYTHONPATH:system/update_engine/scripts \
./build/make/tools/releasetools/check_ota_package_signature.py \
build/target/product/security/testkey.x509.pem \
out/dist/aosp_marlin-ota-eng.zip
Package: out/dist/aosp_marlin-ota-eng.zip
Certificate: build/target/product/security/testkey.x509.pem
...
Whole package signature VERIFIED
Verifying A/B OTA payload signatures...
...
Payload signatures VERIFIED
Bug: 65261072
Test: Signed a package and its payload with the right keys; ran the
command above.
Test: Signed the payload with a different key; ran the command above and
observed the reported verification failure.
Change-Id: If626ecb327a9826cd0956eef94914c939068a7d1
Commit 21803d3574 only added compatibility
metadata for A/B OTA packages, because by that time we didn't use
'ro.treble.enabled' property to determine if a target was Treble-enabled.
This CL uses 'ro.treble.enabled' to guard the packing of compatibility
metadata for both of A/B and non-A/B OTA packages. It also switches to
checking the system/vendor fingerprints to determine if there's an
update to the partition (previously it was computing the SHA-1 of the
images, which may have unintentionally changed due to issues that give
non-repetitive builds).
Bug: 64339310
Test: Generate OTA packages (full and incremental) on Treble-enabled
targets (sailfish, as well as a non-A/B angler target with
Treble-enabled locally); check that the compatibility.zip entry
exists.
Test: Generate OTA packages on Treble-unenabled non-A/B target (angler);
check that the compatibility.zip entry doesn't exist.
Test: Generate OTA packages on Treble-unenabled A/B target; check that
the compatibility.zip entry doesn't exist.
Change-Id: I2a1fcf612439d849ba8ccea217a0faf5d5ba8e14
This entry only exists in non-A/B full OTA path. However, this file has
never been loaded by recovery - recovery was only reading the one from
the recovery image (which also has been removed by a recent change in
[1]).
[1]: commit e35926e1aff2e6b9b54656bd59c8178e295a1b7e in
platform/bootable/recovery.
Bug: 63538434
Test: `m dist` on angler. file_contexts.bin is gone from full OTA
package. Sideloading the generated full OTA on angler.
Change-Id: Iec2e2d36599fa1ed499516a3e74f06cd36ade494
Compressed APKs can be identified by a "compressed=<ext>" entry in
the apkcerts.txt file. When we encounter such an entry, we need to
decompress the file to a temporary location before we process its
certs. When we're signing, we should also recompress the package
after it's signed.
Bug: 64531948
Test: ./build/tools/releasetools/check_target_files_signatures.py
Test: ./build/tools/releasetools/sign_target_files_apks.py
Test: compared signed output before / after this change, verify that
it's bitwise identical when no compressed APKs are present.
Change-Id: Id32e52f9c11023955330c113117daaf6b73bd8c2
In BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS, if we have defined
"--include_descriptors_from_image" with an image file whose path points
to source tree, add_img_to_target_files.py or sign_target_files_apks.py
may fail to find the file. Because these scripts may run without a
source tree, by taking target_files.zip as the only input.
This CL scans additional locations in the input target_files.zip to find
those missing files in avb_vbmeta_args. As long as the files are included
in the target_files.zip, they get a second chance to be found.
Bug: 63910867
Test: As follows:
1. Setup BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS with a local file path;
2. Remove the local file;
3. sign_target_files_apks.py fails without this CL;
4. sign_target_files_apks.py works.
Change-Id: I3c58f80a5535db02b74cfe40d0c0beff72587cf8
(cherry picked from commit 1dc5d47653)
In BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS, if we have defined
"--include_descriptors_from_image" with an image file whose path points
to source tree, add_img_to_target_files.py or sign_target_files_apks.py
may fail to find the file. Because these scripts may run without a
source tree, by taking target_files.zip as the only input.
This CL scans additional locations in the input target_files.zip to find
those missing files in avb_vbmeta_args. As long as the files are included
in the target_files.zip, they get a second chance to be found.
Bug: 63910867
Test: As follows:
1. Setup BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS with a local file path;
2. Remove the local file;
3. sign_target_files_apks.py fails without this CL;
4. sign_target_files_apks.py works.
Change-Id: I3c58f80a5535db02b74cfe40d0c0beff72587cf8
Brotli has a better compression ratio than deflation. So for non-AB
full OTA, we can compress the new.dat with brotli first and store it
in the zip package.
This increase the OTA generation time by ~5 minutes for a full OTA
(measured based on 2.0G system.new.dat)
Bug: 34220646
Test: Generate a full OTA for bullhead
Change-Id: I9c0550af8eafcfa68711f74290c8e2d05a96648f
Accidentally broken by the cherry-pick in commit
f829b40c48 - the original CL in oc-dev
doesn't require the 'import stat' line.
Bug: 63629728
Test: `pylint --rcfile=pylintrc sign_target_files_apks.py`
Test: Successfully sign a build with sign_target_files_apks.py.
Change-Id: I94be613fb2219597148c4339ac33fc93d0cdae47
system_avb_hashtree_enable has been renamed to
avb_system_hashtree_enable in commit
3e599ead66. 'care_map.txt' is missing due
to the change.
Bug: 63142730
Test: `m dist` and check IMAGES/care_map.txt exists in target_files zip.
Change-Id: I60c269b41df844b50353d357bf67c20f15548642
(cherry picked from commit 3f72176da0)
system_avb_hashtree_enable has been renamed to
avb_system_hashtree_enable in commit
3e599ead66. 'care_map.txt' is missing due
to the change.
Bug: 63142730
Test: `m dist` and check IMAGES/care_map.txt exists in target_files zip.
Change-Id: I60c269b41df844b50353d357bf67c20f15548642
Currently we're writing META/misc_info.txt to the new TF.zip during
ReplaceVerityPrivateKey(). We should delay that until we have replaced
everything in need. Otherwise we won't be able to replace/overwrite
that zip entry (unless `zip -d` first).
This CL also cleans up the return value of ReplaceVerityPublicKey() and
ReplaceVerityKeyId(), since the caller no longer needs the values.
Test: sign_target_files_apks.py and check the generated signed TF.zip.
Change-Id: I9fbd7182247728281519e5e3971557f6b018ad65
(cherry picked from commit 46a5999a02)
Radio images are added to INSTALLED_RADIOIMAGE_TARGET, which by default
will be packed under RADIO/ in a target_files zip. This CL introduces
BOARD_PACK_RADIOIMAGES that allows additionally copying them into
IMAGES/, which will then be included into <product>-img.zip.
Bug: 62195105
Test: Define BOARD_PACK_RADIOIMAGES and `m dist`. Check the generated
target_files zip and <product>-img.zip.
Change-Id: I3deafd2dfecd1d4dbfdfc2d002fc40ef22fb60ea
This patch tries to fix the problem where the default properties need
to go with the system image especially on non-AB devices where
/default.prop is on the ramdisk image. A symlink is created at
/default.prop for backward compatibility.
Bug: 37815285
Test: Tested with ag/2416542. Booted pixel phones, checked the location
of prop.default, verified the symlink, checked a few properties
(via adb shell getprop) and manually tested a few apps (Camera,
Maps etc).
sign_target_files_apks.py was tested with:
sign_target_files_apks -o -e DynamiteLoader.apk= -e DynamiteModulesA.apk= \
-e DynamiteModulesB.apk= -e DynamiteModulesC.apk= -e DynamiteModulesD.apk= \
-e GoogleCertificates.apk= out/dist/*-target_files-*.zip signed-target_files.zip
Booted to recovery and ran 'adb sideload' successfully.
Change-Id: I1a9a2ba49c8252afc13ced3dea71253afbd3091e
Merged-In: I1a9a2ba49c8252afc13ced3dea71253afbd3091e
(cherry-picked from 4fbbe4578bb10d54292d9b243edf4999fddf1c93)
This patch tries to fix the problem where the default properties need
to go with the system image especially on non-AB devices where
/default.prop is on the ramdisk image. A symlink is created at
/default.prop for backward compatibility.
Bug: 37815285
Test: Tested with ag/2416542. Booted pixel phones, checked the location
of prop.default, verified the symlink, checked a few properties
(via adb shell getprop) and manually tested a few apps (Camera,
Maps etc).
sign_target_files_apks.py was tested with:
sign_target_files_apks -o -e DynamiteLoader.apk= -e DynamiteModulesA.apk= \
-e DynamiteModulesB.apk= -e DynamiteModulesC.apk= -e DynamiteModulesD.apk= \
-e GoogleCertificates.apk= out/dist/*-target_files-*.zip signed-target_files.zip
Booted to recovery and ran 'adb sideload' successfully.
Change-Id: I1a9a2ba49c8252afc13ced3dea71253afbd3091e
* Owners are selected from top CL approvals or owners.
They will be suggested to review/approve future CLs.
* OWNERS files are recognized by the new find-owners plugin, see .md files in
https://gerrit.googlesource.com/plugins/find-owners/+/master/src/main/resources/Documentation/
Test: build/make/tools/checkowners.py -c -v OWNERS
Change-Id: Ibe6765e5448c471ca413413e3d052507ccee4ea6
Current build system will include AVB metadata from each partition and
store them into /vbmeta partiton when BOARD_AVB_ENABLE is set, which makes
each partition tightly-coupled.
Add the support for 'chain partition':
- The vbmeta of each partition is stored on the same partition itself.
- The public key used to verify each partition is stored in /vbmeta.
For example, the following build variables are required to enable chain
partition for system partition:
- BOARD_AVB_SYSTEM_KEY_PATH := path/to/system_private_key
- BOARD_AVB_SYSTEM_ALGORITHM := SHA512_RSA8192
- BOARD_AVB_SYSTEM_ROLLBACK_INDEX := 1
- BOARD_AVB_SYSTEM_ROLLBACK_INDEX_LOCATION := 2
The corresponding settings will be added into META/misc_info.txt for
build_image.py and/or add_img_to_target_files.py:
- avb_system_key_path=path/to/system_private_key
- avb_system_algorithm=SHA512_RSA8192
- avb_system_add_hashtree_footer_args=--rollback_index 1
- avb_system_rollback_index_location=2
To enable chain partition for other partitions, just replace SYSTEM with
BOOT, VENDOR and/or DTBO in the build variables.
Also switch from `avbtool make_vbmeta_image --setup_rootfs_from_kernel system.img ...`
to `avbtool add_hashtree_footer --image system.img --setup_as_rootfs_from_kernel...`
when BOARD_BUILD_SYSTEM_ROOT_IMAGE is true. This works for both chained
and no-chained:
- chained: `avbtool add_hashtree_footer --setup_as_rootfs_from_kernel` will
add dm-verity kernel cmdline descriptor to system.img
- no-chained: `avbtool make_vbmeta_image --include_descriptors_from_image
system.img` will include the kernel cmdline descriptor from system.img into
vbmeta.img
Bug: 38399657
Test: `make` pass, flash images from $OUT and boot device without chain partitions
Test: `make` pass, flash images from $OUT and boot device with chain partitions
Test: `make dist` pass, flash images from TF.zip and boot device without chain partitions
Test: `make dist` pass, flash images from TF.zip and boot device with chain partitions
Test: follow the same steps in
https://android-review.googlesource.com/#/c/407572/
Change-Id: I344f79290743d7d47b5e7441b3a21df812a69099
Merged-In: I344f79290743d7d47b5e7441b3a21df812a69099
(cherry picked from commit 3e599ead66)
Check the SHA1 of recovery.img and boot.img embedded in the
install-recovery.sh.
Bug: 35411009
Test: validation script detects mismatch for both full recovery and
recovery-from-boot.
Change-Id: I5f07a869d9fa17fad26a22ef9ca3ecb06b1b28e3
The script has been moved to /system/bin since L.
Bug: 35411009
Test: Generate a exact same OTA for bullhead.
Change-Id: Iaecb86e3dabc5b659605837bb06f16d20fa80334
We should only disallow zip64 for the image and OTA zips (because we
don't have zip64 support in libziparchive yet). But target_files zips
are fine to use zip64 with host tools (and we already do that in
add_img_to_target_files.py).
This CL also sets the default compression method to DEFLATED when
creating the signed TF.zip.
Test: sign_target_files.apks.py signing a large TF.zip passes.
Change-Id: I8043739860604134fa1166e920c95c28797bbcc1
In case the system/vendor image is updated for an A/B target file, we'll
delay the write of META/care_map.txt. Specifically, we'll run "zip -d"
to remove the old entry and then zip in the new one from tmp dir.
Bug: 62345693
Test: Run add_img_to_target_files.py on sailfish target files, and
care_map.txt is updated.
Change-Id: Id2f9997aac50c0740e6944d0d0d43d98f5fc6c29
Current build system will include AVB metadata from each partition and
store them into /vbmeta partiton when BOARD_AVB_ENABLE is set, which makes
each partition tightly-coupled.
Add the support for 'chain partition':
- The vbmeta of each partition is stored on the same partition itself.
- The public key used to verify each partition is stored in /vbmeta.
For example, the following build variables are required to enable chain
partition for system partition:
- BOARD_AVB_SYSTEM_KEY_PATH := path/to/system_private_key
- BOARD_AVB_SYSTEM_ALGORITHM := SHA512_RSA8192
- BOARD_AVB_SYSTEM_ROLLBACK_INDEX := 1
- BOARD_AVB_SYSTEM_ROLLBACK_INDEX_LOCATION := 2
The corresponding settings will be added into META/misc_info.txt for
build_image.py and/or add_img_to_target_files.py:
- avb_system_key_path=path/to/system_private_key
- avb_system_algorithm=SHA512_RSA8192
- avb_system_add_hashtree_footer_args=--rollback_index 1
- avb_system_rollback_index_location=2
To enable chain partition for other partitions, just replace SYSTEM with
BOOT, VENDOR and/or DTBO in the build variables.
Also switch from `avbtool make_vbmeta_image --setup_rootfs_from_kernel system.img ...`
to `avbtool add_hashtree_footer --image system.img --setup_as_rootfs_from_kernel...`
when BOARD_BUILD_SYSTEM_ROOT_IMAGE is true. This works for both chained
and no-chained:
- chained: `avbtool add_hashtree_footer --setup_as_rootfs_from_kernel` will
add dm-verity kernel cmdline descriptor to system.img
- no-chained: `avbtool make_vbmeta_image --include_descriptors_from_image
system.img` will include the kernel cmdline descriptor from system.img into
vbmeta.img
Bug: 38399657
Test: `make` pass, flash images from $OUT and boot device without chain partitions
Test: `make` pass, flash images from $OUT and boot device with chain partitions
Test: `make dist` pass, flash images from TF.zip and boot device without chain partitions
Test: `make dist` pass, flash images from TF.zip and boot device with chain partitions
Test: follow the same steps in
https://android-review.googlesource.com/#/c/407572/
Change-Id: I344f79290743d7d47b5e7441b3a21df812a69099
Currently we're writing META/misc_info.txt to the new TF.zip during
ReplaceVerityPrivateKey(). We should delay that until we have replaced
everything in need. Otherwise we won't be able to replace/overwrite
that zip entry (unless `zip -d` first).
This CL also cleans up the return value of ReplaceVerityPublicKey() and
ReplaceVerityKeyId(), since the caller no longer needs the values.
Test: sign_target_files_apks.py and check the generated signed TF.zip.
Change-Id: I9fbd7182247728281519e5e3971557f6b018ad65
Bug: 62252466
Test: Re-generate an incremental OTA that goes from the fingerprint to
thumbprint, and check the updater-script.
Change-Id: I6e2cbf68cbd22fbcf0d200fc3fdc8a33da510a53
board_avb_algorithm and board_avb_key_path are overlapping with
avb_signing_args. In core/Makefile, only avb_signing_args (i.e.
INTERNAL_AVB_SIGNING_ARGS) will be used in the AVB-signing command. It
covers the contents in board_avb_{algorithm,key_path}. We should do the
same thing in tools/releasetools to avoid potential inconsistency.
This CL cleans up the logic in tools/releasetools, by always using
avb_signing_args. This also allows easier signing key replacement (so we
can replace the key/algorithm/signer in 'avb_signing_args').
board_avb_system_add_hashtree_footer_args is unused in releasetools
script, and the same information has been covered by
system_avb_add_hashtree_footer_args. This CL removes this arg as well.
Test: `m dist`. Then a) check the removed three args no longer exist in
META/misc_info.txt; b) check that rebuilding images with
add_img_to_target_files.py uses the same parameters.
Change-Id: I7db890b5c942de5b6868d8d1ebf937586d4729c0
When AVB is enabled, generate care_map.txt and add it to the target
files. Also copy it into the OTA package where it will later be used
by the update_verifier.
Bug: 62208947
Test: \
1. Run add_img_to_target_files on the TF of a new pixel device,
and care_map.txt generates successfully.
2. Make dist in oc-dr1-release and find care_map.txt in the OTA package.
3. update_verifier succeeds in reading all the blocks on the care_map,
and fails to read out-of-bound blocks.
Change-Id: I2881711e6f87789cb7de150dbeca18b756fed68a
This is a step to enable signing a given target_files zip with release
keys.
When calling sign_target_files_apks.py, we will delete all the entries
under IMAGES/ in order to re-generate them (with the proper release
keys). In order to support that, we need to pack everything in need into
TF.zip.
Steps to test the CL.
a) Choose a target that has both AVB and DTBO enabled.
$ m dist
b) Check IMAGES/dtbo.img and PREBUILT_IMAGES/dtbo.img both exist in the
generated out/dist/TF.zip.
c) Remove the entries under IMAGES/ from the generated TF.zip.
$ zip -d TF.zip IMAGES/\*
d) Re-generate the images with TF.zip.
$ build/make/tools/releasetools/add_img_to_target_files.py TF.zip
e) Check that IMAGES/dtbo.img is re-generated, and it's identical to the
image in b). Note that by default the re-generated image will carry a
different footer, because of the random salt. This CL is verified by
specifying the same salt.
Bug: 38315721
Test: see above.
Change-Id: I0bdc4e1cd4800962dc3902ca550dad6a8ca56c78
img_from_target_files.py used to handle the case that a given TF.zip not
containing the image entries under IMAGES/. That is only the case for
pre-Lollipop releases.
Also unzip the needed files only since we know that for sure now.
Test: img_from_target_files.py with an existing bullhead-TF.zip gives
the same bullhead-img.zip.
Change-Id: I892379ba388df80ae63be9d3ce647fbb77fd4753
Also pack the test keys for easier testing.
Bug: 38315721
Test: m otatools-package and avbtool is present in otatools.zip.
Change-Id: Ieb63bf3f4bc211ef1f48ab278cb01b70845d06da
Currently we're building the boot/recovery image twice, which is
redundant. And b/38455129 shows a problematic case when the image
from two builds doesn't match. We should only build the recovery
image once in the add_img_to_target_files.
Bug: 62021378
Test: call sign_target_files_apk on an angler target file,
recovery-from-boot.p generates successfully; and SHA of recovery.img
matches the one in install-recovery.sh.
Change-Id: I01e033501d80c18a87cbb870300eee5c19a04441
If we pass "rebuild_recovery" to add_img_to_target_files, the recovery
patch is rebuilt. But related files under SYSTEM/ (e.g.
SYSTEM/recovery-from-boot.p && SYSTEM/bin/install-recovery.sh) are not
updated.
This may cause a mismatch between system.img and SYSTEM/, and
may lead to a failure in validate_target_files.py.
Bug: 62096364
Test: Rebuild the system image in the TF and observe the recovery files
under SYSTEM/ get updated.
Change-Id: I7d679a612a86d02cf2eff81d1d120c0067138ed9
`make custom_images` supports to build different kinds of *non-droid* images,
e.g., odm.img. Adding the support of signing them with either AVB HASH footer
or AVB HASHTREE footer. The user can use HASH for small images and
HASHTREE for large images.
Sample signing configurations:
* AVB HASH footer:
- CUSTOM_IMAGE_AVB_HASH_ENABLE := true
- CUSTOM_IMAGE_AVB_ADD_HASH_FOOTER_ARGS := --append_to_release_string my_odm_image
* AVB HASHTREE footer:
- CUSTOM_IMAGE_AVB_HASHTREE_ENABLE := true
- CUSTOM_IMAGE_AVB_ADD_HASHTREE_FOOTER_ARGS := --fec_num_roots 8
* Using custom signing key:
- CUSTOM_IMAGE_AVB_ALGORITHM := SHA256_RSA2048
- CUSTOM_IMAGE_AVB_KEY_PATH := external/avb/test/data/testkey_rsa2048.pem
Bug: 36701014
Test: `make custom_images` with AVB HASH footer
Test: `make custom_images` with AVB HASHTREE footer
Test: `make droid` to check system.img is still properly signed with AVB HASHTREE
Test: `make droid` to check vendor.img is still properly signed with AVB HASHTREE
Change-Id: I8dc420e12e37e9a631345c0cd883339db05d489f
package_extract_dir is used in file based OTA only and should be killed.
Bug: 37959785
Test: code search shows no usage of this function in aosp.
Change-Id: Id3719b969c24b7ecef0c7f0e4a3af09a72be54d4
We used to check for 'attr >> 16 == 0xa1ff' (i.e. 0o120777) to detect
symlinks in the input target_files zip (TF.zip). This becomes broken
after we switch to soong_zip, which packs symlinks with 0o120700.
This CL fixes the issue by using stat.S_ISLNK() instead.
Note that we don't need to stage the files with the exact permission
bits as in the input TF.zip. Because this part is covered by mkbootfs
by using the canned or the compiled-in fs_config - as long as the
files/directories are accessible and the symlinks are created.
Bug: 38455129
Test: sign_target_files_apks.py on bullhead TF.zip. Check the
checksums in SYSTEM/bin/install-recovery.sh.
Change-Id: I51c1fc9a257fb3f18c16c2ed71528abaa6f7d9c9
(cherry picked from commit 406050bdb6)
We used to check for 'attr >> 16 == 0xa1ff' (i.e. 0o120777) to detect
symlinks in the input target_files zip (TF.zip). This becomes broken
after we switch to soong_zip, which packs symlinks with 0o120700.
This CL fixes the issue by using stat.S_ISLNK() instead.
Note that we don't need to stage the files with the exact permission
bits as in the input TF.zip. Because this part is covered by mkbootfs
by using the canned or the compiled-in fs_config - as long as the
files/directories are accessible and the symlinks are created.
Bug: 38455129
Test: sign_target_files_apks.py on bullhead TF.zip. Check the
checksums in SYSTEM/bin/install-recovery.sh.
Change-Id: I51c1fc9a257fb3f18c16c2ed71528abaa6f7d9c9
Clear fs_config_generate_extra_partition_list after use.
Test: build fs_config_files and fs_config_dirs targets.
Change-Id: Ie18bcd2df2ff1becb178e9ba45d226e2e9a8faab
Signed-off-by: William Roberts <william.c.roberts@intel.com>
* Add logic to handle decrypted keys from common.GetKeyPassword in
WriteABOTAPackageWithBrilloScript.
* Get the keys passwords in main and store them in OPTIONS.key_passwords.
This allows accessing them in WriteABOTAPackageWithBrilloScript and SignOutput
so it's only required to ask for the password once, while allowing to use
decrypted signing keys.
Test: ota_from_target_files.py on marlin and angler respectively.
Change-Id: I7c9b0198855a4b630c52b8552e904f312f09c4ce
The script prints the meta info, new/patch data size. It also simulates
the transfer commands and calculates amount of I/O as well as cache
needed.
We can add the more options to parse system/vendor.map so that we can
analyze the location change of files.
Bug: 31514709
Test: parse a bullhead incremental/full ota.
Change-Id: I70ed4f4c15e595b1851109e9799d44ac4e815c16
This is useful for devices with low disk space with different
build variants.
Bug: 37469715
Test: Regular image builds successfully, errors occur when
the headroom size is greater than available partition space.
Change-Id: I526cdd0f84981bbd16e3afcfe1cd7fc43dce98ef
Caller can optionally specify the verbose flag which overrides
OPTIONS.verbose. The command line won't be outputed with verbose=False.
This is useful for cases that a) those command lines are less useful
(but will spam the output otherwise); b) sensitive info is part of the
invocation.
'verbose=False' will be consumed by common.Run() only, instead of being
passed to subprocess.Popen().
Test: ota_from_target_files.py on a block based OTA.
Change-Id: I7d5b4094d756a60f84f89c6a965e7ccc68e435f8
Xml files will be generated instead of html files, and then Settings
will generate html file on runtime from xml files of partitions.
Test: build succeeded and tested on sailfish
Bug: 37099941
Merged-In: Id7899381a537b1947cbb6164ed2b4a98492334af
Change-Id: Id7899381a537b1947cbb6164ed2b4a98492334af
(cherry picked from commit 3b7560bf88)
Xml files will be generated instead of html files, and then Settings
will generate html file on runtime from xml files of partitions.
Test: build succeeded and tested on sailfish
Bug: 37099941
Change-Id: Id7899381a537b1947cbb6164ed2b4a98492334af
find_android_root works only when warn.py is found
so it works for build bot, but not when warn.py is
run against a stand alone build.log file.
Now get the TOP directory path from build.log
if it is found.
Bug: 37725427
Test: warn.py --gencsv build.log
Change-Id: I7b776c2e71a8d8d769f0600b08abe37acbd6ce7c
Misspelled fs_config_generate_extra_partition_list when used to
calculate LOCAL_REQUIRED_MODULES dependencies. This resulted in
fs_config_dirs and fs_config_files failing to generate the vendor,
oem or odm variants (ie: fs_config_files_vendor)
Test: build generates correct fs_config override files.
Bug: 37721445
Bug: 36071012
Change-Id: I053dd5cdb357c7cea05a5ff63b0686bd0bc99df6
This reverts commit b1e59b8a73.
This CL enables generating compatibility zip entry in an A/B OTA
package. We're not enabling the support for non-A/B OTAs for now until
they have the necessary support (e.g. /proc/config.gz exists).
Bug: 36810913
Test: Generate full and incremental packages for marlin and applly under
normal boot and sideload respectively.
Change-Id: I69f0a3feb7a0efc7f727f92f31c9e63f47ee6530
(cherry picked from commit b63c952cc7)
Add build targets to generate the vendor, oem and odm slices of the
fs_config_dirs and fs_config_files targets for each of the respective
filesystems, and automatically trigger the individual and enabled
partition build objects based as dependents of fs_config_dirs and
fs_config_files.
Sensitive to the following board definition pairs:
BOARD_VENDORIMAGE_FILE_SYSTEM_TYPE BOARD_USES_VENDORIMAGE
BOARD_OEMIMAGE_FILE_SYSTEM_TYPE BOARD_USES_OEMIMAGE
BOARD_ODMIMAGE_FILE_SYSTEM_TYPE BOARD_USES_ODMIMAGE
Which when defined and set means the images are generated for vendor,
oem or odm respectively.
Test: manually build w/ m 'fs_config_(dirs|files)_(vendor|oem|odm)'
and check the expected products, or fails if not pertinent.
Test: build for product that produces split to system and vendor and
subsequently boots with vendor products correctly configured.
Bug: 36071012
Change-Id: I44289a3dd50b7657367c826ad813c671a3de6afc
Add a -P option to fs_config_generate that allows us to filter on
a comma separated partition whitelist or minus prefixed -blacklist.
A partition is defined as accessible as <partition>/ or
system/<partition>/.
No change for legacy, where no -P flag is specified.
For system.img, but not vendor, oem or odm file references:
fs_config_generate -P -vendor,-oem,-odm
For vendor.img file references:
fs_config_generate -P vendor
For oem.img file references:
fs_config_generate -P oem
For odm.img file references:
fs_config_generate -P odm
Add and fortify tests for each of these cases.
SideEffects: none
Test: gTest host fs_config-unit-test
Bug: 36071012
Change-Id: I0537453bb289d41ef98ad76e69f78ae49358a1b3
This reverts commit b1e59b8a73.
This CL enables generating compatibility zip entry in an A/B OTA
package. We're not enabling the support for non-A/B OTAs for now until
they have the necessary support (e.g. /proc/config.gz exists).
Bug: 36810913
Test: Generate full and incremental packages for marlin and applly under
normal boot and sideload respectively.
Change-Id: I69f0a3feb7a0efc7f727f92f31c9e63f47ee6530
Bug: None
Test: Ran warn.py on a build log. Code links in resultant HTML open in
new tabs when clicked.
Change-Id: Ia3def5ea14fe216700cb2ffabe4a39b98e69d55a
This reverts commit 853cd306b0.
Temporarily reverting the CL to work around the bug in recovery code.
Bug: 37413730
Change-Id: I8e77112ffafcf32332794739ee1c855cc83b32b0
Uses avbtool to sign vendor.img if BOARD_AVB_ENABLE is set.
It also allows appending additional arguments to avbtool via
BOARD_AVB_VENDOR_ADD_HASHTREE_FOOTER_ARGS.
e.g.,
BOARD_AVB_ENABLE := true
BOARD_AVB_VENDOR_ADD_HASHTREE_FOOTER_ARGS := --generate_fec
Bug: 35415839
Test: "make" with the above variables and use avbtool to check vbmeta is
appended to vendor.img
Test: "make dist" with the above variables
Change-Id: I8ada38dff3def6d34613e77c67944def8a49f464
Merged-In: I8ada38dff3def6d34613e77c67944def8a49f464
(cherry picked from commit 8ee4a3db8c)
Tao Bao