Imgdiff expects the input files to be valid zip archives. During the
split of large apks, imgdiff may fail when there's a hole in the input
file; potentially due to the blocks allocation of mke2fs. This CL solves
the issue by falling back to normal split in case of the imgdiff split
failure. The split transfers will then use bsdiff instead.
Bug: 69624507
Test: generate the incremental package for the failed targets and check the
transfers.
Change-Id: I4882452378123e60ad3434053b57f33e53ac4b82
... to include system/update_engine/scripts if ANDROID_BUILD_TOP is set
(e.g. after lunching a target).
Test: ./build/make/tools/releasetools/check_ota_package_signature.py
works without manually setting PYTHONPATH.
Change-Id: I0c5101fab2dd69cb6a598b909aa04bc4d5f45284
Bug: 68224784
Test: Enable AVB on aosp_bullhead-userdebug. `m dist`. Note that the
prebuilt vendor.img needs to be AVB-signed first.
Test: `m dist` with aosp_walleye-userdebug.
Change-Id: I18235e4d0dde6af71a96e46ec434480cc9a22bef
AddImagesToTargetFiles() takes either a zip file, or a zip root as
input. We used to create IMAGES/ directory only when working with zip
root input. Commit 262bf3f0b5 has changed
to also stage boot / recovery images there when working with a zip file.
This CL makes sure the directory is always available under both modes.
Bug: 63456822
Test: zip -d target_files.zip IMAGES/\*;
add_img_to_target_files.py target_files.zip
Test: sign_target_files_apks.py target_files.zip signed-target_files.zip
Change-Id: Iea91d0403cdec1b16bb93bb71d3ed06856b8f7c3
Most logtags files only have entries with hardcoded numbers.
Allow these files to be converted to java without needing the
merged logtags file.
Test: m checkbuild
Change-Id: I4cd38aa502fc9b8e50af92ff8296d3ad6abfb56e
If only vbmeta.img is missing from the target_files.zip,
'add_img_to_target_files.py -a' will fail to re-create vbmeta.img.
++++ vbmeta ++++
Traceback (most recent call last):
File "./build/make/tools/releasetools/add_img_to_target_files.py", line 693, in <module>
main(sys.argv[1:])
File "./build/make/tools/releasetools/add_img_to_target_files.py", line 687, in main
AddImagesToTargetFiles(args[0])
File "./build/make/tools/releasetools/add_img_to_target_files.py", line 584, in AddImagesToTargetFiles
boot_contents = boot_image.WriteToTemp()
AttributeError: 'NoneType' object has no attribute 'WriteToTemp'
Because it has skipped loading the boot.img, which leads to the above error.
Bug: 63456822
Test: As follows:
$ zip -d target_files.zip vbmeta.img
$ add_img_to_target_files.py -a target_files.zip
Test: `m dist` on aosp_arm64-userdebug
Change-Id: Ieba5f0b6848c3ec5d8bcfd24d48cc5af8b39f06c
When generating full OTAs for AVB-enabled targets, it fails the size
assertion for boot.img.
ERROR: boot size (31457280) is 100.00% of limit (31457280)
The size assumption is no longer true for AVB-signed images, because
the actual image size should be always identical to the limit.
Bug: 67841177
Test: Enable AVB on bullhead. `m dist`.
Change-Id: I9bd432ccd88094628eb71308554839549eccb6d9
With the new implementation of handling large apks, we need to call
imgdiff with block-limit to split the apk and generate the patch at
the same time. The call to imgdiff would significantly increase the
time consumption of the "FindTransfers" function which we used to
execute sequentially. This cl addresses this issue and speeds up the
process by making the imgdiff call parallel.
Bug: 34220646
Test: Create and sideload an incremental package for angler
Change-Id: Id62e348418fc1d22e32ea6c8ac16d9ab3ec92d7b
With the update of brotli to version 1.0.1 the new binary tool is now
called "brotli". This patch updates the scripts to use the new name.
Bug: 34220646
Test: `make dist`; ota_from_target_files on non-A/B device.
Change-Id: Ie8da4333388d029d5960a723a55efd875d8ee820
When signing a target_files.zip, the OTA certificate specified by
default_system_dev_certificate could be replaced with a mapped key. When
that happens, we must explicitly specify --package_key when generating
OTA packages with ota_from_target_files.py. Otherwise the OTA package
will be signed with the wrong key, which leads to verification failures.
This CL updates the default_system_dev_certificate value in
misc_info.txt accordingly.
Test: Sign a target_files.zip and replace the OTA key. Check
META/misc_info.txt in the generated target_files.zip.
$ ./build/make/tools/releasetools/sign_target_files_apks.py -v \
--replace_ota_keys \
-k build/target/product/security/testkey=build/target/product/security/platform \
out/dist/aosp_marlin-target_files-eng.tbao.zip \
signed-marlin-target_files-test.zip
Change-Id: I093234b5add3e27c5b3887cefeffd74e6f0a3e98
test_common constructs a few 2GiB strings in memory, which leads to huge
memory footprint (18GiB). This CL moves away from in-memory strings to
generators, which reduces the memory use down to 41MiB. It also reduces
the time cost from 294s to 139s as an extra benefit for free.
The CL addresses some trivial pylint warnings as well.
* Before
$ /usr/bin/time -v python -m unittest -v test_common
...
----------------------------------------------------------------------
Ran 11 tests in 294.986s
OK
Command being timed: "python -m unittest -v test_common"
User time (seconds): 110.51
System time (seconds): 109.34
Percent of CPU this job got: 74%
Elapsed (wall clock) time (h:mm:ss or m:ss): 4:55.06
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 18894172
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 1
Minor (reclaiming a frame) page faults: 20774908
Voluntary context switches: 48
Involuntary context switches: 3241
Swaps: 0
File system inputs: 184
File system outputs: 8406424
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
* After
$ /usr/bin/time -v python -m unittest -v test_common
...
----------------------------------------------------------------------
Ran 11 tests in 139.100s
OK
Command being timed: "python -m unittest -v test_common"
User time (seconds): 59.00
System time (seconds): 4.73
Percent of CPU this job got: 45%
Elapsed (wall clock) time (h:mm:ss or m:ss): 2:19.17
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 41252
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 106569
Voluntary context switches: 44
Involuntary context switches: 103
Swaps: 0
File system inputs: 8
File system outputs: 8422808
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
Fixes: 68988396
Test: See above.
Change-Id: I00f16603a4ee59fb085b189c6f5b5ee9d2378690
imgdiff fails on plain strings after we put more checks the input
data. This leads to failure when running unittest
"test_recovery_from_boot". The CL fixes the issue by serving gzipped
dummy images as the test input.
Bug: 68988329
Test: InstallRecoveryScriptFormatTest passes
Change-Id: I1952b311681ac4f39c0e51aef1de1177aebeac43
$ python rangelib.py
**********************************************************************
File "rangelib.py", line 323, in __main__.RangeSet.next_item
Failed example:
list(rangelib.RangeSet("10-19 3 5 7").next_item())
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.7/doctest.py", line 1315, in __run
compileflags, 1) in test.globs
File "<doctest __main__.RangeSet.next_item[2]>", line 1, in <module>
list(rangelib.RangeSet("10-19 3 5 7").next_item())
NameError: name 'rangelib' is not defined
**********************************************************************
1 items had failures:
1 of 3 in __main__.RangeSet.next_item
***Test Failed*** 1 failures.
Test: python rangelib.py
Change-Id: I6c791ff4453f40d90b34f97b881460da9878fe64
"ro.vendor." prefix will be used only for vendor-specific properties
except for ro.vendor.build.date, ro.vendor.build.date.utc and
ro.vendor.build.fingerprint.
Those exceptions have been used since Android L. So we can't rename them
to support the system-only upgrade from old Android.
Bug: 36796459
Test: confirmed that ro.vendor.product.* were changed to
ro.product.vendor.* in vendor/build.prop
Merged-In: Ibc107b00264a2dc194261d1968b27a9b17b5d798
Change-Id: Ibc107b00264a2dc194261d1968b27a9b17b5d798
(cherry picked from commit 8cf6443264)
It has been long deprecated since commit
fdd8e69c42 (Donut).
Test: `m dist` with aosp_marlin-userdebug and aosp_angler-userdebug.
Test: No user of "--board_config" in code search.
Change-Id: Ic0336c22cac32de5de88161748d09b579f892e99
Commit b937ead5d9 added the fallback to
bsdiff on imgdiff failures. However, it missed setting the transfer
style accordingly, which led to patch header mismatch.
Bug: 68659848
Test: Generate an incremental that has the fallback from imgdiff to
bsdiff. Examine the generated transfer list and verify that it has
"bsdiff" for the fallback command.
Change-Id: I55e46879d590a8af82ea796b9d98ffdb30360408
Following the new API contract, this effectively add extra padding
before central dir to make it 4KB aligned.
Test: build succeeded
Bug: 30972906
Change-Id: I7cac9d2c4371b473c88df867b3b2ae906443db10
Use zip_archive directly. Note that this codepath is used only
when recompressing archives with zopfli during the alignment step.
It's unclear whether this is in use at all, but I verified that the
results are identical (note the usage of the "-z" flag in the test
below).
Test: make && out/host/linux-x86/bin/zipalign -v -f -z 8 \
out/target/product/marlin/system/app/Email/Email.apk ./out.zip
Bug: 35246701
Change-Id: I641cdb6d409cc07974d49d42c9f9e6d4f905e472
Duplicate essential product sysprops for manufacturer, model, brand, name, and device
from system on vendor so that mixed builds (vendor.img + GSI system.img) have correct
product information in their CTS/VTS reports.
Test: make
Bug: 64458205
Merged-In: Ib63b37772be493b9e035d9b7e8c5e2cf66f2fa8c
Change-Id: Ib63b37772be493b9e035d9b7e8c5e2cf66f2fa8c
(cherry picked from commit aea0f92ff3)
When generating block based OTAs, we read files from the sparse image
directly with the help of block map file. However, the block map info
might not be accurate if the image is created with mke2fs. Because
mke2fs may skip allocating actual blocks if they contain all zeros.
ota_from_target_files.py consequently passes incomplete APK files to
imgdiff, which fails to generate patches.
This CL works around the issue by falling back from imgdiff to bsdiff on
failures. We should figure out a better way in b/68016761 to remove the
workaround, which would otherwise hide other issues in imgdiff.
Bug: 67824829
Bug: 68016761
Test: ota_from_target_files.py passes on previously failing TF zips.
Change-Id: Ib24c5b5f89812b97a0c87c6bf0dc147ae39bc92f
Tianjie Xu