In FinalizeMetadata and PropertyFiles, we need to reserve space between
the calls to Compute() and Finalize(). We used to put a 10-byte
placeholder, in the hope of covering the 'offset:length' space for the
metadata entry, as well as the possible value changes in other entries.
However, this could fail in two possible cases: (a) metadata entry
itself has a large offset (e.g. staying near the end of a 1-GiB package,
where the offset itself has 10-digit); or (b) the offsets for other
entries change substantially due to entry reordering. Note that for case
(b), it's space inefficient to always reserve 15-byte for _each_ token
in the property-files.
This CL handles both of these two cases. For (a), we bump up the 10-byte
to 15-byte, which is large enough to cover a package size up to 10-digit
number (i.e. ~9GiB) with a metadata entry size of 4-digit. All these
15-byte will be used for the metadata token alone.
For (b), we add a fallback flow that would retry one more time, but
based on the already signed package that has entries in desired order.
Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate aosp-bullhead full OTA with '--no_signing' flag.
Change-Id: If20487602d2ad09b3797465c01972f2fa792a1f1
(cherry picked from commit 3bf8c65029)
This CL handles a path that uses system-root-image on non-A/B device.
For this path, we can't generate recovery-from-boot patch with imgdiff,
because boot/recovery images contain different number of entries (only
recovery iamge has ramdisk image now).
Using BOARD_USES_FULL_RECOVERY_IMAGE can work around the issue, at the
cost of extra size. Compared to carrying full recovery image, this CL
saves the cost of the kernel size, by putting a patch that's roughly the
size of the recovery ramdisk.
The applypatch executable already detects and handles a bsdiff patch
automatically. No change required to that end.
Note that it won't further reduce the patch size by handling that
ramdisk entry specially, because (a) that's the only difference between
the two images; and (b) there's no corresponding data in boot image to
be diff'd against.
Bug: 72731506
Test: `m dist` with aosp_angler-userdebug. Check the device can install
recovery image successfully (after intentionally corrupting the
recovery image).
Test: Build aosp_angler-userdebug with BOARD_BUILD_SYSTEM_ROOT_IMAGE set.
Verify the generated patch files.
Test: Run validate_target_files.py with the target_files.zips.
Change-Id: I69c06f51ba8c39ae059c5e9a6872a9f10600cf17
Merged-In: I69c06f51ba8c39ae059c5e9a6872a9f10600cf17
(cherry picked from commit 6d5d623987)
This CL exposes ota-property-files flag for non-A/B OTA packages.
Currently the line only contains the info for the METADATA entry, for
example "ota-property-files=metadata:69:286". This allows system updater
to just download the METADATA entry, as opposed to downloading the
entire package, to learn about the info regarding the OTA package (e.g.
post-OTA build fingerprint). Note that this requires the OTA server-side
support to pass down the flag along with the update URL.
Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate a non-A/B package and check the property-files string.
Change-Id: I1482c587e18ea7101c8328777ea988c2d8ca06ac
Merged-In: I1482c587e18ea7101c8328777ea988c2d8ca06ac
(cherry picked from commit c0746f4e94)
AbOtaPropertyFiles writes 'ota-property-files' into package metadata.
Comparing to StreamingPropertyFiles, AbOtaPropertyFiles additionally
writes a token of 'payload-metadata.bin' for a virtual entry that's head
of the 'payload.bin'. This entry can be used by the system updater to
verify if a payload is applicable to the underlying device.
AbOtaPropertyFiles intends to replace StreamingPropertyFiles, as it
covers all the info of the latter. We keep them both in P, and will
remove the latter in later release.
Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate full and incremental A/B OTAs. Check the property-files
in the METADATA entry.
Change-Id: If5872c8ee0a97a8a95b07d4c69b463aa3fe1b9b9
(cherry picked from commit b63046750c)
And move StreamingPropertyFiles as its subclass. We will need similar
PropertyFiles instance for non-A/B OTA as well (to expose the
offset/size for the METADATA entry).
Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate an A/B OTA. Check the generated property-files string.
Test: pylint --rcfile=pylintrc \
ota_from_target_files.py \
test_ota_from_target_files.py
Change-Id: If90d97f0b330749fd8a6cde2ed9d0d6cd6ea60a8
(cherry picked from commit 432f374a1701909ca324f8b047666614684568c0)
This CL breaks down ComputeStreamingMetadata() into mutiple member
functions of StreamingPropertyFiles class, which correspond to the
two-pass logic when generating streaming property files (aka streaming
metadata).
StreamingPropertyFiles.Compute() does the work for the first pass, by
putting placeholders before doing initial signing. Finalize()
corresponds to the second pass, where the placeholders get replaced with
actual data. Verify() can be optionally called to assert the correctness
of the work.
The separation between Compute() and Finalize() is to allow having
multiple StreamingPropertyFiles instances (in coming up CLs). This way
we can call Compute() multiple times for each instance, followed by only
one call to SignOutput(). And similarly for Finalize().
Bug: 74210298
Test: Generate an A/B OTA package. Check the METADATA entry.
Test: python -m unittest test_ota_from_target_files
Change-Id: I45be0372a4863c4405e6d8e20bcb9ccdc29e7e11
(cherry picked from commit ae5e4c30fe)
The file was broken due to earlier touches:
- Missing 'import zipfile';
- Mismatching arguments when calling GetSparseImage().
Bug: 73996151
Test: Run validate_target_files.py with a walleye-target_files.zip.
Test: pylint --rcfile=pylintrc validate_target_files.py
Change-Id: I3692bd51fb27a3da698e06b75155e84502549f66
(cherry picked from commit c63626b4a3)
And add unittests for ComputeStreamingMetadata().
This prepares for the changes that add additional property-files (for
both of A/B and non-A/B).
Bug: 74210298
Bug: 72751683
Test: python -m unittest test_ota_from_target_files
Test: Generate A/B OTA package. Check the ota-streaming-property-files
in the METADATA entry.
Change-Id: Ib4b069f61c2c06c035c0cff73a55112f3936b969
(cherry picked from commit f5110498c0)
There's a mismatch in WriteABOTAPackageWithBrilloScript().
temp_zip_file = tempfile.NamedTemporaryFile()
...
common.ZipClose(temp_zip_file)
It's benign since common.ZipClose() happens to be calling
"temp_zip_file.close()". This CL moves the use of tempfile to
common.MakeTempFile(), so that the tempfile will be cleaned up
automatically as part of the call to common.Cleanup(). (Not fixing the
close() directly, since the nearby lines will be refactored into another
function shortly.)
Also remove one assert in the same function, which trivially holds in
the current code.
Test: Generate an A/B OTA.
Change-Id: I53b375d1150820de718dec0ead55abf5f4951071
Also minor cleanups to make it pylint clean.
Test: Run check_target_files_signatures.py with a target-files.zip.
Test: pylint --rcfile=pylintrc check_target_files_signatures.py
Change-Id: Ife3b54c7805c2f2562e87e91ab4b4de355782012
In addition to the unzipping work, common.UnzipTemp() kindly bundles an
open ZipFile object as part of the return value. It doesn't look very
helpful to the callers though. It also looks less obvious that the
caller needs to properly close the handle (missing the close here is
benign though). This CL just removes the ZipFile object out of the
return value, and leaves the work to callers.
Test: `m dist` on both of A/B and non-A/B target.
Test: python -m unittest test_add_img_to_target_files
Test: python -m unittest test_common
Test: python -m unittest test_ota_from_target_files
Test: Check the callers to common.UnzipTemp() in code search.
Change-Id: Id47da3fd42a0e76d6ae8851f05780db319ee48cf
This should be the last case to be moved over.
Test: Generate an incremental BBOTA (which exercises the changed code).
Test: `rgrep mkdtemp` gives no more instance.
Change-Id: I76db069476201cdfaf3a2de9d9635dfe54507f7a
... from the following functions in add_img_to_target_files.py.
AddSystem
AddSystemOther
AddVendor
AddProduct
AddDtbo
AddUserdata
AddVBMeta
AddPartitionTable
AddCache
The last user of the parameter in img_from_target_files.py has been
removed in commit 2bb109709a (in O).
Test: pylint --rcfile=pylintrc add_img_to_target_files.py
Test: Check all the callers to the above functions.
Test: m dist
Change-Id: I551d1683def8f8535062fc90f68dafa0f4252822
The file has been removed from target-files.zip since commit
c19a8d5590 (Gingerbread), whose info has
been consolidated into META/misc_info.txt.
Test: `m dist`
Change-Id: Ic144457954f5742ea082dcd9ffbea71df4afe46e
With test_utils.construct_sparse_image(), creating valid sparse image is
no longer a blocker for testing these functions.
Test: python -m unittest test_add_img_to_target_files
Change-Id: Iff1f15837cc744bad52e2120f9a9ad94d7db50d5
This CL adds a new flag '--skip_postinstall' that allows skipping all
the postinstall hooks when generating an A/B OTA package (default:
False). Note that this discards ALL the hooks, including non-optional
ones. Should only be used if caller knows it's safe to do so (e.g. all
the postinstall work is to dexopt apps and a data wipe will happen
immediately after).
Bug: 73547992
Test: python -m unittest test_ota_from_target_files
Test: Generate a full OTA package for walleye. Examine the generated
payload.
Change-Id: Ifc069e897b4019605051eabfd221230a6a37867c
All new instrumentation tests shall use AndroidJUnitRunner, thus default
test class to AndroidJUnitTest.
Bug: 73679956
Test: unittest
Change-Id: I52df4d01b8459a4a6e5c2cbe15973dab079fb136
Tao Bao