In BBOTAs if the last block of a DataImage is padded, we should always
write the whole block even for incremental OTAs. Because otherwise the
last block may be skipped if unchanged, but would fail the post-install
verification if it has non-zero contents in the padding bytes.
Bug: 23828506
Change-Id: I6f6789eec568ed0b808b9e425f5ecdbea1bdf446
(cherry picked from commit 7589e961a7)
For BBOTA v3, we need to stash source blocks to support resumable
feature. However, with the growth of file size and the shrink of the
cache size, source blocks that represent a file are too large to be
stashed as a whole. CL in [1] solves the issue by replacing the diff
command with a "new" command. However, it may increase the generated
package size substantially (e.g. from ~100MB to ~400MB).
With this CL, if a file spans too many blocks, we split it into smaller
pieces by generating multiple commands. For the same case above, it
reduces the package size to ~150MB.
One potential downside is that after splitting, files like .jar,
.apk and .zip can no longer use imgdiff. We may lose the potential
benefit of using imgdiff for patch size reduction.
[1] commit 82c47981bd
Bug: 22430577
Change-Id: Iee1ad6543f3d40368e079e418cc31728e1ab3f48
(cherry picked from commit 9a5caf2c30)
When generating incremental BBOTAs (v2 and above), we need to ensure
that the needed runtime stash is below the given threshold. If it's
running out of space on /cache, we replace the command that uses a
stash with a "new" command instead.
This may increase the OTA package size, since it is carrying more full
blocks instead of patches. It gets even worse for large files that span
a number of blocks, because currently we will store all the blocks for
the file as "new" blocks if stashing cannot be satisfied. We may further
optimize by splitting them into smaller chunks so that most of them can
still be stashed.
Bug: 22430577
Change-Id: Ieae5243d461e3f899f613f76a380f6f7c3edb356
(cherry picked from commit 82c47981bd)
With block-based OTA v2 and v3, it requires stash space on the /cache
partition to back up blocks during an update. We need to ensure that
it doesn't exceed the partition size. Since there might be other files
on /cache as well, we use cache_size * threshold as the maximum allowed
size. The threshold defaults to 0.8, which can be overridden by command
line option '--stash_threshold'.
Change-Id: Ieee5d373c9bfb2ea401d85ca8a3adb491579de76
(cherry picked from commit 23ac404212)
dm-verity may touch some blocks that are not in the care_map due to
block device read-ahead. It will fail if such blocks contain
non-zeroes. As a workaround, we mark them as extended blocks and
zero out explicitly to avoid dm-verity failures.
Bug: 20881595
Change-Id: I54e24e70ad822c0d6d7af43301f74d24505f4461
Similar to the assertations in file-based OTA, we perform verification
for block-based OTAs (BBOTAs) after updating a partition, for both of
the incremental and full OTAs. It increases the update time (~20s on
Nexus 6), but will capture unnoticed errors right away.
Bug: 21500869
Change-Id: I0f8b27734caaa0f41f9c1b904d55af2112784a68
(cherry picked from commit 68658c0f4f)
In ext4 filesystems, some blocks might be changed even being mounted
R/O, such as the superblock (block 0). We need to exclude such blocks
from integrity verification. Plus such blocks should always be
written to the target by copying instead of patching.
Bug: http://b/20939131
Change-Id: I657025b7b1ad50d4365e7b18dc39308facfe864e
(cherry picked from commit ff7778166b)
The updater automatically stashes overlapping source blocks to
make it possible to resume if the command performing the update
is interrupted. However, blockimgdiff.py does not take this into
account and therefore, max_stashed_blocks indicates only the
number of explicitly stashed blocks. Change blockimgdiff.py to
correctly calculate the maximum used stash space, including any
automatically stashed blocks.
Bug: 20309033
Change-Id: Ibf5cb1abc38c54e60a6ad3ce4cbce43556b08ac2
This caught a few bugs/syntax errors (a few character classes were not
escaped properly in regex patterns, some indentation was illegal,
etc).
Change-Id: I50637607524e68c4fb9cad7167f58a46b8d26b2c
The generated OTAs are supposed to be repeatable. The use of dict in
blockimgdiff.py breaks the assumption for block-based OTAs. Fixed by
using OrderedDict instead.
Change-Id: I945bdc879912ba174ca330c152b1f8fc7ed211ac
Add source and target block hashes as parameters to transfer list
commands that copy or patch data to a partition. This allows the
updater to verify the status of each command in the transfer list
and makes resuming block based OTAs possible. Due to the changes,
update the transfer list version to 3.
Needs matching changes from
I1e752464134aeb2d396946348e6041acabe13942
Bug: 18262110
Change-Id: Ia5c56379f570047f10f0aa7373a1025439495c98
(cherry picked from commit cac671a9d1)
Add source and target block hashes as parameters to transfer list
commands that copy or patch data to a partition. This allows the
updater to verify the status of each command in the transfer list
and makes resuming block based OTAs possible. Due to the changes,
update the transfer list version to 3.
Needs matching changes from
I1e752464134aeb2d396946348e6041acabe13942
Bug: 18262110
Change-Id: Ia5c56379f570047f10f0aa7373a1025439495c98
The erase command we insert at the top for full OTAs was getting
inserted in the wrong place for version 2.
Change-Id: I9caf03a40efbdba79f3428f73e50d4319d9ba371
Generate version 2 of the block_image_update transfer list format.
This improves patch size by a different strategy for dealing with
out-of-order transfers. If transfer A must be done before transfer B
due to B overwriting A's source but we want to do B before A, we
resolve the conflict by:
- before B is executed, we save ("stash") the overlapping region (ie
the blocks B will overwrite that A wants to read)
- when A is executed, it will read those parts of source data from
the stash rather than from the image.
This reverses the ordering constraint; with these additions now B
*must* go before A. The implementation of the stash is left up to the
code that executes the transfer list to apply the patch; it could hold
stashed data in RAM or on a scratch disk such as /cache, if available.
The code retains the ability to build a version 1 block image patch;
it's needed for processing older target-files.
Change-Id: Ia9aa0bd45d5dc3ef7c5835e483b1b2ead10135fe
The erase command we insert at the top for full OTAs was getting
inserted in the wrong place for version 2.
Change-Id: I816e1023c3c6d581e14af374c330b8b55337f38f
Generate version 2 of the block_image_update transfer list format.
This improves patch size by a different strategy for dealing with
out-of-order transfers. If transfer A must be done before transfer B
due to B overwriting A's source but we want to do B before A, we
resolve the conflict by:
- before B is executed, we save ("stash") the overlapping region (ie
the blocks B will overwrite that A wants to read)
- when A is executed, it will read those parts of source data from
the stash rather than from the image.
This reverses the ordering constraint; with these additions now B
*must* go before A. The implementation of the stash is left up to the
code that executes the transfer list to apply the patch; it could hold
stashed data in RAM or on a scratch disk such as /cache, if available.
The code retains the ability to build a version 1 block image patch;
it's needed for processing older target-files.
Change-Id: I7259c1fcd41a0aa7767dab63f086951afa2aa657
Move BlockDifference into common and make its script generation code
more complete, so that it can be use by releasetools.py to do diffs on
baseband images.
Bug: 16984795
Change-Id: Iba9afc1c7755458ce47468b5170672612b2cb4b3
Replace the xdelta/xz-based block OTA generation with a new system
based on the existing bsdiff/imgdiff tools.
Bug: 16984795
Change-Id: Ia9732516ffdfc12be86260b2cc4b1dd2d210e886
Tao Bao