With the change in [1], it zeroes out extra blocks that might be read
by dm-verity. We should update the expected number of blocks that get
touched.
[1] commit e9b619108d.
Bug: 23786644
Change-Id: I7a7c3531e5af6bf381d3a14832e77abfa9a8efce
For a DataImage, we may have no mapping of ZERO or NONZERO blocks. Don't
write those groups if they are empty.
Bug: 19024504
Change-Id: I1e921effcdbf0661d4f4b541c104dd456ac0c582
In BBOTAs if the last block of a DataImage is padded, we should always
write the whole block even for incremental OTAs. Because otherwise the
last block may be skipped if unchanged, but would fail the post-install
verification if it has non-zero contents in the padding bytes.
Bug: 23828506
Change-Id: I6f6789eec568ed0b808b9e425f5ecdbea1bdf446
(cherry picked from commit 7589e961a7)
In BBOTAs if the last block of a DataImage is padded, we should always
write the whole block even for incremental OTAs. Because otherwise the
last block may be skipped if unchanged, but would fail the post-install
verification if it has non-zero contents in the padding bytes.
Bug: 23828506
Change-Id: I4b0af7344d18261258cd48d18c029c089d6ff365
For file-based OTAs, we used to remove unneeded files in ascending
order, which failed to delete non-empty directories. Reverse the order
to fix the issue.
For example, now we have the following in our generated script:
delete("/system/app/Calculator/arm/Calculator.odex",
"/system/app/Calculator/arm/",
"/system/app/Calculator/Calculator.apk",
"/system/app/Calculator/");
Bug: 22960996
Change-Id: I0d36d29b7862fb53bf55bf5685a990180f9c0b3b
For file-based OTAs, symlinks in the source build but not in the target
build will be deleted. However, if a symlink is replaced by a regular
file in the target build, the file will be accidentally deleted when
applying (resuming) the same package again.
Verify the checksum of a symlink that will be unpacked or renamed to.
Delete the file only if it doesn't have the target checksum.
Bug: 23646151
Change-Id: I77bae035e39f2e0be25f7f6d71c5882464e3d50f
(cherry picked from commit 84006eacd0)
For file-based OTAs, symlinks in the source build but not in the target
build will be deleted. However, if a symlink is replaced by a regular
file in the target build, the file will be accidentally deleted when
applying (resuming) the same package again.
Verify the checksum of a symlink that will be unpacked or renamed to.
Delete the file only if it doesn't have the target checksum.
Bug: 23646151
Change-Id: Ie322abb6022b6fa812c6b36a3127872d9614fc3b
* commit 'e337093e6c5d502f0f4094639b41c001b3048d0f':
Split large files for BBOTA v3.
Revise stash for BBOTAs when needed.
Assert the stash size when generating OTAs.
Changes in [1] break the backward compatibility of the OTA scripts. Fix
the issue by extracting the file basename from the info dict in
META/misc_info.txt.
[1] commit d3a803e668
Bug: 23620049
Change-Id: I40a5625bbe0beb295f34b2328906a4a4283fe59b
RangeSet("2-10").monotonic gives a wrong result of "False". Fix the bug
and add more tests into test_rangelib.py.
Change-Id: I04780571b45ecafd34040f405756b9745a9e21a5
Separate the test cases in rangelib.py comments into a python unittest
file.
To invoke:
$ python -m unittest test_rangelib
Change-Id: I59e4ce0aaccc83bcbb42fee15775683dcd559d99
For BBOTA v3, we need to stash source blocks to support resumable
feature. However, with the growth of file size and the shrink of the
cache size, source blocks that represent a file are too large to be
stashed as a whole. CL in [1] solves the issue by replacing the diff
command with a "new" command. However, it may increase the generated
package size substantially (e.g. from ~100MB to ~400MB).
With this CL, if a file spans too many blocks, we split it into smaller
pieces by generating multiple commands. For the same case above, it
reduces the package size to ~150MB.
One potential downside is that after splitting, files like .jar,
.apk and .zip can no longer use imgdiff. We may lose the potential
benefit of using imgdiff for patch size reduction.
[1] commit 82c47981bd
Bug: 22430577
Change-Id: Iee1ad6543f3d40368e079e418cc31728e1ab3f48
(cherry picked from commit 9a5caf2c30)
When generating incremental BBOTAs (v2 and above), we need to ensure
that the needed runtime stash is below the given threshold. If it's
running out of space on /cache, we replace the command that uses a
stash with a "new" command instead.
This may increase the OTA package size, since it is carrying more full
blocks instead of patches. It gets even worse for large files that span
a number of blocks, because currently we will store all the blocks for
the file as "new" blocks if stashing cannot be satisfied. We may further
optimize by splitting them into smaller chunks so that most of them can
still be stashed.
Bug: 22430577
Change-Id: Ieae5243d461e3f899f613f76a380f6f7c3edb356
(cherry picked from commit 82c47981bd)
With block-based OTA v2 and v3, it requires stash space on the /cache
partition to back up blocks during an update. We need to ensure that
it doesn't exceed the partition size. Since there might be other files
on /cache as well, we use cache_size * threshold as the maximum allowed
size. The threshold defaults to 0.8, which can be overridden by command
line option '--stash_threshold'.
Change-Id: Ieee5d373c9bfb2ea401d85ca8a3adb491579de76
(cherry picked from commit 23ac404212)
For BBOTA v3, we need to stash source blocks to support resumable
feature. However, with the growth of file size and the shrink of the
cache size, source blocks that represent a file are too large to be
stashed as a whole. CL in [1] solves the issue by replacing the diff
command with a "new" command. However, it may increase the generated
package size substantially (e.g. from ~100MB to ~400MB).
With this CL, if a file spans too many blocks, we split it into smaller
pieces by generating multiple commands. For the same case above, it
reduces the package size to ~150MB.
One potential downside is that after splitting, files like .jar,
.apk and .zip can no longer use imgdiff. We may lose the potential
benefit of using imgdiff for patch size reduction.
[1] commit 82c47981bd
Bug: 22430577
Change-Id: I5684fab0ac41c8d94fe18f52a34cef77e06f741c
For squashfs, we currently don't have a system.map. So the whole system
image will be treated as a single file. But for some unknown bug, the
updater will be killed due to OOM when writing back the patched image to
flash (observed on lenok-userdebug MEA49). Prior to getting a real fix,
we evenly divide the non-zero blocks into smaller groups (currently 1024
blocks or 4MB per group).
Bug: 23227672
Change-Id: Ifeddd8d802f01f8cd2a743a1d1217a284fb6e182
When generating incremental BBOTAs (v2 and above), we need to ensure
that the needed runtime stash is below the given threshold. If it's
running out of space on /cache, we replace the command that uses a
stash with a "new" command instead.
This may increase the OTA package size, since it is carrying more full
blocks instead of patches. It gets even worse for large files that span
a number of blocks, because currently we will store all the blocks for
the file as "new" blocks if stashing cannot be satisfied. We may further
optimize by splitting them into smaller chunks so that most of them can
still be stashed.
Bug: 22430577
Change-Id: I5a49e361adc7d3d41de2e9c08ee9b08c1e6c091a
In BBOTA v2, the stash size computation is inaccurate. A stash will be
used only once, so we deduct its size from stashed_blocks right after
its use. However, the stash doesn't actually get freed until being
overwritten by another stash or destroyed at the end of an update. This
leads to OTA failures due to insufficient space on /cache. This CL
changes the BBOTA v2 script to generate an explicit free right after
its use.
Bug: 23119955
Change-Id: Icdd8352bb0ff778945937c870990a888c7a4d38b
In ext4 filesystems, some blocks might be changed even being mounted
R/O, such as the superblock (block 0). We need to exclude such blocks
from integrity verification. Plus such blocks should always be
written to the target by copying instead of patching.
Bug: http://b/20939131
Change-Id: If9efab3d7a0071a87b4d4297daf7963c3b818afe
(cherry picked from commit ff7778166b)
With the change in vboot_signer (CL:744257), an additional parameter
is required to vboot_signer script. This change adds the required
parameter to releasetools as well.
BUG=23076037
Change-Id: Ice5329578b3a9bc459e278a9d404a1981b35ca88
Signed-off-by: Furquan Shaikh <furquan@google.com>
For some old builds, we may not define cache partition size. Change the
exception into a warning to make the script backward compatible.
Change-Id: Ie94c7fbb1a9f3a7db3f16e8d845e493a534aac5b
For system images that contain the root directory, we need to find the
root directory at ROOT/ instead of BOOT/RAMDISK/.
Change-Id: Ica345c8b1b03475f8ac6c44fd576045fcf17c882
Change all uses of the file_contexts file to use the
file_contexts.bin file instead.
Depends on
I75a781100082c23536f70ce3603f7de42408b5ba
I43806d564b83d57f05f5c36c8eba7b1ff4831b04
Id560d093440a2aba99cef28c20133b35feebf950
I15660f4b3e4c5cb8ae0ec1498c74d6fcbb9a0400
Change-Id: Iaf8c4b2e420f610425a07f48db7af32bda3f5b3a
Signed-off-by: Stephen Smalley <sds@tycho.nsa.gov>
When the root directory is included in system, the assumption that
system is mounted at /system breaks down. Redirect /system to /
in that case.
Change-Id: I8031b32d7cb9ca699c1cc22939e28afce9d1c847
When creating incremental OTAs, we should always use the fstab from the
source build instead of the target.
Bug: 21948974
Change-Id: I8b6a18f64b412f782184f9d382544ed0ce093fb4
(cherry picked from commit dd24da9ec9)
When creating incremental OTAs, we should always use the fstab from the
source build instead of the target.
Bug: 21948974
Change-Id: Iff44130f88a679b253ea1adcfac4ed8566ce9cc7
If BOARD_USES_FULL_RECOVERY_IMAGE == true, a full copy of recovery
image will be carried as /system/etc/recovery.img instead of a patch
at /system/recovery_from_boot.p.
Bug: 22641135
Change-Id: Ie271d3e2d55d7b003f667ac5b44203d69b23c63b
(cherry picked from commit 8beab69bd5)
A few properties are stored as links to the files in the out/
directory, which are no longer available when (re)packing from
target_files zip. Redirect those properties to the actual files in
the unzipped directory when repacking.
Change-Id: I18e930cbea224ab9c6db8d4e721b1c266a671db6
With block-based OTA v2 and v3, it requires stash space on the /cache
partition to back up blocks during an update. We need to ensure that
it doesn't exceed the partition size. Since there might be other files
on /cache as well, we use cache_size * threshold as the maximum allowed
size. The threshold defaults to 0.8, which can be overridden by command
line option '--stash_threshold'.
Change-Id: I5c2e353ea4716328d6cdc6a7f758520de8d88b8e
(cherry picked from commit 23ac404212)
file_contexts (specified by SELINUX_FC) is needed both when building
and (re)packaging. We used to use the copy in out/ when building, and
looked for the copy in BOOT/RAMDISK/ when packaging from target_files
zip. With system_root_image enabled, the file_contexts needed for
building and packaging might be different from the one on device. So
we explicitly pack the file as META/file_contexts in target_files zip.
Also refactor out the overriding of selinux_fc property into
common.LoadInfoDict().
Change-Id: I94f9ea6671b3792c12c1c21573840743d63da39a
(cherry picked from commit aa7318c384)
dm-verity may touch some blocks that are not in the care_map due to
block device read-ahead. It will fail if such blocks contain
non-zeroes. As a workaround, we mark them as extended blocks and
zero out explicitly to avoid dm-verity failures.
Bug: 20881595
Change-Id: Id097138bfd065c84eac088b3ad49758010b2927b
(cherry picked from commit 2fd2c9bfd6)
dm-verity may touch some blocks that are not in the care_map due to
block device read-ahead. It will fail if such blocks contain
non-zeroes. As a workaround, we mark them as extended blocks and
zero out explicitly to avoid dm-verity failures.
Bug: 20881595
Change-Id: I54e24e70ad822c0d6d7af43301f74d24505f4461
fs_config function from libcutils, in order to properly set the xattr FS
used to define device specific uid, gid, permissions and capabilities,
now needs TARGET_OUT make variable to be passed explicitly to all tools
using it:
fs_config
fs_get_stats used by mktarball.sh
make_ext4fs used by mkuserimg.sh
mksquashfs used by mksquashfsimage.sh
Bug: 21989305
Bug: 22048934
Change-Id: I6caf9cf870882fce2ead93027767092c29b75ded
Signed-off-by: Thierry Strudel <tstrudel@google.com>
Clockwork builds may rename init.rc to init.core.rc. Change the OTA
script to scan all init.*.rc files to determine the proper location for
install-recovery.sh.
Bug: 22128990
Change-Id: Icad3497d1663e074b938919b40fc239c84fbeb2b
(cherry picked from commit 610754e5ad)
Clockwork builds may rename init.rc to init.core.rc. Change the OTA
script to scan all init.*.rc files to determine the proper location for
install-recovery.sh.
Bug: 22128990
Change-Id: If96bd0b81090683ad0bbfddb735d390204849d9f
In certain cases, we may have no mapping of ZERO or NONZERO blocks in
the image (such as squashfs at the moment). Don't write those groups if
they are empty.
Change-Id: I4a1f0105ada2e1573c3d6162457b83725e0d8971
In certain cases, we may have no mapping of ZERO or NONZERO blocks in
the image (such as squashfs at the moment). Don't write those groups if
they are empty.
Change-Id: I82aa05fbbd79130b08868585edd5298e531c4f5a
Due to the change in https://lwn.net/Articles/546473/, kernel reserves a
few extra blocks (lesser of 2% and 4096 blocks) on ext4 FS which leads to
OTA update failures. Adjust the size computation if the device has
BOARD_HAS_EXT4_RESERVED_BLOCKS := true.
It amends the last attemp in [1]. Now it computes the used blocks from the
make_ext4fs output, instead of altering its argument.
[1]: commit efbb5d2e69.
Bug: 21522719
Bug: 22023465
Bug: 22174684
Change-Id: Iaae6507f6de68a5892f2e3035d330039287b4492
(cherry picked from commit c7a6f1e4f8)
Due to the change in https://lwn.net/Articles/546473/, kernel reserves a
few extra blocks (lesser of 2% and 4096 blocks) on ext4 FS which leads to
OTA update failures. Adjust the size computation if the device has
BOARD_HAS_EXT4_RESERVED_BLOCKS := true.
It amends the last attemp in [1]. Now it computes the used blocks from the
make_ext4fs output, instead of altering its argument.
[1]: commit efbb5d2e69.
Bug: 21522719
Bug: 22023465
Bug: 22174684
Change-Id: I9783a51abe6581ff5c75db81e78ac606d0f32c4c
We should not change the parameter to mkfs_ext4. Bug filed at
b/22174684. Will come up with a new CL.
This reverts commit efbb5d2e69.
Change-Id: I15f7a9a07a65a936ba186d2ce672ec4832ed3e17
In certain cases, we may have no mapping of ZERO or NONZERO blocks in
the image (such as squashfs at the moment). Don't write those groups if
they are empty.
Change-Id: I82aa05fbbd79130b08868585edd5298e531c4f5a
When generating incremental OTAs, we should use the fstab in the source
build instead of the target one. Similarly for recovery_mount_options.
Bug: 22011892
Change-Id: Ia02584c71af7d47835b1f62e7daa46f27940a545
(cherry picked from commit bebd3cfbf9)
Due to the change in https://lwn.net/Articles/546473/, kernel reserves a
few extra blocks (lesser of 2% and 4096 blocks) on ext4 FS which leads to
OTA update failures. Adjust the size computation if the device has
BOARD_HAS_EXT4_RESERVED_BLOCKS := true.
Bug: 21522719
Bug: 22023465
Change-Id: I49f16adbf2dedc5279fbb8622bf99ef71dcc494f
When generating incremental OTAs, we should use the fstab in the source
build instead of the target one. Similarly for recovery_mount_options.
Bug: 22011892
Change-Id: Idb5c72d1a792e8bb40376a380e3dc06136b13652
If there is a SELinux context label in the recovery.fstab, we should
honor that when mounting the partition.
Bug: 19764039
Change-Id: Ic80a3377a5a94c9d10dd464eb1257b157a947510
(cherry picked from commit 548eb76c8f)
If there is a SELinux context label in the recovery.fstab, we should
honor that when mounting the partition.
Bug: 19764039
Change-Id: I4df71a092b418cd340a18b2e3b0936de33d8e3f5
check_target_files_signatures.py was accidentally broken due to the
renaming of a variable in last commit. It escaped the checking from
pylint because it was a keyword ('all').
Bug: 21611858
Change-Id: I58c983e59c4a3e0018481d3c9ba328ed76f5c08a
Build additional images requested by the product makefile.
This script gives the ability to build multiple additional images and
you can configure what modules/files to include in each image.
1. Define PRODUCT_CUSTOM_IMAGE_MAKEFILES in your product makefile.
PRODUCT_CUSTOM_IMAGE_MAKEFILES is a list of makefiles.
Each makefile configures an image.
For image configuration makefile foo/bar/xyz.mk, the built image
file name
will be xyz.img. So make sure they won't conflict.
2. In each image's configuration makefile, you can define variables:
- CUSTOM_IMAGE_MOUNT_POINT, the mount point, such as "oem", "odm"
etc.
- CUSTOM_IMAGE_PARTITION_SIZE
- CUSTOM_IMAGE_FILE_SYSTEM_TYPE
- CUSTOM_IMAGE_DICT_FILE, a text file defining a dictionary
accepted by BuildImage() in tools/releasetools/build_image.py.
- CUSTOM_IMAGE_MODULES, a list of module names you want to include
in the image; Not only the module itself will be installed to proper
path in the image, you can also piggyback additional files/directories
with the module's LOCAL_PICKUP_FILES.
- CUSTOM_IMAGE_COPY_FILES, a list of "<src>:<dest>" to be copied to
the image. <dest> is relativ to the root of the image.
To build all those images, run "make custom_images".
Bug: 19609718
Change-Id: Ic73587e08503a251be27797c7b00329716051927
(cherry picked from commit 5fcf1094f9)
Similar to the assertations in file-based OTA, we perform verification
for block-based OTAs (BBOTAs) after updating a partition, for both of
the incremental and full OTAs. It increases the update time (~20s on
Nexus 6), but will capture unnoticed errors right away.
Bug: 21500869
Change-Id: I0f8b27734caaa0f41f9c1b904d55af2112784a68
(cherry picked from commit 68658c0f4f)
Similar to the assertations in file-based OTA, we perform verification
for block-based OTAs (BBOTAs) after updating a partition, for both of
the incremental and full OTAs. It increases the update time (~20s on
Nexus 6), but will capture unnoticed errors right away.
Bug: 21500869
Change-Id: I89ac8fe623b855721b7affd07cf9426a23433ab0
When passing a ZipInfo instance to common.ZipWriteStr(), the
external_attr attribute should not be overwritten unless specified.
We didn't have the issue previously because we were calling
ZipFile.writestr() directly until [1] merged.
[1] commit 2ed665a033.
Bug: http://b/21309935
Change-Id: I8c0190362c60d7d78965ecfe5e484f8398ddc5f2
(cherry picked from commit 9773465409)
When passing a ZipInfo instance to common.ZipWriteStr(), the
external_attr attribute should not be overwritten unless specified.
We didn't have the issue previously because we were calling
ZipFile.writestr() directly until [1] merged.
[1] commit 2ed665a033.
Bug: http://b/21309935
Change-Id: I374ccd40c174ff9259844f38bdbf187dfe82552d
In ext4 filesystems, some blocks might be changed even being mounted
R/O, such as the superblock (block 0). We need to exclude such blocks
from integrity verification. Plus such blocks should always be
written to the target by copying instead of patching.
Bug: http://b/20939131
Change-Id: I657025b7b1ad50d4365e7b18dc39308facfe864e
(cherry picked from commit ff7778166b)
In ext4 filesystems, some blocks might be changed even being mounted
R/O, such as the superblock (block 0). We need to exclude such blocks
from integrity verification. Plus such blocks should always be
written to the target by copying instead of patching.
Bug: http://b/20939131
Change-Id: I991169ec307dfb231b2fe8908a0668595ecb2060
In order to work around the zip 2GiB limit, we need to wrap the related
functions in zipfile. Calls to those functions should always be replaced
with calls to the wrappers instead.
Bug: 18015246
Change-Id: I499574cee51ec4804bc10cbefe0b17940afed918
(cherry picked from commit 2ed665a033)
In order to work around the zip 2GiB limit, we need to wrap the related
functions in zipfile. Calls to those functions should always be replaced
with calls to the wrappers instead.
Bug: 18015246
Change-Id: Ice494371ca6654e88ded2ae0eb680f51082effcb
leozwang