Added support to build system.img that combines contents of /system and
the ramdisk, and can be mounted at the root of the file system.
To enable this feature, define BoardConfig.mk variable:
BOARD_BUILD_SYSTEM_ROOT_IMAGE := true
Ideally we would just change TARGET_OUT (the path of the staging system
directory) to under TARGET_ROOT_OUT. But at this point many places in
the build system assume TARGET_OUT is independent of TARGET_ROOT_OUT and
we can't make it easily configurable.
Instead this implementation takes the least intrusive approach:
We don't change TARGET_OUT or TARGET_ROOT_OUT. We just assemble a
temporary staging directory that contains contents of both TARGET_OUT
and TARGET_ROOT_OUT, in build_image.BuildImage() of
tools/releasetools/build_image.py.
When build_image.py is directly called from the makefile, we pass in the
parameters from the global dictionary; when build_image.BuildImage() is
called from add_img_to_target_files.py, we need to override values to
point to files extracted from the target_files zip file.
We need to combine the fs_config files of both /system and ramdisk,
when fs_config is enabled.
Also this change refactored build_image.BuildImage() by moving the extra
parameters to the image property dictionary.
(cherry-picked from commit 0eabd4f2c5)
Bug:19868522
Change-Id: Iafc467a0e3427b0d6ad3b575abcc98ddcc9ea0f1
This caught a few bugs/syntax errors (a few character classes were not
escaped properly in regex patterns, some indentation was illegal,
etc).
Change-Id: I50637607524e68c4fb9cad7167f58a46b8d26b2c
We need to patch zipfile during close() too, because it refers to the
ZIP64 file size threshold when writing out the central directory
Bug: 18015246
Bug: 19888174
Change-Id: I1b49d653d0831fcc2106808f86c929d7a2b22ff3
Signed-off-by: Greg Hackmann <ghackmann@google.com>
The generated OTAs are supposed to be repeatable. The use of dict in
blockimgdiff.py breaks the assumption for block-based OTAs. Fixed by
using OrderedDict instead.
Change-Id: I945bdc879912ba174ca330c152b1f8fc7ed211ac
Python 2.7's zipfile implementation wrongly thinks that zip64 is
required for files larger than 2GiB. We can work around this by
adjusting their limit. Note that `zipfile.writestr()` will not work
for strings larger than 2GiB. The Python interpreter sometimes rejects
strings that large (though it isn't clear to me exactly what
circumstances cause this). `zipfile.write()` must be used directly to
work around this.
This mess can be avoided if we port to python3.
The bug (b/19364241) in original commit has been fixed.
Bug: 18015246
Bug: 19364241
Bug: 19839468
(cherry picked from commit cd082d4bfe)
Change-Id: I7b5cc310e0a9ba894533b53cb998afd5ce96d8c6
BlockImageDiff has three versions. Only the incremental OTAs generated
with the latest version (3) can be re-applied to the system that's
already on the target build. Otherwise, operations like move will make
unconditional changes and damage the system. During the verification
phase, abort the OTA update if BlockImageDiff is less than 3 and it
doesn't match the checksum of the source build.
Change-Id: Ic630346eab2a993a84d0aeaacd7167ef62cc24f6
(cherry picked from commit daebaa6ed3)
This will only be used when the block file format is at least
version 3. For V1/V2 (L, L MR1) block versions, fall back to
the old range_sha1 check.
Bug: 19357591
Change-Id: I7cb178b70d48ec3c98cdb88ed1c94cf7797a01d0
(cherry picked from commit cad78c12fb)
Add source and target block hashes as parameters to transfer list
commands that copy or patch data to a partition. This allows the
updater to verify the status of each command in the transfer list
and makes resuming block based OTAs possible. Due to the changes,
update the transfer list version to 3.
Needs matching changes from
I1e752464134aeb2d396946348e6041acabe13942
Bug: 18262110
Change-Id: Ia5c56379f570047f10f0aa7373a1025439495c98
(cherry picked from commit cac671a9d1)
Python 2.7's zipfile implementation wrongly thinks that zip64 is
required for files larger than 2GiB. We can work around this by
adjusting their limit. Note that `zipfile.writestr()` will not work
for strings larger than 2GiB. The Python interpreter sometimes rejects
strings that large (though it isn't clear to me exactly what
circumstances cause this). `zipfile.write()` must be used directly to
work around this.
This mess can be avoided if we port to python3.
Bug: 18015246
Change-Id: I8a476d99c5efdef6ea408373b706e9fbd3a798be
When building images set the mount point (data,cache,system) as the
filesystem label on the generated image. This is required for the
Android Emulator.
IMPORTANT: This depends on the following patch to system/extras/:
https://android-review.googlesource.com/#/c/120047/
which has already been submitted to AOSP and merged into the
internal tree.
Change-Id: Iaeb45462570c26d37980f1cc5d8a1b929e476c89
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Allow a simple mechanism to call tune2fs to modify an existing
partition without reformatting.
Bug: 18430740
Change-Id: I9210355b6bfec74d002d1f40b930330740f379a5
A device can set up BOARD_SYSTEMIMAGE_JOURNAL_SIZE (number of journal blocks)
in its BoardConfig.mk.
To disable journaling, set "BOARD_SYSTEMIMAGE_JOURNAL_SIZE := 0".
BOARD_VENDORIMAGE_JOURNAL_SIZE and BOARD_OEMIMAGE_JOURNAL_SIZE work in
similar way.
Bug: 18430740
(cherry picked from commit f3b86357e2)
Change-Id: Icc6c7e1b3be25b8e21b6a737492d944428e4a35a
A device can set up BOARD_SYSTEMIMAGE_JOURNAL_SIZE (number of journal blocks)
in its BoardConfig.mk.
To disable journaling, set "BOARD_SYSTEMIMAGE_JOURNAL_SIZE := 0".
BOARD_VENDORIMAGE_JOURNAL_SIZE and BOARD_OEMIMAGE_JOURNAL_SIZE work in
similar way.
Bug: 18430740
Change-Id: I0594814fe19ffc54dff41ec2464e2e3a802b9419
There may be cases where various partitions were
modified (such as extra signing of boot/recovery)
which require the system partition to be regenerated
with a new recovery patch script. Allow a request
for the recovery patch to be rebuilt, and for
missing images to be readded to the existing images.
Change-Id: Ie93c5bc1161a0d5002002dca23e07742ce2ba468
Change boot, recovery, and verity metadata signing keys to use the
same PKCS8 / X.509 PEM format as the other signing keys, and update
build scripts to use correct arguments for the updated signing
tools.
Bug: 15984840
Bug: 18120110
Change-Id: I23ed5a004ecdad6cf7696487935ad5031eb8adf8
(cherry picked from commit 72d90eb189)
When rebuilding recovery, the boot images created for patching
purposes still were being signed with the old verity key and
not the new one specified on the command line.
In addition, the replacement verity public key in the boot ramdisk
wasn't being used.
Change-Id: I451e17d1cf08c507580c4b58134c1069532740e8
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This will allow safer mount options to be added per mount FS
type, to better ensure data is written during an OTA.
Bug: 18079773, 18092222
Change-Id: I1e3e4fd4639c6fd263e550b770cc3c858ef1e03b
The if statements appear to check for public_signing
and then calls the private_signing function, and vice
versa.
Change-Id: I4511b9bcf0e03b9ba49b69eb80db84cf31d77020
The erase command we insert at the top for full OTAs was getting
inserted in the wrong place for version 2.
Change-Id: I9caf03a40efbdba79f3428f73e50d4319d9ba371
Generate version 2 of the block_image_update transfer list format.
This improves patch size by a different strategy for dealing with
out-of-order transfers. If transfer A must be done before transfer B
due to B overwriting A's source but we want to do B before A, we
resolve the conflict by:
- before B is executed, we save ("stash") the overlapping region (ie
the blocks B will overwrite that A wants to read)
- when A is executed, it will read those parts of source data from
the stash rather than from the image.
This reverses the ordering constraint; with these additions now B
*must* go before A. The implementation of the stash is left up to the
code that executes the transfer list to apply the patch; it could hold
stashed data in RAM or on a scratch disk such as /cache, if available.
The code retains the ability to build a version 1 block image patch;
it's needed for processing older target-files.
Change-Id: Ia9aa0bd45d5dc3ef7c5835e483b1b2ead10135fe
There may be cases where there is an OEM partition
but it has no effect on the OTA itself. In these
cases, ignore an empty value from the misc_info.txt
Change-Id: I5f467e873030765af12810a07ddd5f302ca8cc0b
When generating incrementals for the system and vendor partitions,
check the first block (which contains the superblock) of the partition
to see if it's what we expect. If this check fails, give an explicit
log message about the partition having been remounted R/W (the most
likely explanation) and the need to flash to get OTAs working again.
Bug: 17393999
Change-Id: Ifd2132b428dbc4907527291712690204a3664ac0
Move BlockDifference into common and make its script generation code
more complete, so that it can be use by releasetools.py to do diffs on
baseband images.
Bug: 16984795
Change-Id: Iba9afc1c7755458ce47468b5170672612b2cb4b3
Replace the xdelta/xz-based block OTA generation with a new system
based on the existing bsdiff/imgdiff tools.
Bug: 16984795
Change-Id: Ia9732516ffdfc12be86260b2cc4b1dd2d210e886
Now that we're building and saving images in the target_files at build
time, we should use those images instead of rebuilding them.
Bug: 17201052
Change-Id: I459e650f66f1e0bdf01ad54df9e34f36bf2ee899
This ensures that when the verity key is rotated to a release key
both the boot and recovery images will be correctly signed. It does
mean that they will both be signed with the same key for now, but
as that doesn't change the threat model separating them is just a
distant nice-to-have.
Bug: 15725238
Change-Id: I5b75e4346fe0655065643ab553431690cc1a8cb0
The AOSP bootimage format allows the use of a second stage image
however the BuildBootableImage function does not allows the "second"
optional argument. This patch adds the support of this argument.
Bug: 17035158
Change-Id: I8ed9d9e56449945c2d42fc908269921c394f68c0
Signed-off-by: Benoit Fradin <benoit.fradin@intel.com>
Signed-off-by: Jeremy Compostella <jeremy.compostella@intel.com>
Signed-off-by: Patrick Tjin <pattjin@google.com>
make_ext4fs can now output a file listing the blocks used for each
file in the image. Request this file and save it in the target_files;
it will be used for future improvements to block OTAs.
Bug: 16984795
Change-Id: Id1e60465e3b5a9d126a7934b4d089cf34d8fec44
When making bsdiff/imgdiff patches, give up after 5 minutes. (On
certain large files it can take hours to build a patch, if it ever
even completes.)
Change-Id: I123c06f8194f85f6f4e640f7eb31c7746f76ba4d
Block incremental OTA generation can currently fail on some
target-files pairs. Fall back to generating a full OTA so that the
script succeeds rather than failing.
Change-Id: Ide70395d1f3759aa2076bd173836f6a5e5b397c0
Store sparse images in the target-files, and use those (when they're
available) for building block OTAs.
- New script add_img_to_target_files is added to make the images and
add them to the IMAGES/ subdir in the target-files. It gets run
from the Makefile when building a target-files.
- img_from_target_files becomes mostly vestigial: it creates the
img.zip by just copying the images out of the target-files. (It
still knows how to build images for use on older target-files.)
- ota_from_target_files uses images from the target-files in
preference to rebuilding images from the source files.
- sign_apk_target_files builds images and includes them in its output
target files (even if the input target-files didn't have them).
Bug: 16488065
Change-Id: I444e0d722d636978209467ffc01750a585c6db75
In the past, there was an exception for ext-base fs types to
deal with the lack of image size.
Back then it was only yaffs and ext*.
So now we explicitely only allow yaffs to have no userdata image size.
Change-Id: Ie354ee6222a58228dbcce2c6934971a0737422af
Signed-off-by: JP Abgrall <jpa@google.com>
- Support TARGET_USERIMAGES_USE_F2FS.
- Support BOARD_USERDATAIMAGE_FILE_SYSTEM_TYPE.
- Support "userdata_fs_type" in the prop dict.
- Update build_image to recognize f2fs and call the correct command.
Change-Id: If31cc8bd235f93a4c7814fab36e6e2d13d3037ad
Signed-off-by: JP Abgrall <jpa@google.com>
Make vendor partition a first-class member of the OTA system (for
target_files that contain a VENDOR/ subdirectory).
Build vendor images in a way that is compatible with block-based OTA.
Support updating the vendor partition in both full and incremental,
block and file OTAs. In most cases this is handled by refactoring the
existing code to handle the system partition to handle either, and
then calling it twice.
Currently we don't support incremental OTAs from a target-files
without a VENDOR subdirectory to one with one, or vice versa. To add
or remove a vendor partition a full OTA will need to be done.
Bug: 15544685
Change-Id: I9cb9a1267060bd9683a9bea19b43a26b5a43800d
When building images, we want to use the file_contexts and fs_config
data contained in the target_files zip, rather than whatever happens
to be in the current client.
Change-Id: I13df2405898039f5a9b4bb4837147e76b31b068a
Read and write the same file when testing block patches, which can
turn up errors that don't show up otherwise. (And will appear on the
device.)
Change-Id: Ic9b8d93ec980d13163b135f619af589f41433d7f
Emit script commands to produce a more accurate progress bar (for full
block OTAs) and a working progress bar (for incremental block OTAs) --
ones that are driven by the progress callback from the thing actually
writing the system image.
Change-Id: Ifca10be68cfdaab7135d23515bd0ae5be2f98a16
img_from_target_files.py just skips the boot.img and recovery.img since
there is no kernel or recovery.fstab for emulator.
Bug: 15383279
Change-Id: I4035193e6ab933194ff1417dfae4eab963fe5301
img_from_target_files.py just skipps the boot.img and recovery.img since
there is no kernel or recovery.fstab for emulator.
Bug: 15383279
Change-Id: I4035193e6ab933194ff1417dfae4eab963fe5301
Prior to this change salts were randomly generated, causing OTAs
to fail when they tried to check whether the system they were
installing on was binary identical to the source from which they
were generated.
Bug: 15018770
Change-Id: I86fffe2264a0c263aaf47112fb82fa71240e3403
NamedTemporaryFile's aggressive caching behavior can cause an issue
where changes made by another process aren't visible even after the
fseek() below or a flush(). To avoid this, simply open the file
again and read from the fresh version.
This fixes an issue where verity metadata written by append2simg
doesn't become visible to img_from_target_files.
Change-Id: I291fb3a95d5b532218ac6205ecc9e9b4f3a36bd4
Without this, system images will be built that do not contain the
necessary bits for verification.
Change-Id: I87c15282b26377d7a2a1540e3d0e30b0299622e3
Mohamad Ayyash