This prepares for upcoming changes that refactor verity-related
functions into classes.
This CL makes minimal changes to the moved functions, by creating a new
BuildVerityImageError class in verity_utils.py replacing the former
BuildImageError that's specific to build_image.py. As part of the
change, it also moves the tests for AVBCalcMinPartitionSize into
test_verity_utils.py.
Test: python -m unittest test_verity_utils
Test: `m dist` with aosp_marlin-userdebug (Verified Boot 1.0)
Test: `m dist` with aosp_taimen-userdebug (Verified Boot 2.0)
Change-Id: I19b52714d8980705ea1f9484ac03eb0af9483240
All the releasetools unittests extend the common base class of
test_utils.ReleaseToolsTestCase. Define tearDown() in the base class to
do the clean-up works.
Test: `pylint --rcfile=pylintrc test_*.py`
Test: `python -m unittest discover .`
Change-Id: I51775d964ef032dcdf3bb89c55e1a31371cde708
There's a naming convention between the testcase and the class/method
being tested.
Also remove VerifiedBootVersion2HashtreeInfoGenerator as it's merely a
placeholder without telling anything much (e.g. whether it's to be
implemented or working as intended).
Test: python -m unittest test_verity_utils
Change-Id: Ieb156765c865aa551e3882f381e3a6db2cac4cbc
build_image.RunCommand and common.Run are essentially doing the same
work, but with different interfaces and return values. This CL
consolidates them by moving build_image.RunCommand into common.py as
common.RunAndCheckOutput, with the former calling common.Run internally.
common.RunAndCheckOutput checks the exit code: it returns the output
string on success, or raises common.ExternalError otherwise.
Test: `m dist` with aosp_marlin-userdebug
Test: python -m unittest test_build_image
Test: python -m unittest test_common
Change-Id: I3cd0e9301c576572194618eba8dc00db06c3c35f
I really only care about code that's built for Windows, but I may as
well clean up anywhere that's easy to clean up too...
Bug: N/A
Test: builds
Change-Id: I3ef34fb12ac90e9411b6421e9c23dd8524f056ae
stdout and stderr will default to subprocess.PIPE and subprocess.STDOUT
respectively (which is the expected behavior from most of the existing
callers), unless caller specifies any of them.
Test: `m dist`
Test: python -m unittest \
test_common \
test_add_img_to_target_files \
test_ota_from_target_files \
test_validate_target_files
Change-Id: I43b3f08edfa8a9bcfe54baf9848dc705c048e327
We don't need verity_root_hash or verity_salt to be in the prop dict.
Test: `m dist` with aosp_marlin-userdebug
Test: python -m unittest test_verity_utils
Change-Id: I5a9c50f4741dfb1083b3f590136335b6bc0e5216
Generate the transfer command "compute_hash_tree" for incremental
updates of the non-A/B devices that enable verified boot 1.0
Other changes include:
i. factor out verity_utils to use both in build_image and blockimgdiff
ii. add a new flag 'hashtree_info_generator' in sparse_image to generate
the hashtree information.
Bug: 25170618
Test: generate a package for aosp_angler; and run simulator
Change-Id: I4d4d7a3e41dc3da810d2cbf8988e85d187c9ab0e
The image building functions in build_image.py have been returning
(success, result) or special values to indicate the validity of the
result. The same logic can be better expressed by raising exceptions
instead, because
a) using a special value relies on caller to check for that magic
value;
b) exceptions can carry additional messages other than a boolean does,
e.g. the output from the failing command;
c) caller can have cleaner code flow without explicitly checking for
the validity of the result.
This CL changes such functions to raise on errors. The majority of these
functions are internal to build_image.py only, except for BuildImage()
that has a few callers in add_img_to_target_files.py (which all die upon
error anyway).
Test: `m dist`
Test: python -m unittest test_build_image
Test: python -m unittest test_add_img_to_target_files
Test: python -m unittest test_validate_target_files
Test: Inject an error to the depended binaries (e.g. avbtool), and check
that build_image.py exits with error messages.
Change-Id: Ibe4d51e267756bb1a00fa9238a213f9d55fd9b58
When AVB is enabled with PRODUCT_USE_DYNAMIC_PARTITION_SIZE, AVB
metadata (e.g., hash tree, fec metadata) will consume _RESERVED_SIZE,
resulting in smaller reserved size in file system (e.g., ext4).
Adding additional space for AVB signing and keep the _RESERVED_SIZE only
for file system. This is done by adding a function to binary search an
optimal partition size for a given image size (disk usage + _RESERVED_SIZE).
Bug: 112322265
Test: Build aosp_arm64-userdebug, calculate the running time of
AVBCalcMinPartitionSize() is about 0.3-0.4 seconds.
Test: python -m unittest test_build_image
Change-Id: I8f0051b57701d6fbba6a9db3756dd194066c74b8
Unless using dynamic partitions, `partition_size` should be a fixed
value that equals to the partition size in BoardConfig. It should stay
the same when building an image for that partition. Only the actual
image size that's used to hold the filesystem could be adjusted. This CL
cleans up the uses of `partition_size` and `image_size` to better
reflect such logic.
With dynamic partitions, the only thing that changes is the need to
compute `partition_size` upfront. Once that's done, `partition_size`
should remain unchanged.
Test: `m dist`
Test: `python -m unittest test_add_img_to_target_files`
Test: `python -m unittest test_validate_target_files`
Change-Id: Idedb3e018c95e8f63dc4d9c423be27f30ebb584f
Reads the fingerprint of a partition from the build props and embeds it
into the care_map. Later, the update_verifier will compare the
fingerprint before it performs the verification.
Bug: 114778109
Test: generate care_map for sailfish and new pixel
Change-Id: I06afa01bf6dd3de4456a08ee7960954facc775fb
The recovery image already contains a copy of first stage init, so we
can boot unconditionally to the recovery image and instruct first
stage init whether or not to boot to Android or to recovery. In this
case, we need neither the kernel to mount /system as / nor a separate
partition for recovery, so this change modifies the build scripts to
allow this combination.
Bug: 114062208
Test: Boot pixel from recovery to Android with BOARD_USES_RECOVERY_AS_BOOT
But without BOARD_BUILD_SYSTEM_ROOT_IMAGE
Change-Id: Icd047afb7f22d2724b3bcaca1aa0c837426dcce7
The new suffix distinguishes the new care_map from the ones in plain
text format; and thus the old update_verifier won't report an error
upon parsing failures.
Bug: 115740187
Test: Generate OTA files for Pixels
Change-Id: Ia782afd8cbb0f4bb8c363edaa00e92ab302d5d1b
Migrate off of one target onto another that should be identical.
"conscrypt" is about to be refactored for on-device use and signapk
needs to be unaffected.
Bug: 113148576
Test: make checkbuild
Change-Id: I3cecb4cc8fb8349383a82c8e430c936651e0835a
Bug: 112007947
Test: `m dist` with aosp_taimen-userdebug.
Test: Set up a target that uses chained VBMeta images of
`vbmeta_mainline` and `vbmeta_vendor`. `m dist` and check the
build log, as well as outputs from
`avbtool info_image --image vbmeta.img`,
`avbtool info_image --image vbmeta_mainline.img`,
`avbtool info_image --image vbmeta_vendor.img`.
Change-Id: Ib1d4e97f583b65245703eae15d211adcd9e83741
Also removing the requirement of BOARD_BOOTIMAGE_PARTITION_SIZE when
BOARD_AVB_ENABLE is set. Some targets (e.g., emulator) doesn't build
boot.img when AVB is enabled.
Bug: 112293933
Test: build a failure case and checks the error output is expected
Change-Id: I290c707719193ddaedcd9fd0b31de566ad17078c
This is in line with the change of mkuserimg_mke2fs.sh to python binary.
Bug: 112555072
Bug: 63866463
Test: unittests pass
Change-Id: I82c0be1e5bbc685edc15120da73aa43fdc9f2f05
Commit d572632f3d added support in build
system to blacklist given vendor properties. When
ro.vendor.build.fingerprint gets blacklisted, the OTA generation script
can't rely on those properties to determine a change in vendor images.
This CL considers such a case as "vendor images must have changed"
between the two builds.
Bug: 113892939
Test: Generate an incremental package with builds not having
ro.vendor.build.fingerprint.
Test: python -m unittest test_ota_from_target_files
Change-Id: I188de9c3cbeecf26132c92b9356e9d5fef75205e
super_empty.img is needed for "fastboot update" to work, as such, it
must be included in the update package. This change adds lpmake
parameters to misc_info.txt for add_img_to_target_files.py, and ensures
that lpmake is packaged with otatools.zip.
The build-superimage-target macro is now split into two functions - one
to generate the device-specific arguments, and another to build the full
command-line for convenience. The former is used to pass arguments
directly to add_img_to_target_files.
Bug: 113524256
Test: make updatepackage builds and includes super_empty.img
make otatools includes lpmake
Change-Id: I25091c964b036beeea2a8b8f738e2c18937c1eb4
It used to accept a non-None input_dir to indicate the need to
re-generate images (called from add_img_to_target_files.py). During that
flow, both of the two parameters redundantly refer to the same input
dir. This CL replaces the second parameter with a bool value instead.
Test: `python -m unittest test_common`
Test: `m dist` with aosp_taimen-userdebug
Test: `zip -d aosp_taimen-target_files-eng.zip IMAGES/\* &&
add_img_to_target_files.py -a aosp_taimen-target_files-eng.zip`
Change-Id: I0a5a164366acb116407f94bb350872a3a0b207d1
In particular, it replaces the generated calls to edify function of
`apply_patch()` and `apply_patch_check()` with `patch_partition()` and
`patch_partition_check()` instead.
It adds two functions to EdifyGenerator: PatchPartition() and
PatchPartitionCheck() for Python releasetools callers. It also tries to
handle the callers of obsolete EdifyGenerator.PatchCheck() and
EdifyGenerator.ApplyPatch(), if their inputs are in known format.
Otherwise it raises an exception to avoid generating an OTA that updater
doesn't understand.
It requires the matching updater changes in the same topic.
Bug: 110106408
Test: Generate an incremental package with the new script and updater.
Apply the package on device.
Test: Revert the change in ota_from_target_files.py, so that it calls
the obsolete EdifyGenerator functions. Check that it generates the
same incremental package.
Change-Id: Ifc55cb40d3a45116fd4d408536d037eca249effa
Support a first stage ramdisk which will include the targets placed
into TARGET_RAMDISK_OUT. This replaces the existing ramdisk on
existing devices.
All system images are now built to be mounted as the root dir.
Devices with a first stage ramdisk will switch root to the system
partition.
BOARD_BUILD_SYSTEM_ROOT_IMAGE remains and is used to specify if the
system partition is going to be directly used as rootfs without the
ramdisk.
Bug: 79173823
Bug: 79758715
Test: hikey boots, sailfish boots
Test: OTA walleye from P to master
Change-Id: Idbb2dccc6340b0235a4bef03e11e420a9ed154b6
Bug: 111179267
Test: flashed a module with NOTICE file on a devices and checked the
Legal information webview.
Change-Id: I6e52dee0ea23f05ea73d7cc747c2db1c1525f4b2
This adds a set of standard build properties that are shared
between the system, vendor, odm, product, product_services and
bootimage partitions. The following properties are added:
ro.X.build.date
ro.X.build.date.utc
ro.X.build.fingerprint
ro.X.build.id
ro.X.build.tags
ro.X.build.type
ro.X.build.version.incremental
ro.X.build.version.release
ro.X.build.version.sdk
ro.product.X.brand
ro.product.X.device
ro.product.X.manufacturer
ro.product.X.model
ro.product.X.name
vendor and odm already had all of these, but bootimage, product
and product_services only had a subset and system didn't have
any.
Bug: 78359442
Bug: 111491184
Test: m out/target/product/hikey64/system/build.prop and inspect
Change-Id: Ia346d61aec331ed1b9fea12e6a50f1b79147473d
Otherwise it gives a floating-point value (i.e. 1230768000.0). e2fsdroid
(i.e. the user of the value) didn't complain though, because it calls
strtoul(3) without checking endptr.
Bug: 80600931
Test: `m dist` with aosp_taimen-userdebug, check the build log.
Change-Id: I7467a60c947134b42f079c7a57fb8a9a9d4277da
The recovery image will be packed under BOOT/RAMDISK only if
system_root_image and recovery_as_boot both are true (e.g. non-A/B
devices launched since P).
Bug: 113191245
Test: Run sign_target_files_apks.py on a target_files file that uses
system-as-root but not recovery-as-boot.
Change-Id: I262a268055c6b5078d21694b5094a1c393d0d37c
The change in [1] moved the recovery etc files from /etc to /system/etc.
However, we may use the latest OTA tools to build incremental OTAs for
old target_files zips. This CL adds a workaround to look at both of the
old and new locations.
[1] commit 696bb33676
Bug: 113073663
Test: Build a previously failing incremental OTA for taimen.
Change-Id: Ie07aa1713e616d523838b1260a992b20f5a11612
When the update package gets larger than 2 GiB, payload.bin offset
mentioned in metadata file for ota-streaming-property-files gets
shifted (CrAU of payload.bin) because ZipInfo FileHeader() returns
incorrect value. To solve the issue, offset is re-calculated from
fixed bytes of central directory file header, filename length and
extra length.
This patch is to sync with update_device.py script.
Test: manually create an A/B update package and run it using
update_device.py
Bug: 111198589
Change-Id: I9bf5a5ca24938cad3206d04af529f70d45e992c0
They used to be installed under recovery/root/etc. This CL moves the
files to the new location and creates a symlink from /etc to /system/etc
(done by the rule in system/core/rootdir). This gives similar layout
between normal boot and recovery, and allows installing prebuilt_etc
files with Soong (`recovery_available: true`).
As part of the change, we no longer need the whitelisting rule for
mke2fs.conf.
Bug: 112780007
Test: Build with other changes in the topic (aosp_taimen-userdebug).
Check the generated files under recovery (/etc being a symlink to
/system/etc).
Test: Boot into recovery. Verify basic functionalities (`adb shell` and
`adb sideload`, factory reset).
Test: `build/soong/build_test.bash --dist`
Change-Id: Ibb6dea6f179a339f0c2d0fd8ba05ec0085b79a12
Tao Bao