The oem partition allows system build.prop to import properties from it
by "import /oem/oem.prop xxxx". An _ImportParser() error was raised on
this case.
BUG: 154171021
Test: 1) "atest --host releasetools_test releasetools_py3_test -c"
2) On a device who has oem partition, "make dist" and sign its
target zip file.
Change-Id: I47875bf7a698390e11690150e6516a3064550ca0
The secondary payload is used by factory OTA to speed up the first
boot of the device. The payload should only contain bootloaders
images and the precompiled apps in system_other. Remove vendor_boot
to reduce the secondary payload size.
Test: build a package with secondary payload
Change-Id: I3a9f6dda00a5aac038438065e2bc6483d71b5e95
The different boot images in aosp_arm64 have different partition size
requirements. Add support for defining a partition size associated with
each boot image variant.
This support is limited to aosp_arm64 currently, which is built with
recovery-as-boot.
Bug: 156036850
Bug: 155049180
Change-Id: I73b70a4d52c2662c3015899632bb8a2b424108d5
This reverts commit b21e48b499.
In practice, some partners use the 'import' statement to override
the device fingerprint at runtime. The runtime fingerprint will
later add to the metadata of OTA package, so that the OTA server
can deliver the package to corresponding devices correctly.
This CL supports parsing a subset of import statement that the init
process recognizes. And we loose the restriction based on how the
dynamic fingerprint is used in practice. Right now, we only searches
for the override of brand, name and device. And the placeholder
format should be ${placeholder}, with its value supplied by the
script caller.
As part of the implementation, we generate all the possible
combinations of the input boot variables. And recalculate the
fingerprint for each of the combination. Though we load the
build.prop multiple times, the logic is easier to follow. Also,
it's more convenient to enhance the logic if we only want to
allow some of the boot variables combination later.
Bug: 152167826
Change-Id: I4a9fa35c7ac037ff1cf4f9a4bdff602beac3894b
Test: unittests pass
This reverts commit 6022545272.
The build prop for a partition used to be a simple key:value
dictionary. But we need more fields to hold the alternative build
props overriden by the 'import' statement. Therefore, add a new
class as a wrapper for these props first.
Bug: 152167826
Change-Id: I5c952cd2a976ba1a09ddc66d56c2b8b55a61986b
Test: unittests pass
In practice, some partners use the 'import' statement to override
the device fingerprint at runtime. The runtime fingerprint will
later add to the metadata of OTA package, so that the OTA server
can deliver the package to corresponding devices correctly.
This CL supports parsing a subset of import statement that the init
process recognizes. And we loose the restriction based on how the
dynamic fingerprint is used in practice. Right now, we only searches
for the override of brand, name and device. And the placeholder
format should be ${placeholder}, with its value supplied by the
script caller.
As part of the implementation, we generate all the possible
combinations of the input boot variables. And recalculate the
fingerprint for each of the combination. Though we load the
build.prop multiple times, the logic is easier to follow. Also,
it's more convenient to enhance the logic if we only want to
allow some of the boot variables combination later.
Bug: 152167826
Test: unittests pass
Change-Id: Iac093a40dc6f873c5e1858efa44cb2bd6082508a
The build prop for a partition used to be a simple key:value
dictionary. But we need more fields to hold the alternative build
props overriden by the 'import' statement. Therefore, add a new
class as a wrapper for these props first.
Bug: 152167826
Test: unittests pass
Change-Id: I2fe7e93a2f4de8e55f5f8051b000b96b5efdc85a
The oem_dicts in BuildInfo is only available to ota_from_target_files
when it's called with "--oem_settings" input. However, aosp/1135332
starts to use BuildInfo in add_img_to_target_files w/o oem_dicts. An
assert is triggered due to it when oem_fingerprint_properties is in the
info_dict.
This change skip _oem_props reading if oem_dicts is None to allow to
this case.
BUG: 155360923
BUG: 154171021
Test: Sign images with oem_fingerprint_properties.
Change-Id: I6a73feecd9567fd4d85f4eab3d4e11c5df28fe39
For older target_files.zip, since "recovery_mkbootimg_args" may not be
set fallback to the earlier config -"mkbootimg_args" for recovery image.
This will make releasetools backward compatible, and usable for
older/already generated target-files package.
Change-Id: If40359130b8e6da48cb8c345d8cb079f457fe36f
check_vintf now checks deprecation as well, which
requires matrix fragments to exist. Update tests to
reflect this.
The test also uses legacy vendor manifest path, which makes
it not combined with the ODM manifests, causing sku_compat
test to fail. Fix it by using the new vendor manifest path
(which libvintf correctly combines it with the ODM manifests.)
Fixes: 155131894
Test: python -m unittest test_check_target_files_vintf
Change-Id: Ib660f8796efc3465d9513688695dac29c63a7514
When loading build info from a previous version of Android, the set of
allowed property sources should match those available in that version.
In this particular case, the product_services partition was a valid
property source in Android 10.
Bug: 155053195
Test: ran unit tests from test_common.py
Test: generated an incremental OTA which previously failed
Change-Id: Ic0b0a112656533eca78dee31517deff7e3c8d7cc
The arguments transparency_log_servers and transparency_log_pub_keys
have been merged. Update the caller in the OTA script accordingly.
Also disable the test to contact aftl server until we have
a public server.
Bug: 153940575
Test: check the argument
Change-Id: If6a7e7d644884d395c75c2fcdfd6aa7c2380d851
Support for generating multiple boot.img files is required in the
release tools to enable GKI distribution/signing.
Bug: 151094943
Change-Id: I536a286d3123f35918106a52c49b1148d746370f
Add support for building a non-v3 recovery image along with a
v3 boot.img and vendor_boot image.
Bug: 150825361
Change-Id: I80d4e71ff09b83dd1c7f9accb4de99a30f95f4bf
Also add multi-cert support to prebuilt apps so that they can benefit
from the new lineage feature.
(This is a cherry-pick change.)
Test: m GoogleServicesFramework w/ modified build rules
Test: m PrebuiltGmsCore w/ modified build rules
Test: apksigner lineage -v --print-certs -in <built_module_path>
Fixes: 152897457
Change-Id: If7d5d4bd308629c8340231520214c76c8a568a65
Merged-In: If7d5d4bd308629c8340231520214c76c8a568a65
Since we might use ToT release tools to sign a package
generated by an older build, we make the new
`partition=` tag optional. This also means we need to be
careful to use non-greedy regex matching.
Bug: 153133823
Test: python3 -m unittest
Test: input with and without the new `partition=` tag
Test: new test_ReadApkCerts_WithWithoutOptionalFields
Change-Id: Ic57efd34e745ad302ae17150c6f2318f0b4524cb
As we support zip64 in libziparchive, we change the parameter of
the ReadAtOffset() in zip reader from uin32_t to off64_t. So the
derived class needs to be updated as well.
Bug: 150900468
Test: build
Change-Id: Icbfd2dd54b01ff62da988ba8598c1057f9bb6368
Right now we assert that the aftltool should exist since the one
in aosp won't work due to grpc dependencies.
Bug: 147870995
Test: build
Change-Id: Iabb2c375167572a965493a7648fdc1abe287af67
Some required modules from host tool (such as imgdiff) are bound to
required from releasetools_common, but unfortunately required modules
from python_library_host does not work (b/140144201). To make this work
we should mark dependency from each host tool.
Bug: 152258157
Test: Build succeeded without system image
Change-Id: Iedfda9b48d78687d8f622b818eeb93896d4f119d
Propagate partition tag data to apexkeys.txt and
apkcerts.txt so that merge_target_files.py can build
merged versions of these files by filtering the
framework files for framework partitions and filtering
the vendor files for vendor partitions.
Bug: 138942268
Change-Id: Ic3226728e97dae84d38ec230ccc86d1b124bea94
* When $OUT is on a compressed disk,
`du -k -s` might not return the correct sizes.
* Adding `-b` gets us close to the actual size.
toybox commit: https://github.com/landley/toybox/pull/177
Test: m, builds and boots.
Devices: crosshatch blueline bonito sargo (anything with dynamic partitions)
When doing apk-in-apex signing, the package name is not reserved
during the apex repacking. As a result, the name accidentally
reverts to 'com.android.wifi' from 'com.google.android.wifi'.
This cl changes the behavior to call 'apexer' by passing the
'--payload_only' argument. So we don't build the apex file from
scratch and the old AndroidManifest.xml will be reused.
Test: 152084536
Bug: unit tests pass
Change-Id: I8332b2ee84832fb196f2e1c4309abac5ab92e153
The otatools should talk to the aftl server and append the inclusion
proofs when building the vbmeta image. We should only do this during the
signing process when the network is always available.
Also the inclusion proof doesn't impact the final vbmeta image size on
coral, where the final size is 8192 after padding. This is below the
physical image of 65536.
Bug: 147870995
Test: unit tests pass, run sign_target_file_apks
Change-Id: If84c6bf5198c9b05f5e0c16ae6335971915f47e3
In the case when both ODM_MANIFEST_SKUS and ODM_MANIFEST_FILE are
both empty, also checks VINTF compatibility for empty SKU case.
Same for device manifest. Although right now, DEVICE_MANIFEST_FILE and
DEVICE_MANIFEST_SKUS cannot be both empty (because libvintf requires
vendor manifest exists barring manifest fragments), let libvintf throws
an error for this case. In the build system, simply handle vendor manifest
the same way as ODM manifest.
Bug: 148601741
Test: manual
Change-Id: Ia0655fd4d37f6286b164a3221ec4b2716f29147d
If DEVICE_MANIFEST_FILE / ODM_MANIFEST_FILES is set, also include check
for empty vendor / odm SKU, respectively. Then, do cross product of
vendor SKUs and ODM skus, and check each possible combination.
Test: build with and without vendor skus
Test: m check-vintf-all
Bug: 148601741
Change-Id: I23cc81998e7afc36c43540bf6d615b4d4b288447
Problem:
Logical partitions do not update right since commit
f67dd8059d
Generate block based OTA for product partitions
due to the wrong order of parameters. This is logged in
/tmp/recovery.log:
skipping 0 blocks already patched to 202
Bug: 149794632
Change-Id: I92f7a8a226a8a96b892254362d362c515bfb6dc6
This causes the output image files of a merged build to be identical
to the image files of the input partial builds, for each images in
PARTITIONS_WITH_CARE_MAP.
Test: python -m unittest test_common
Test: `m dist`; `unzip out/dist/target_files.zip IMAGES/\*`;
`zip -d out/dist/target_files.zip IMAGES/\*`
`add_img_to_target_files -a out/dist/target_files.zip`.
Verify that the rebuilt images are identical to the deleted ones.
Test: Build a merged target (using merge_target_files.py). Verify that
the partial target-files.zip IMAGES are identical to the merged
target-files.zip IMAGES for PARTITIONS_WITH_CARE_MAP images.
Bug: 150405807
Change-Id: I5fdf5783c1aff9c14cf5408090389b1f65b69ca6
The monotonic property of the range is lost after the recent change to
subtract 0th block from the file range. As a result, we read the block
in the wrong order when pull files from the raw image. This yields a
incorrect hash during validation.
Bug: 150334561
Test: run validate_target_file. unittests
Change-Id: Id4fedaaf273c42c8d21a4549fda4c9a45e22066a
* add more Java, Kotlin, and Rust warning patterns
* recognize more clang-tidy cert-* and bugprone-* warnings;
still keep the catch-all 'cert' and 'bugprone' groups.
Test: warn.py build.log > warnings.html
Change-Id: I909ec09822f42f60493ebe943c055bf0d4b69938
* add chrome_project_list.py
* use newer Chrome's warn_common.py and severity.py
* modify warn_common.py to work with both python2 and python3,
to work with and without google3
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I9aeee7031c11988d8ead1733ff4846ae6cb1ee73
AVB public keys might be installed into the first-stage ramdisk.
This CL supports removing some AVB keys during the signing process,
which can be useful when a key needs to be revoked.
Also correcting the path of force_debuggable detection.
Bug: 150109393
Test: make dist
Test: ./build/tools/releasetools/sign_target_files_apks \
--remove_avb_public_keys key1.avbpubkey,key2.avbpubkey \
out/dist/*-target_files-*.zip signed-target_files.zip
Change-Id: I1af95adbbec655f40121f3392965dd1950aca4e5
e2fsdroid can export holes as a "0" block, in order to make sure logical
and physical block assignments are maintained properly. Exclude these
hole blocks from range sets in LoadFileBlockMap.
Bug: 139201772
Test: m dist on non-A/B
Change-Id: Ic04e869572229678fdeaaa6fbf6f464d6c672051
This change makes the oemaids_headers header library
available to vendor side.
Bug: 149665231
Test: Tested compilation, oemaids_headers is available to
a sample vendor binary.
Change-Id: I87369f1db883e40e83d6076defdd4c0d84aac46f
Regardless of the value of "ext4_share_dup_blocks" "-c" flag is passed
unconditionally. The correct behavior is to check if
"ext4_share_dup_blocks" is set to "true".
Test: 1. build bonito with "BOARD_EXT4_SHARE_DUP_BLOCKS := false"
2. check for "shared_blocks" feature of system.img
./out/host/linux-x86/bin/tune2fs -l \
./out/target/product/bonito/system.img | \
grep -o shared_blocks
Change-Id: I8c0e5606658dd0f47b71cfc6c453a77a57da4f42
If a key is specified, the avbtool always use the input key to verify
all the chained images. And this will cause failures when the vbmeta &
system use different keys (e.g. RSA 4096 vs RSA2048). Because the public
key to vbmeta will always fail to verify the system image. Remove the
'--key' parameter in the verification command, so the avbtool will use
the embedded public in the image.
Test: validate target-file from sdk_gphone_x86_64
Bug: 148916990
Change-Id: I9d31be0f8c32af605af94fa73d07818f40f51ec4
We should add quote when passing the signing arguments to apexer. So
the final argument whould look like
apexer ... --signing_args "--signing_helper_with_files=%path"
And the argument parser in avbtool will eventually parse the
"--signing_helper_with_files" correctly.
Bug: 148627666
Test: unittest pass, run a smoke sign_target_files_apk with signing arguments
Change-Id: Ie203d26a508cb98ca94e6c364a2640b681ad79f2
In the build signing environment, we don't necessary have access to the
physical private key. And we work around it by passing an extra
signing_helper to the avbtool instead of a private key. Since we need
to perform signing to add the hashtree footer when rebuilding the apex
image. We should propogate the signing_helper to apexer and eventually
to avbtool.
Bug: 148627666
Test: unit tests pass. Smoke test by pass a public key to apexer and
hardcode the private key in signing helper script.
Change-Id: I56d57485493a5ba2cbe4c49e493ee9604f68088b
We should only provide the --assets_dir option when the assets directory
is available in the original apex.
Bug: 148452817
Test: unit tests pass
Change-Id: I44308b80a43ff3f441223f0ecc248991d1d83f31
So when $ANDROID_BUILD_TOP is missing, the error message print out the
path of the missing file; instead of a type error.
Bug: 148452817
Test: check result with missing ANDROID_BUILD_TOP
Change-Id: I5f841f6626eb3843364f20e78b278d3244505396
This adds the ability to format f2fs and ext4 in a way that supports
casefolding and project quotas. Project quotas require wider inodes, and
thus cannot be enable apart from at build time. Casefolding can be
configured with f2fs.fsck/tune2fs later.
Both features are enabled via inheriting emulated_storage.mk in the
device.mk
Bug: 138322712
Bug: 138321217
Test: Enable on device with related patches, view fs configuration on /data
Change-Id: Ib1ab0589006ff1b3fb7478b7e9b0a8aad71e4b09
Some apex payload images contain apk files. And these apks need to be
signed during the signing processed when sign_target_files_apks is
called. To support the signing, we can extract the payload and repack
the apex file with the (de)apexer tool. Add the signing support in the
apex_util.
Bug: 146508800
Test: unit tests pass, run sign_apex, sign_target_files_apks
Change-Id: If6d58975248709a144b07dbabf47c27916e5695e
Previously, we only check VNDK core and vendor variants are identical
when a VNDK library is not declared to have different variants AND the
target has TARGET_VNDK_USE_CORE_VARIANT set. Therefore, it is fairly
easily to break a TARGET_VNDK_USE_CORE_VARIANT target as it needs to be
tested explicitly.
This change uses the new LOCAL_CHECK_SAME_VNDK_VARIANTS and expands the
check to run regardless of TARGET_VNDK_USE_CORE_VARIANT. Also adds
support for VNDK-in-product.
Bug: 145157349
Test: Build success for targets with and without
TARGET_VNDK_USE_CORE_VARIANT.
Test: With the corresponding change in build/make, remove libbinder
from build/soong/cc/config/vndk.go and check build fails even
when TARGET_VNDK_USE_CORE_VARIANT is not set.
Change-Id: Iec708b971072e6580f77a03e243b30b89b3b054d
* HTML emit functions now take a writer parameter.
This makes warn_common.py one step closer to the ChromeOS version.
* Add new found warning patterns from java and yacc.
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I5c446ca767746598f07603591fdf98f7d82cae17
* Remove the useless 'option' key.
It is only used in some C/C++ warning patterns
to give a hint of options to turn to -Werror.
Now the global default is -Werror.
* Factor out common code patterns into high/medium/low functions.
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: Ibd3f768b1552ada925eb5afb0f01ab674c968a87
* This new class definition and patterns are
shared between Android and ChromeOS compiler tools.
* Suppress hard to fix and false positive linter warnings.
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: Icb47809100ad30796cb1da82610e989d450194fa
* Mostly new Java and Kotlin warnings found in nightly builds.
* More clean up of Java and C++ warning patterns will follow.
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: I8776401d15c3a59535722d52a5eec03f954d3b15
Change I366099c3dfd5fa4282745ef258a8cf35338e1e42 introduced a more
restricted search where we only look in out/host/x/bin instead of
out/host/x when looking for files.
Because signapk is in out/host/x/framework, and lib64 is in out/host/x,
we need to go one level up for this to work properly.
Fix: 147096688
Test: atest --host releasetools_test releasetools_py3_test
Change-Id: I597e1d95c09eb0e27b2faebb107218761901d9ac
While merging the Dynamic Partition info Dicts in the case of
split builds, pick the virtual-a/b flags like virtual_ab,
virtual_ab_retrofit from the vendor half.
Bug: 146811936
Test: Merged build contains the virtual a/b defines.
Change-Id: I0171b074094e47cfd12dc6af2565acdcd1b65163
Compared with board_config.mk and found this one is missing.
Also add docs to explain where are these values from.
Test: pass
Change-Id: Ic22d75a538657bd286edcae6d95f9ee462911975
* Split warning patterns into *_warn_patterns.py
* Split project list into android_project_list.py
* Split out the Severity class, to be changed later
* Split core of warn.py into warn_common.py and
leave only platform dependent code in warn.py.
This allows the core logic be used with different
parallel processing libraries.
* Old warn.py just calls -m warn.warn.
Test: path_to_build/tools/warn.py build.log
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Change-Id: I6734e4472a21018cd5ce06d549f6dbca24f4de54
Note that we are currently excluding this flag for retrofit devices,
since it necessitates support for a newer super.img format, which
retrofit devices may not fully support.
Bug: 134949511
Test: m -j, lpdump super_empty.img
Change-Id: Id69a8d755b9f4c4bb5fbd456a416ced1a5f1d135
Adds AFTL relevant options in preparation for avbtool changes. The
options are:
aftl_server: A string representing the URL of the transparency log.
aftl_key_path: The path to the public key associated with the
transparency log (in PEM format).
aftl_manufacturer_key_path: The path to the key associated with the
manufacturer for AFTL message signing.
aftl_signing_helper: The signing helper for use with AFTL message
signing.
Change-Id: I73d466f24b171c16dbdaa40c589779bed52be551
Test: Ran add_img_to_target_files with all AFTL options.
Bug: 138779249
Share the code to handle base_fs files rather than duplicate it for
system/vendor. At the same time, add support for system_ext and product
partitions.
Bug: 139201772
Test: manual test
Change-Id: Ib1b7c018c828901e24eefc01e3faead03e77bc2d
There is no functional change
Bug: 130257221
Test: generated an archive using the script and ran the result
Change-Id: I946f2827a9519873be2173fc50d9bc1cc46a326f
Hongguang Chen