Commit Graph

5941 Commits

Author SHA1 Message Date
Tao Bao aa7e993a22 releasetools: Support signing APEXes.
Bug: 123716522
Test: Run sign_target_files_apks.py to sign a target_files with APEXes.
Test: Run check_target_files_signatures.py on signed artifact.
Test: python -m unittest test_sign_target_files_apks
Change-Id: I3fa13e3d9461cf5e0838e0572d436e218164fe41
2019-03-15 22:15:51 -07:00
Tao Bao 1cd59f2a26 releasetools: Add apex_utils.py.
The added `testdata/testkey_with_passwd.key` is generated with:

$ openssl pkcs8 -inform DER -in testdata/testkey_with_passwd.pk8 \
    -out testdata/testkey_with_passwd.key -passin pass:foo

Bug: 123716522
Test: python -m unittest test_apex_utils
Change-Id: I7b7f00178ecab35c3e41ecf4d10295fe9bf480ab
2019-03-15 22:08:42 -07:00
Tao Bao d8469727bc releasetools: check_target_files_signatures.py checks APEXes.
Only the container certs will be checked and reported. For the payload
within an APEX, we can't easily extract the cert info.

It needs to go along a longer path, if ever needed, by:
 - extracting public keys from all the available certs;
 - using each of them to verify against an APEX payload to find a match
   (`avbtool verify_image --image payload --key public_key`).

Bug: 123716522
Test: Run check_target_files_signatures.py on target_files with APEXes.
Change-Id: I2ef318e05433d2d65ab84e2dff9e01fb6ee3373d
2019-03-15 20:56:13 -07:00
Tao Bao 2cc0ca1770 releasetools: Add common.ExtractAvbPublicKey().
Bug: 123716522
Test: python -m unittest test_common
Change-Id: I1f645008a14cc882ef280f169c36e4b14a53ef88
2019-03-15 10:47:46 -07:00
Tao Bao badceb2d60 releasetools: Initalize logger when signing.
Other modules have switched to logging module. sign_target_files_apks.py
needs to init the logger to get the logs.

Test: Run `sign_target_files_apks.py -v`. Check outputs.
Test: Run `check_target_files_signatures.py -v`.
Change-Id: Ic68c019f6fb14840561885f1194ad6efdfdb7d82
2019-03-15 09:53:00 -07:00
Chih-Hung Hsieh cc47729719 Accept the "file:(project:)?filePath" directive.
Test: checkowners.py some_owner_files
Bug: 126777637
Bug: 126761563
Change-Id: Iee03521681bc38fa28970a015058e1eba3c8b87a
2019-03-12 11:56:33 -07:00
Daniel Norman 2c99c5bbd2 Adds support for optional config files to configure the merge.
Bug: 124467065
Test: Running `python merge_target_files.py` using the three new flags
and observing that their contents are passed to the merge_target_files() function.
Change-Id: I4de46f041f5ae8bc8be2730313ce873a952bf78e
2019-03-12 16:51:18 +00:00
Treehugger Robot b6f2f8b2d9 Merge "releasetools: Add compatibility for custom out directory." 2019-03-11 19:51:14 +00:00
Pavel Salomatov 3267655004 releasetools: Add compatibility for custom out directory.
sign_target_files_apks script looks for the signapk.jar inside the out dir.
If the our dir is set to a different directory via OUT_DIR_COMMON_BASE the script does not work properly.

From now script checks if the OUT_DIR_COMMON_BASE is set, then searches the jar in the proper path.
If OUT_DIR_COMMON_BASE is unset, searches in "out" like it did before.

Test: Build with OUT_DIR_COMMON_BASE set and unset and verify that sign_target_files_apks works in both cases

Change-Id: I9218b98ff79526184f8353705640193405afac9e
2019-03-11 19:22:29 +03:00
Tao Bao 28a11ef040 Merge "releasetools: Include retrofit info into package metadata." 2019-03-07 18:11:30 +00:00
Treehugger Robot fa2e796ce2 Merge "Remove logical partitions from ab_partitions.txt in retrofit package." 2019-03-07 01:06:42 +00:00
Tao Bao 393eeb4918 releasetools: Include retrofit info into package metadata.
Fixes: 126754990
Test: Generate retrofit and non-retrofit OTAs. Check the package
      metadata entry.
Test: python -m unittest test_ota_from_target_files
Change-Id: I076a013a6224a2ffea8e0b8581890b66f847f165
2019-03-06 16:19:11 -08:00
Yifan Hong b433eba6a6 Remove logical partitions from ab_partitions.txt in retrofit package.
For retrofit updates:
ab_partitions -= dynamic_partition_list
ab_partitions += super_block_devices

For example,
if super_block_devices == ["system", "vendor", "odm"] and
   dynamic_partition_list == ["system", "vendor", "product"],
product partition needs to be removed from ab_partitions.txt
otherwise brillo_update_payload will generate a payload with
product partition in it.

Test: retrofit update that adds a partition
Fixes: 127425410

Change-Id: Id79a410cee3c611ac50d27f14282916aea34f938
2019-03-06 13:35:20 -08:00
Bill Peckham 8ff3fbdd08 Extract common.UnzipToDir, invoke that from merge_target_files.py
This change adds another utility function to common.py: UnzipToDir, which is
generally useful. Refactor merge_target_files.py to use it, and also refactor
other uses in common.py to use it.

Test: ota_from_target_files.py, validate_target_files.py, test_common.py
Bug: 124464492
Change-Id: Ia571070bceb7d3c8002304836bdf688485bf0dd9
2019-02-22 10:57:43 -08:00
Bill Peckham 889b0c6b09 Convert common.RunAndWait method to raise an exception on subprocess failure.
Then refactor the code in merge_target_files.py to adapt to this semantic
change. This makes the code more consistent with existing releasetools code,
and it's easier to follow.

Test: Failure cases (verify exception), success cases (merged target generated)
Bug: 124521133
Change-Id: I56f04e360d8ff8ffcd6245359cdeb79f4565a9c4
2019-02-21 19:02:07 -08:00
Bill Peckham 30ee35f908 Merge "Include merge_target_files.py deps in otatools.zip, use common argv processing." 2019-02-21 04:13:23 +00:00
Tianjie Xu ef7737f921 Merge "Check the completeness of original blocks during target files validation" 2019-02-21 03:41:35 +00:00
Bill Peckham f753e15791 Include merge_target_files.py deps in otatools.zip, use common argv processing.
The merge_target_files.py script needs fc_sort and sefcontext_compile, so
include these tools into otatools.zip via core/Makefile.

Modify tools/releasetools/merge_target_files.py to use the otatools common argv
processing to take advantage of the '--path' option so that we add point the
'--path' option to an extracted otatools.zip package to gain access to fc_sort,
sefcontext_compile, and soong_zip (previously included in otatools.zip).

Bug: 123600124
Test: extract otatools.zip, use --path option to point to it, verify result
Change-Id: I7d84525981b8741c6bdbcac9984256920fc7f417
2019-02-21 00:50:54 +00:00
xunchang c0f77ee489 Check the completeness of original blocks during target files validation
The validate_target_files.py checks the 'incomplete' field of the range
in file_map. And range has already considered the shared blocks and
could be smaller in size than the original file range. Therefore, the
'incomplete' flag was set on the original range in common.py; and we
should switch to use the original range also during validation.

I also checked another flag usage in CanUseImgdiff(), and it has
explicitly rejected cases of shared blocks.

Bug: 124868891
Test: unit tests pass
Change-Id: I03959625d7b81fd83420db98f01d23f54064bcd2
2019-02-20 15:47:30 -08:00
xunchang 1cfe25155e OTA: Add an option to dump the metadata in ota package
Add a flag to write a copy of the metadata to a separate file. Therefore,
users can read the post build fingerprint without extracting the OTA package.

Bug: 124783265
Test: Check the dumped metadata file after generate A/B and non-A/B OTA.
Change-Id: I8918aec87bb81906ef0a7eee774178e9f689d91d
2019-02-20 15:32:09 -08:00
xunchang abfa265f81 Add a flag to skip compatibility check
This serves as a workaround to skip the compatibility check for devices
with incompatible kernels.

Bug: 114240221
Test: generate and check the OTA package for wear device
Change-Id: I65b523a66648af7a77fc3ea79176764fe8ae8d02
2019-02-19 16:27:10 -08:00
Treehugger Robot 01a05483ed Merge "Support verifying system_other" 2019-02-18 05:21:27 +00:00
Bill Peckham 194b1c8508 Merge "Adding initial merge_target_files.py script to merge two target files packages." 2019-02-16 01:40:36 +00:00
Bill Peckham e9eb5f96e0 Adding initial merge_target_files.py script to merge two target files packages.
This script takes as input two partial target files (one contains system bits,
and the other contains non-system, or other, bits). The script merges the
contents of the two partial target files packages to produce a complete target
files package.

Bug: 123430711
Test: Build two partial target files, merge, compare with full target files.
Test: Validate merged target files via validate_target_files.py.
Change-Id: Ic24acf43b86fc703fb4c970688b006291a1861f8
2019-02-15 13:25:16 -08:00
Tom Cherry 9d924f67d3 Use fs_config_generator.py to generate fs_config_files/dirs directly
We want to remove target specific host tools and since
fs_config_generate is compiled with a target specific header file, we
instead remove fs_config_generate entirely and allow python to build
the fs_config_files/dirs files directly from config.fs files and
parsed C headers.

Test: associated unit tests and new end to end test
Test: aosp_sailfish, aosp_crosshatch build produces valid fs_config files
Test: aosp_cf_x86_phone build correctly produces empty fs_config files
Change-Id: Idbc63ff56c0979e1e4c17721371de9d9d02dc8ff
2019-02-15 09:44:09 -08:00
Tom Cherry 1a9f8356fd Error if $(TARGET_DEVICE_DIR)/android_filesystem_config.h exists
This was deprecated along with TARGET_ANDROID_FILESYSTEM_CONFIG_H so
warn if it still exists.

Test: successful error if it exists, no error if it does not
Change-Id: I7dde3c88aaf7ecec23ffe018cbe8b42ebadb2bf8
2019-02-14 10:02:04 -08:00
Tom Cherry f83ba4e8b1 Merge "Fix failing test_fs_config_generator.py tests" 2019-02-14 02:18:11 +00:00
Bill Peckham cc57de3256 Skip some build steps if not building system image.
tools/releasetools/add_img_to_target_files.py: This patch excludes the
inclusion of the system path into the target files if it does not exist (which
it will not if we are not building the system image).

Bug: 123430711
Test: Disable building system image, verify that target files builds without system.
Change-Id: Iaf964ede2b1df5ea4e004b572fd91187a366e75e
2019-02-13 22:57:14 +00:00
Tom Cherry 766adc9cbc Fix failing test_fs_config_generator.py tests
Also run pylint and yapf to fix the style issues.

Test: these same tests
Change-Id: I0a783f7a6f08a90f615d94e9d16da0f314b7813f
2019-02-13 14:34:42 -08:00
Tianjie Xu b8ac90803c Merge "Code improvement to adjusted the blocks size for care_map file" 2019-02-13 19:49:48 +00:00
Shashikant Baviskar 16a73897c3 Code improvement to adjusted the blocks size for care_map file
Bug: 123931528
Test: python -m unittest test_add_img_to_target_files
Change-Id: I589bc9681bccfa07588cab94f53f69d637d6f0f9
2019-02-13 18:34:50 +00:00
Bowgo Tsai 45db7cefb3 Support verifying system_other
This commit extracts the AVB key used to sign system_other.img into
system.img, for init to verify system_other's AVB metadata.

The extracted key will locate in:
    /system/etc/security/avb/system_other.avbpubkey

Bug: 123611926
Test: build and checks the following is generated
      $OUT/system/etc/security/avb/system_other.avbpubkey

Change-Id: Icdc703ff5a0d50f8140bb652507b9b4cbc8a2118
2019-02-13 17:00:49 +08:00
Tom Cherry 8bca10ade4 Merge "Remove TARGET_ANDROID_FILESYSTEM_CONFIG_H" 2019-02-12 21:10:41 +00:00
Steven Laver 9e73e82a12 Allow properties to be derived from partition-specific properties
This change allows ro.product.[brand|device|manufacturer|model|name] and
ro.build.fingerprint to be derived at boot time (and in the OTA
generation scripts) from partition-specific properties.

Test: booted system image, verified properties
Test: booted recovery image, verified properties
Test: unpacked OTA package, verified build fingerprint
Bug: 120123525
Change-Id: Iadd230a0577f35c7c37b0f911e91a5c2863ed1fe
2019-02-11 23:03:00 +00:00
Tom Cherry e0f3ed5cd0 Remove TARGET_ANDROID_FILESYSTEM_CONFIG_H
TARGET_FS_CONFIG_GEN has existed as the preferred mechanism for two
releases, so we finally deprecate TARGET_ANDROID_FILESYSTEM_CONFIG_H.

Test: build
Change-Id: I299a4d1d1d893ac16d6e6ce2ec4659bfcdc19095
2019-02-11 13:45:01 -08:00
Yifan Hong 5cc9230914 Merge "Add kernel info to compatibility.zip" 2019-01-31 19:06:19 +00:00
Yifan Hong 770ab053b0 Add kernel info to compatibility.zip
Add kernel configs / version to verified_assembled_vendor_manifest.xml
so that the kernel of the incoming package can be checked against
the framework. Previously, the running kernel was used instead.

Bug: 111125947
Test: test_extract_kernel
Test: manual OTA on Pixel 3 from build:
      Android P (kernel version 4.9.96)
      to ToT build:
      device kernel version = (manually modified) framework requirement = latest,
      PRODUCT_OTA_ENFORCE_VINTF_KERNEL_REQUIREMENTS = true

Change-Id: Id524a58e94bdb6bba348ca461c9d33614ce451a9
2019-01-30 13:28:03 -08:00
Bowgo Tsai 867ab6606e DAP: building system_other.img for retrofit devices
For dynamic partitions in retrofit devices, system partition will be
a logical partition but system_other is not. However, current
build system use the same settings (logical) for both system.img and
system_other.img, leading AVB unable to locate the footer from the end
of system_other partition.

This commit support building system_other.img with correct partition size
while building system.img as a dynamic image.

Bug: 123506156
Test: check there is "system_other_size=2952790016" in file
      $OUT/obj/PACKAGING/system_other_intermediates/system_other_image_info.txt
Test: build system_other.img, then
      `simg2img $OUT/system_other.img system_other.img.raw`, checks the
      raw image size.

Change-Id: I748320a7770c694d06f06f4a35bfceb622849aa8
2019-01-30 09:58:07 +08:00
Tom Cherry c683233b64 Merge "Remove ro.expect.recovery_id" 2019-01-29 17:07:51 +00:00
Yifan Hong bb2658d5e2 releasetools: Really allow removing partitions
- Add hashtree_info to EmptyImage so that BlockDifference.Compute()
can accept EmptyImage() as target image, which is the case when
a partition is removed.

- BlockDifference also checks source_info_dict to determine
whether a partition is dynamic. When a partition is removed,
its name does not appear in target_info_dict.

- Add tests to ensure DynamicPartitionDifference() still works.

Test: DynamicPartitionDifferenceTest
Test: test_blockimgdiff

Change-Id: Iadb1db075f5dc344db6d5ade358c83b01231e443
2019-01-28 11:10:48 -08:00
Tom Cherry 56ff89a90a Remove ro.expect.recovery_id
This is not used by anyone and the other half of the code to compare
against it is already functionally dead, so remove this.

Test: build
Change-Id: I44ed087cb7735bbc23e30b6c310c80eb3b7b6488
2019-01-28 10:56:37 -08:00
Yifan Hong 79997e5a18 dynamic partition update order is deterministic.
Use an OrderedDict.
Test: builds

Change-Id: Ic3782c1961f0c2a735540b054eb3cace720c50bd
2019-01-25 12:36:17 -08:00
Treehugger Robot d2da2ce398 Merge "Sign system_other.img with AVB" 2019-01-24 05:31:37 +00:00
Yifan Hong 2c0a7bc56d Merge "Really support removing / adding partitions." 2019-01-24 00:51:32 +00:00
Treehugger Robot 2463c3ab2a Merge "Support including DTB image" 2019-01-23 20:56:55 +00:00
Hridya Valsaraju 9683b2f136 Support including DTB image
Bug: 111136242
Test: When BOARD_PREBUILT_DTB_DIR is set correctly,
generated $OUT/boot.img contains the DTB image.

Change-Id: I282e31b04cc60383377b9e9b54f8fe64a8140242
2019-01-23 09:14:55 -08:00
Bowgo Tsai 1e04bf7260 Sign system_other.img with AVB
Support signing system_other.img but shouldn't include it into the
top-level vbmeta.img. system_other verifiation will not be included
in /vbmeta chains and will be done separately.

Bug: 112103720
Test: avbtool info_image --image $OUT/system_other.img
Test: avbtool info_image --image $OUT/vbmeta.img, checks 'system_other' is NOT included.
Test: Checks $OUT/obj/PACKAGING/system_other_intermediates/system_other_image_info.txt
      See the following:
        avb_system_other_hashtree_enable=true
        avb_system_other_add_hashtree_footer_args=--rollback_index 1551744000
        avb_system_other_key_path=external/avb/test/data/testkey_rsa4096.pem
        avb_system_other_algorithm=SHA256_RSA4096

Change-Id: Ia152aaab1387dcf556a42222adb39ea76881263a
2019-01-23 15:17:41 +00:00
Logan Chien 0e53d887d6 Add prebuilt ELF binaries checker
This commit introduces a prebuilt ELF binaries checker.  The checker
will check:

1. Whether all DT_NEEDED shared libraries are specified in
   `shared_libs` (Android.bp) or `LOCAL_SHARED_LIBRARIES` (Android.mk).

2. Whether all undefined symbols in the prebuilt binary can be resolved
   to defined symbols exported by its dependencies.

This ensures that prebuilt binaries won't silently become ABI
incompatible.

To check the prebuilt binaries, all of the dependencies must be
specified in `shared_libs` (Android.bp) or `LOCAL_SHARED_LIBRARIES`
(Android.mk).

If your prebuilt binaries cannot be checked for some reason, you may add
the following property to Android.bp:

    check_elf_files: false,

Or, add the following setting to Android.mk:

    LOCAL_CHECK_ELF_FILES := false

Bug: 119084334
Test: CHECK_ELF_FILES=true make check-elf-files
Change-Id: I523d3083f22fd4053c096d26f61f8375800281c8
2019-01-21 21:35:01 +08:00
Yifan Hong 45433e44dd Really support removing / adding partitions.
- When removing a partition, BlockDifference() object
will have tgt=EmptyImage(). Fix the asserts accordingly.
Also, BlockDifference object now allow tgt=None case.

- When adding a partition, BlockDifference() object
will have src=None. Fix the asserts accordingly.

Also, add unit tests to DynamicPartitionsDifference.

Test: create incremental OTA
Test: test_common.DynamicPartitionsDifferenceTest
Bug: 111801737

Change-Id: I3a35378ecf93111b8f44545cff6ae9696b6b4851
2019-01-18 17:09:33 -08:00
Mark Salyzyn 60fa99d491 build_image: Increase spare inodes to 12.
Test: builds device with DAP
Change-Id: I816a7da54bbe23d4eb1140ac0d101c0fee175f0a
2019-01-17 11:30:12 -08:00