Commit Graph

5739 Commits

Author SHA1 Message Date
Tom Cherry e64694118a Merge "Require a vendor_ prefix for users/groups in /vendor/etc/{passwd,group}" 2018-05-15 22:36:34 +00:00
Tao Bao cf7000e2da Merge "releasetools: Use "ota-downgrade=yes" for --override_timestamp." am: 3d840c6b98
am: 82054712c9

Change-Id: I0e0aa585a8f15624519220479bf064745f1e224c
2018-05-15 10:12:01 -07:00
Tao Bao 3d840c6b98 Merge "releasetools: Use "ota-downgrade=yes" for --override_timestamp." 2018-05-15 16:58:11 +00:00
Jiyong Park 6cb36a5d6c Merge "Convert apicheck tool build to Soong." am: 5e2056b906
am: 90525df267

Change-Id: Ia8242a167369f545c24f76911d315cf7e2b30d4c
2018-05-15 00:05:58 -07:00
Jiyong Park 5e2056b906 Merge "Convert apicheck tool build to Soong." 2018-05-15 05:30:41 +00:00
Jiyong Park 1c671fd7dd Convert apicheck tool build to Soong.
Bug: b/78034256
Test: m clean && m checkapi
Change-Id: Iaced7335714138730349bbea8a993705f197ae77
2018-05-15 11:38:44 +09:00
Tom Cherry 2d197a1e19 Require a vendor_ prefix for users/groups in /vendor/etc/{passwd,group}
Bug: 79528966
Test: successful build with vendor_ prefixed users/groups
Test: expected build failure when not using vendor_ prefixed users/groups
Change-Id: If006c70178aa5bdcc9521a06ef8df2500f70bbb9
2018-05-14 13:16:50 -07:00
Tao Bao 143d0925d9 Merge "releasetools: Skip validating non-sparse images." am: 138df4ef77
am: 30846b5745

Change-Id: If0130952f0c6723ed74931a6ba7e7b977176f2f4
2018-05-14 10:53:11 -07:00
Tao Bao 138df4ef77 Merge "releasetools: Skip validating non-sparse images." 2018-05-14 17:37:53 +00:00
Tao Bao 63e2f49fdd releasetools: Skip validating non-sparse images.
Targets can define 'TARGET_USERIMAGES_SPARSE_EXT_DISABLED := true' to
generate non-sparse system images, but validate_target_files.py doesn't
work with such images. This CL adds a workaround to temporarily skip the
file consistency check for such images.

Bug: 79616357
Test: Run validate_target_files.py on a target_files.zip that's not
      using sparse image.
Test: Run validate_target_files.py on marlin target_files.zip (which
      uses sparse image).
Change-Id: I1f4066c5b3fec595b10cab10283d62c1c5a6c624
2018-05-11 23:38:46 -07:00
Daniel Colascione 4e7b25b67a Merge "Add zip hint generation support to signapk tool" am: 198b9f4f6a
am: 202f9d4ce1

Change-Id: I80c1232d6a84319a2331813fc4fade686b614b5a
2018-05-11 16:45:39 -07:00
Daniel Colascione 334ece99dd Add zip hint generation support to signapk tool
Test: unzip -q -c myapp.apk.signed pinlist.meta | od --endian=big -w8 -tx4
Bug: 79259761
Bug: 65316207
Change-Id: I71c01ac24e93afe75f60697a9849e1dd35e1b49d
2018-05-10 14:39:20 -07:00
Tao Bao faa8e0b7e9 releasetools: Use "ota-downgrade=yes" for --override_timestamp.
We used to write a fake post-timestamp into package metadata for
"--override_timestamp", which allows a package to be pushed based on
the fake timestamp. This CL stops doing that by using the designated
"ota-downgrade=yes" instead, but keeping the "--override_timestamp" flag
for backward compatibility.

Now both "--override_timestamp" and "--downgrade" allow pushing packages
with downgrade timestamps. The only differenece is that "--downgrade"
enforces a data wiping, because it's intended a real downgrade (based on
Android version or security patch level). "--override_timestamp" serves
the path of a legit "upgrade", but unfortunately with build timestamps
in reverse order (e.g. the two builds are cut from different branches).

With this CL, we write "post-timestamp" to all packages.

Bug: 77811049
Test: Generate an incremental pakcage with "--override_timestamp".
Test: Generate an incremental pakcage with "--downgrade".
Test: python -m unittest test_ota_from_target_files
Change-Id: I4ae4ce77cb600a06ffd9b15b6d4c730314aa0b79
2018-05-09 10:23:14 -07:00
Zhomart Mukhamejanov 29edf01060 Merge "releasetools: Make _GetPropertyFilesString public" am: f6806a7ff3
am: f537477022

Change-Id: I4c243f04baaceab6fc0750533a4d1c18c2531efb
2018-05-04 14:35:19 -07:00
Zhomart Mukhamejanov 603655f5b5 releasetools: Make _GetPropertyFilesString public
ota_from_target_files.py:

Rename PropertyFiles._GetPropertyFilesString to
PropertyFiles.GetPropertyFilesString.

Currently only PropertyFiles.Compute and
PropertyFiles.Finalize gives access to _GetPropertyFilesString.
But Compute force sets reserve_space=True,
and Finalize requires reserved_length.

_GetPropertyFilesString is useful method that can
be used outside releasetools.

One of the usage is in bootable/recovery/updater_sample/.

Test: python -m unittest test_ota_from_target_files
Change-Id: I2cc44ec46a0e68ba071531b003af8cdbfe90e588
Signed-off-by: Zhomart Mukhamejanov <zhomart@google.com>
2018-05-04 12:35:09 -07:00
android-build-prod (mdb) 87a21b5b5b Merge "Revert "Support regenerating partition table with bpttool in sign_target_files_apks"" am: f45bf2d59f
am: 401da96dae

Change-Id: Ibeb9a7bda48533bda5e5c762747bdf14bc791246
2018-04-27 13:59:11 -07:00
Bryan Henry 7febcdf75f Revert "Support regenerating partition table with bpttool in sign_target_files_apks"
This reverts commit 2a40cc6996.

Reason for revert: Broke a bunch of builds, platform/system/tools/bpt must be missing from more manifests.

Change-Id: I4c367a1fec5c62c247544227e9d3b596227e31da
2018-04-27 19:02:51 +00:00
Bryan Henry 7db558c229 Merge changes from topic "sign_target_files_iot-aosp" am: dde5f11e6a
am: f5c481aee9

Change-Id: I036c3f5ca1dd6f6d6900e507de3c5c79656bd1c3
2018-04-27 11:18:00 -07:00
Bryan Henry f130a239a1 Write combined bpt definition directly into META/partition-table.bpt
...instead of copying from IMAGES/partition-table.bpt during
sign_target_files_apks. This addresses a comment on go/oag/665565, but
is done as a separate change so that it can be cherry-picked later (or
not at all) to downstream branches to avoid breaking signing of existing
target-files zips.

Bug: 72837107
Test: Local sign_target_files_apks run of locally built target-files
Change-Id: Id14c859eefe075fc56b15869f2f21c570eb07e65
2018-04-26 12:43:27 -07:00
Bryan Henry 2a40cc6996 Support regenerating partition table with bpttool in sign_target_files_apks
For Android Things targets (or any other target which has
BOARD_BPT_INPUT_FILES defined), add_img_to_target_files will generate a
partition-table.img using bpttool. It also adds the final combined .bpt
definition file into target-files in IMAGES/partition-table.bpt.

When we're signing using sign_target_files_apks, add_img_to_target_files
needs to regenerate the partition table, but META/misc_info.txt still
contains the original list of bpt input files from the build that aren't
available. This change extracts the final bpt from the input
target-files, adds it to META/ in the output target-files, and then
updates the board_bpt_input_files property to point to it.

Bug: 72837107
Test: Local sign_target_files_apks run of locally built target-files
Change-Id: Id79125208f31c78b1ac2079172f9c91a9203849b
2018-04-26 12:42:59 -07:00
Bryan Henry 69d3feb23a Don't suppress stderr for 'avbtool make_vbmeta_image'
Output is useful for debugging.

Bug: 72837107
Test: Local sign_target_files_apks run of locally built target-files
Change-Id: I5c27fcc86fa3a51080e0502eb8f5f01a40b033c3
2018-04-26 12:42:40 -07:00
Tao Bao 133d720ee3 Merge "releasetools: Group the option descriptions in ota_from_target_files.py." am: eafa928737
am: 1456eed8fb

Change-Id: I8273f6e0501cfb5421be4549d26210e1a0b46032
2018-04-23 18:07:23 -07:00
Tao Bao 30df8b4e2e releasetools: Group the option descriptions in ota_from_target_files.py.
This CL separates the options into three groups (excluding the global
options provided via common.py).
 - Non-A/B OTA specific options;
 - A/B OTA specific options;
 - Common options that apply to both.

It mostly reshuffles the lines, with minor change to "--verify" that
removes the obselete remounting behavior (which doesn't apply to
block-based OTA).

Hopefully this makes the expected behavior of some options less
confusing.

Test: `build/make/tools/releasetools/ota_from_target_files.py`
Change-Id: I194ea52c4f7d6a3c7f34531abbcf3fdc7b7f4fa8
2018-04-23 16:00:14 -07:00
Tao Bao 332a96b8b1 releasetools: Default journal size to 0 for system images.
This CL defaults the journal size to 0 for system images (i.e. system,
vendor, system_other, oem, product). We used to do this by explicitly
defining BOARD_{SYSTEM,VENDOR,OEM,PRODUCT}IMAGE_JOURNAL_SIZE to 0 in
device-specific BoardConfig. Targets can still specify a non-zero
journal size as needed.

With this CL, marlin/sailfish gets 32MB / 16MB free space back for
system and vendor respectively.

Bug: 75975085
Test: `m dist`. Check the journal size in the generated images.
Change-Id: Ib3185d07c49be2b4f0fac5fe17ec1a82093ba0c0
(cherry picked from commit 965542fe69)
2018-04-22 22:04:21 -07:00
Patrick Tjin 3f5f9934e9 build_image: default extfs reserved size to 0 on RO partitions.
Defaults the reserved blocks for root to 0% on read only partitions
(system, system_other, vendor, oem).  It also adds support for
explicitly specifying the extfs reserved percentage via
BOARD_{SYSTEM,VENDOR,OEM,PRODUCT}IMAGE_EXTFS_RSV_PCT.

This eventually translates down to the -m option for mkfs.

Removing the reserved space can save at least  5% from the default.

  dumpe2fs system:
    Reserved blocks uid:      0 (user root)
    Reserved blocks gid:      0 (group root)

Bug: 75975085
Test: Build, verify reserved space is changed accordingly
Change-Id: I212d82741908b636db0d658a1c4847bbaadfd5ba
(cherry picked from commit 5ff758799c)
2018-04-22 22:04:21 -07:00
Tao Bao 3f4c246ced Merge changes from topic "space_saver" into pi-dev
am: 9afed6c9ff

Change-Id: I6f6bc337357f9442bc2a8e41882420a90019ec0e
2018-04-20 17:38:15 -07:00
Patrick Tjin 4ea7444789 build_image: default extfs reserved size to 0 on RO partitions.
am: 5ff758799c

Change-Id: I944ed917a69655930fafea3faf2c7529787b840a
2018-04-20 17:37:24 -07:00
TreeHugger Robot 9afed6c9ff Merge changes from topic "space_saver" into pi-dev
* changes:
  releasetools: Default journal size to 0 for system images.
  build_image: default extfs reserved size to 0 on RO partitions.
2018-04-21 00:25:32 +00:00
Tao Bao 684c0605b5 Merge "releasetools: Stop copying images from RADIO/ to IMAGES/." am: f7501cce04
am: d69c274920

Change-Id: Idcc85d6f9335a8cbea8d554bfb90c5bda759c2be
2018-04-20 14:32:11 -07:00
Tao Bao 5277d1015f releasetools: Stop copying images from RADIO/ to IMAGES/.
We've added support in brillo_update_payload that allows additionally
looking for images under RADIO/ in the given target_files zips [1]. This
avoids having duplicate radio images in target_files zips.

Also adjust the unittest in test_ota_from_target_files.py to cover this
path.

As a result of this CL, the radio images will no longer appear in the
image archive (i.e. <target>-img.zip) as well - they are less useful
anyway because we have packed only the _updatable_ pieces that are part
of full bootloader/radio images.

Bug: 77218220
Test: `python -m unittest test_ota_from_target_files`
Test: `python -m unittest test_add_img_to_target_files`
Test: `m dist` produces the same full OTA package
Test: Build marlin-userdebug in internal branch. Check the image zip.
Change-Id: I05579480f0bb9ab90aaeecf75969ee29b6904ad6
2018-04-20 10:20:42 -07:00
Tao Bao bf46541b73 Merge "Remove the support for BRILLO_VENDOR_PARTITIONS." am: c904a89c93
am: cd64a00c03

Change-Id: Ic56244f0e323c7a56214cc10a29a75ea89ae0cbf
2018-04-18 14:51:23 -07:00
Treehugger Robot c904a89c93 Merge "Remove the support for BRILLO_VENDOR_PARTITIONS." 2018-04-18 21:32:35 +00:00
Tao Bao 36d7c5666d Remove the support for BRILLO_VENDOR_PARTITIONS.
It was initially introduced in commit
2e735ca34e, where it packs additional
vendor images into target_files zip in order to generate OTAs. We can
acheive the same goal with INSTALLED_RADIOIMAGE_TARGET, which is the way
being actively used across all targets, including IoT (the former
Brillo) targets.

Bug: 78201540
Test: `m dist` with aosp_marlin-userdebug
Test: Code search shows no active user of BRILLO_VENDOR_PARTITIONS.
Test: `python -m unittest test_add_img_to_target_files`
Change-Id: I8803d5377b5a39304a701cceafb243f9a228347d
2018-04-18 09:50:47 -07:00
Nan Zhang e35161cd7c Merge "Remove make/tools/droiddoc/test/stubs/Android.mk" am: 2fb914c56b
am: 3a966b7247

Change-Id: Ibf7f1edfdaf7dfc112ee119f7e352dd88588aff5
2018-04-16 16:45:52 -07:00
Treehugger Robot 2fb914c56b Merge "Remove make/tools/droiddoc/test/stubs/Android.mk" 2018-04-16 23:30:56 +00:00
Dan Willemsen fcf0b6c861 Merge "Add USER to the docker environment" am: e3bb8e10dc
am: 5aac651374

Change-Id: If83def12536be0452a5c80eec6065cb2a811b45b
2018-04-12 18:36:25 -07:00
Nan Zhang d196ee4019 Remove make/tools/droiddoc/test/stubs/Android.mk
This Android.mk hasn't been changed since the initial-contribution.

And the droiddoc_templatedir:tools/droiddoc/templates-google is gone now.

Test: manual
Bug: b/70351683
Change-Id: I6a364a96c02218cdc012c6350844d562b0e6c142
2018-04-12 17:00:25 -07:00
Dan Willemsen c7f16e4fd4 Add USER to the docker environment
Apparently Jack needs this, and my previous testing didn't show it.

Test: build & run image; echo $USER
Change-Id: I66766b230f2f3e0762a49efc9bf2c6212b2b2c4d
2018-04-12 16:41:17 -07:00
Tom Cherry aa66121df1 Merge "Convert liboemaids to oemaids_headers as a headers only library" into pi-dev 2018-04-05 16:03:45 +00:00
Tom Cherry 87d5583d12 Merge "Convert liboemaids to oemaids_headers as a headers only library" am: 946b5c5e7f
am: 7044082d04

Change-Id: Ia39ebe6bcd691ee31d2438dd279e41917e8278b5
2018-04-04 16:26:13 -07:00
Tom Cherry 1355c4603c Convert liboemaids to oemaids_headers as a headers only library
Bug: 77298768
Test: test that config.fs AIDs are visible through this library
Change-Id: Ifbf3276212ea6904533ac23021bfce29d9a3c5d9
(cherry picked from commit 7a95c15c15)
2018-04-04 16:05:49 -07:00
Tom Cherry 946b5c5e7f Merge "Convert liboemaids to oemaids_headers as a headers only library" 2018-04-04 23:02:34 +00:00
Tom Cherry 7a95c15c15 Convert liboemaids to oemaids_headers as a headers only library
Bug: 77298768
Test: test that config.fs AIDs are visible through this library
Change-Id: Ifbf3276212ea6904533ac23021bfce29d9a3c5d9
2018-04-04 12:29:14 -07:00
Tao Bao 965542fe69 releasetools: Default journal size to 0 for system images.
This CL defaults the journal size to 0 for system images (i.e. system,
vendor, system_other, oem, product). We used to do this by explicitly
defining BOARD_{SYSTEM,VENDOR,OEM,PRODUCT}IMAGE_JOURNAL_SIZE to 0 in
device-specific BoardConfig. Targets can still specify a non-zero
journal size as needed.

With this CL, marlin/sailfish gets 32MB / 16MB free space back for
system and vendor respectively.

Bug: 75975085
Test: `m dist`. Check the journal size in the generated images.
Change-Id: Ib3185d07c49be2b4f0fac5fe17ec1a82093ba0c0
2018-03-31 13:01:07 -07:00
Patrick Tjin 5ff758799c build_image: default extfs reserved size to 0 on RO partitions.
Defaults the reserved blocks for root to 0% on read only partitions
(system, system_other, vendor, oem).  It also adds support for
explicitly specifying the extfs reserved percentage via
BOARD_{SYSTEM,VENDOR,OEM,PRODUCT}IMAGE_EXTFS_RSV_PCT.

This eventually translates down to the -m option for mkfs.

Removing the reserved space can save at least  5% from the default.

  dumpe2fs system:
    Reserved blocks uid:      0 (user root)
    Reserved blocks gid:      0 (group root)

Bug: 75975085
Test: Build, verify reserved space is changed accordingly
Change-Id: I212d82741908b636db0d658a1c4847bbaadfd5ba
2018-03-31 10:22:42 -07:00
Hridya Valsaraju d67d8609bb Allow addition of recovery DTBO to recovery image
Non-A/B devices need to include the DTBO image
within the recovery partition to be self-sufficient
and prevent OTA failures.

Test: Ran 'm dist' and verified that the DTBO image
was included in recovery.img using unpack_bootimg.
Also ran 'make' and verified that the DTBO image was
included in recovery.img using unpack_bootimg.

Bug: 74763691

Change-Id: I38c9c395c95d21f4da42cfa646063bd4416f6bd8
Merged-In: I38c9c395c95d21f4da42cfa646063bd4416f6bd8
(cherry picked from commit e74a38bc6d)
2018-03-29 17:51:57 +00:00
Hridya Valsaraju e474297039 Merge "Allow addition of recovery DTBO to recovery image" am: df2a21794c
am: 9ce04eef0c

Change-Id: I46cfc87494f360149031f0b7b012db24a6a80a1b
2018-03-29 00:34:45 +00:00
Hridya Valsaraju e74a38bc6d Allow addition of recovery DTBO to recovery image
Non-A/B devices need to include the DTBO image
within the recovery partition to be self-sufficient
and prevent OTA failures.

Test: Ran 'm dist' and verified that the DTBO image
was included in recovery.img using unpack_bootimg.
Also ran 'make' and verified that the DTBO image was
included in recovery.img using unpack_bootimg.
Also tested that aosp_angler-userdebug could
boot into recovery(including a fake dtbo.img
in recovery image).

Bug: 74763691

Change-Id: I38c9c395c95d21f4da42cfa646063bd4416f6bd8
2018-03-26 22:54:55 +00:00
Tao Bao cb41f6dea5 Merge "releasetools: Fix an issue in GetMinSdkVersion." am: f18ba63095
am: f3d0d80d81

Change-Id: I09848ccf4daedd77ee62ca48e4bfb7a4c590dba3
2018-03-24 06:52:25 +00:00
Treehugger Robot f18ba63095 Merge "releasetools: Fix an issue in GetMinSdkVersion." 2018-03-23 18:08:40 +00:00
Tao Bao e0bbacf193 Merge "releasetools: Capture and dump stdout/stderr outputs on errors." am: f3e39e5878
am: 49df32de92

Change-Id: I14cd733bfd0bb6bf2ec23b3834e33c1ba41bcc1f
2018-03-23 01:23:44 +00:00
Treehugger Robot f3e39e5878 Merge "releasetools: Capture and dump stdout/stderr outputs on errors." 2018-03-23 01:05:27 +00:00
Tao Bao 701d830acb Merge "Remove the hard-coded path of build_verity_metadata.py." am: 0793683e86
am: 2a8e13f75d

Change-Id: Ifb02ec1a445595f3858cbbbc96155fbe6b6b7e4b
2018-03-22 19:01:44 +00:00
Tao Bao f47bf0fecf releasetools: Fix an issue in GetMinSdkVersion.
The following is a buggy pattern that won't capture anything into err.
The issue is benign, since a failed run would be eventually captured by
a subsequent check.

  p = Run(["aapt", ...], stdout=subprocess.PIPE)
  output, err = p.communicate()
  if err:
    raise ...

This CL changes the error detection to be based on the return code from
aapt. It also adds some sanity test to ensure the call to aapt works.
The test app is built from AOSP com.android.cts.ctsshim (chosen mostly
because of its small size).

Test: python -m unittest test_common
Change-Id: I337f141bd0fc5f0801dfc628c601b88b7640789c
2018-03-22 10:27:04 -07:00
Tao Bao 80921986d1 releasetools: Capture and dump stdout/stderr outputs on errors.
For the functions in common.py, capture the stdout/stderr outputs when
shelling out to external tools. Dump the outputs on errors.

Bug: 76123422
Test: Inject errors to signapk.jar / brotli / unzip. Check the outputs.
Change-Id: Ib2d4272528b83a50bb727788cf8a5211e2ddade8
2018-03-22 00:09:33 -07:00
Tao Bao b4ec6d752a Remove the hard-coded path of build_verity_metadata.py.
build_image.py used to invoke build_verity_metadata.py with a hard-coded
path of 'system/extras/verity/build_verity_metadata.py', which makes it
hard to run unittests from non-$(ANDROID_BUILD_TOP) directory.

This CL adds the dependency on the tool, so that it gets installed to
$(HOST_OUT_EXECUTABLES), then removes the hard-coded path.

Bug: 74544459
Bug: 76015688
Test: `m dist`
Test: python -m unittest test_validate_target_files
Change-Id: I0dcf4eb067a0db6f099cb589eb99a151a05c7f2b
2018-03-21 21:15:59 +00:00
Tao Bao f279b6b66b Merge "releasetools: Handle two edge cases in FinalizeMetadata()." into pi-dev
am: b3e5d54efa

Change-Id: I0b0afd630a26e59ac4e94c1c921a65585aafb536
2018-03-21 20:42:26 +00:00
Treehugger Robot 2ebcf419e4 Merge "releasetools: Handle two edge cases in FinalizeMetadata()." 2018-03-21 20:29:25 +00:00
Tao Bao d2ce2ed5de releasetools: Handle two edge cases in FinalizeMetadata().
In FinalizeMetadata and PropertyFiles, we need to reserve space between
the calls to Compute() and Finalize(). We used to put a 10-byte
placeholder, in the hope of covering the 'offset:length' space for the
metadata entry, as well as the possible value changes in other entries.

However, this could fail in two possible cases: (a) metadata entry
itself has a large offset (e.g. staying near the end of a 1-GiB package,
where the offset itself has 10-digit); or (b) the offsets for other
entries change substantially due to entry reordering. Note that for case
(b), it's space inefficient to always reserve 15-byte for _each_ token
in the property-files.

This CL handles both of these two cases. For (a), we bump up the 10-byte
to 15-byte, which is large enough to cover a package size up to 10-digit
number (i.e. ~9GiB) with a metadata entry size of 4-digit. All these
15-byte will be used for the metadata token alone.

For (b), we add a fallback flow that would retry one more time, but
based on the already signed package that has entries in desired order.

Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate aosp-bullhead full OTA with '--no_signing' flag.
Change-Id: If20487602d2ad09b3797465c01972f2fa792a1f1
(cherry picked from commit 3bf8c65029)
2018-03-21 06:19:28 +00:00
Tao Bao 2d2ce7dd02 Use bsdiff to generate recovery-from-boot.p for system-root-image.
This CL handles a path that uses system-root-image on non-A/B device.
For this path, we can't generate recovery-from-boot patch with imgdiff,
because boot/recovery images contain different number of entries (only
recovery iamge has ramdisk image now).

Using BOARD_USES_FULL_RECOVERY_IMAGE can work around the issue, at the
cost of extra size. Compared to carrying full recovery image, this CL
saves the cost of the kernel size, by putting a patch that's roughly the
size of the recovery ramdisk.

The applypatch executable already detects and handles a bsdiff patch
automatically. No change required to that end.

Note that it won't further reduce the patch size by handling that
ramdisk entry specially, because (a) that's the only difference between
the two images; and (b) there's no corresponding data in boot image to
be diff'd against.

Bug: 72731506
Test: `m dist` with aosp_angler-userdebug. Check the device can install
      recovery image successfully (after intentionally corrupting the
      recovery image).
Test: Build aosp_angler-userdebug with BOARD_BUILD_SYSTEM_ROOT_IMAGE set.
      Verify the generated patch files.
Test: Run validate_target_files.py with the target_files.zips.
Change-Id: I69c06f51ba8c39ae059c5e9a6872a9f10600cf17
Merged-In: I69c06f51ba8c39ae059c5e9a6872a9f10600cf17
(cherry picked from commit 6d5d623987)
2018-03-20 11:38:50 +08:00
Tao Bao 1f95c2bb41 Merge "Use bsdiff to generate recovery-from-boot.p for system-root-image." am: ad29fe419e
am: 40feb95108

Change-Id: I54bf5debc6ffcf8d069211385b55844dcaaca605
2018-03-20 00:27:50 +00:00
Tao Bao 491d7e2ea8 releasetools: Add NonAbOtaPropertyFiles for non-A/B OTA packages.
This CL exposes ota-property-files flag for non-A/B OTA packages.
Currently the line only contains the info for the METADATA entry, for
example "ota-property-files=metadata:69:286". This allows system updater
to just download the METADATA entry, as opposed to downloading the
entire package, to learn about the info regarding the OTA package (e.g.
post-OTA build fingerprint). Note that this requires the OTA server-side
support to pass down the flag along with the update URL.

Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate a non-A/B package and check the property-files string.
Change-Id: I1482c587e18ea7101c8328777ea988c2d8ca06ac
Merged-In: I1482c587e18ea7101c8328777ea988c2d8ca06ac
(cherry picked from commit c0746f4e94)
2018-03-19 14:53:10 -07:00
Tao Bao 6d5d623987 Use bsdiff to generate recovery-from-boot.p for system-root-image.
This CL handles a path that uses system-root-image on non-A/B device.
For this path, we can't generate recovery-from-boot patch with imgdiff,
because boot/recovery images contain different number of entries (only
recovery iamge has ramdisk image now).

Using BOARD_USES_FULL_RECOVERY_IMAGE can work around the issue, at the
cost of extra size. Compared to carrying full recovery image, this CL
saves the cost of the kernel size, by putting a patch that's roughly the
size of the recovery ramdisk.

The applypatch executable already detects and handles a bsdiff patch
automatically. No change required to that end.

Note that it won't further reduce the patch size by handling that
ramdisk entry specially, because (a) that's the only difference between
the two images; and (b) there's no corresponding data in boot image to
be diff'd against.

Bug: 72731506
Test: `m dist` with aosp_angler-userdebug. Check the device can install
      recovery image successfully (after intentionally corrupting the
      recovery image).
Test: Build aosp_angler-userdebug with BOARD_BUILD_SYSTEM_ROOT_IMAGE set.
      Verify the generated patch files.
Test: Run validate_target_files.py with the target_files.zips.
Change-Id: I69c06f51ba8c39ae059c5e9a6872a9f10600cf17
2018-03-19 14:13:02 -07:00
Tao Bao 3bf8c65029 releasetools: Handle two edge cases in FinalizeMetadata().
In FinalizeMetadata and PropertyFiles, we need to reserve space between
the calls to Compute() and Finalize(). We used to put a 10-byte
placeholder, in the hope of covering the 'offset:length' space for the
metadata entry, as well as the possible value changes in other entries.

However, this could fail in two possible cases: (a) metadata entry
itself has a large offset (e.g. staying near the end of a 1-GiB package,
where the offset itself has 10-digit); or (b) the offsets for other
entries change substantially due to entry reordering. Note that for case
(b), it's space inefficient to always reserve 15-byte for _each_ token
in the property-files.

This CL handles both of these two cases. For (a), we bump up the 10-byte
to 15-byte, which is large enough to cover a package size up to 10-digit
number (i.e. ~9GiB) with a metadata entry size of 4-digit. All these
15-byte will be used for the metadata token alone.

For (b), we add a fallback flow that would retry one more time, but
based on the already signed package that has entries in desired order.

Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate aosp-bullhead full OTA with '--no_signing' flag.
Change-Id: If20487602d2ad09b3797465c01972f2fa792a1f1
2018-03-19 13:35:38 -07:00
Tao Bao ad5440d20e Merge "releasetools: Add NonAbOtaPropertyFiles for non-A/B OTA packages." am: 93a84dd9a6
am: f13591bbb3

Change-Id: I43114d7b8ce5900e706e9f234c0fe67cff7cf268
2018-03-19 20:05:28 +00:00
Tao Bao c0746f4e94 releasetools: Add NonAbOtaPropertyFiles for non-A/B OTA packages.
This CL exposes ota-property-files flag for non-A/B OTA packages.
Currently the line only contains the info for the METADATA entry, for
example "ota-property-files=metadata:69:286". This allows system updater
to just download the METADATA entry, as opposed to downloading the
entire package, to learn about the info regarding the OTA package (e.g.
post-OTA build fingerprint). Note that this requires the OTA server-side
support to pass down the flag along with the update URL.

Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate a non-A/B package and check the property-files string.
Change-Id: I1482c587e18ea7101c8328777ea988c2d8ca06ac
2018-03-18 11:59:38 -07:00
Tao Bao 2462a1eb82 releasetools: Add AbOtaPropertyFiles.
am: 85f1698fa0

Change-Id: Ia829f4417659946fcc99b22b66bb392a0a75ae49
2018-03-16 05:08:10 +00:00
Tao Bao 85f1698fa0 releasetools: Add AbOtaPropertyFiles.
AbOtaPropertyFiles writes 'ota-property-files' into package metadata.
Comparing to StreamingPropertyFiles, AbOtaPropertyFiles additionally
writes a token of 'payload-metadata.bin' for a virtual entry that's head
of the 'payload.bin'. This entry can be used by the system updater to
verify if a payload is applicable to the underlying device.

AbOtaPropertyFiles intends to replace StreamingPropertyFiles, as it
covers all the info of the latter. We keep them both in P, and will
remove the latter in later release.

Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate full and incremental A/B OTAs. Check the property-files
      in the METADATA entry.
Change-Id: If5872c8ee0a97a8a95b07d4c69b463aa3fe1b9b9
(cherry picked from commit b63046750c)
2018-03-15 16:23:52 -07:00
Tao Bao f8b2b15a9c Merge "releasetools: Create PropertyFiles class." am: febe7b0736
am: 624f900c01

Change-Id: I58fed3c6e450e3bed7af603d4ad0664618d4d319
2018-03-15 23:05:54 +00:00
Tao Bao b63046750c releasetools: Add AbOtaPropertyFiles.
AbOtaPropertyFiles writes 'ota-property-files' into package metadata.
Comparing to StreamingPropertyFiles, AbOtaPropertyFiles additionally
writes a token of 'payload-metadata.bin' for a virtual entry that's head
of the 'payload.bin'. This entry can be used by the system updater to
verify if a payload is applicable to the underlying device.

AbOtaPropertyFiles intends to replace StreamingPropertyFiles, as it
covers all the info of the latter. We keep them both in P, and will
remove the latter in later release.

Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate full and incremental A/B OTAs. Check the property-files
      in the METADATA entry.
Change-Id: If5872c8ee0a97a8a95b07d4c69b463aa3fe1b9b9
2018-03-15 14:58:31 -07:00
Tao Bao 69203525e4 releasetools: Create PropertyFiles class.
And move StreamingPropertyFiles as its subclass. We will need similar
PropertyFiles instance for non-A/B OTA as well (to expose the
offset/size for the METADATA entry).

Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate an A/B OTA. Check the generated property-files string.
Test: pylint --rcfile=pylintrc \
          ota_from_target_files.py \
          test_ota_from_target_files.py
Change-Id: If90d97f0b330749fd8a6cde2ed9d0d6cd6ea60a8
2018-03-15 14:50:55 -07:00
Tao Bao d3fc38a073 releasetools: Create PropertyFiles class.
And move StreamingPropertyFiles as its subclass. We will need similar
PropertyFiles instance for non-A/B OTA as well (to expose the
offset/size for the METADATA entry).

Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate an A/B OTA. Check the generated property-files string.
Test: pylint --rcfile=pylintrc \
          ota_from_target_files.py \
          test_ota_from_target_files.py
Change-Id: If90d97f0b330749fd8a6cde2ed9d0d6cd6ea60a8
(cherry picked from commit 432f374a1701909ca324f8b047666614684568c0)
2018-03-15 14:17:44 -07:00
Tao Bao 69cc0de37b Merge "releasetools: Support validating Verified Boot images." am: 1a6220f598
am: b5376ee331

Change-Id: I3db23dc63900c3e3bc92f1b02033dfa4ffeceefe
2018-03-12 17:49:01 +00:00
Tao Bao ba557707d8 releasetools: Support validating Verified Boot images.
For a given (signed) target-files.zip, this CLs allows verifying the
Verified Boot related images. It works with both of VB 1.0 and VB 2.0
images.

As part of the CL, it also moves validate_target_files.py to argparse,
which is more flexible than the traditional getopt module.

Also add unittests for the VB 1.0 path. VB 2.0 tests will be added in
follow-up CL.

Example usage:

- Run the script on aosp_bullhead target-files.zip.

$ ./build/make/tools/releasetools/validate_target_files.py \
    --verity_key build/target/product/security/verity.x509.pem \
    --verity_key_mincrypt build/target/product/security/verity_key \
    aosp_bullhead-target_files-4522605.zip

- Run the script on aosp_walleye target-files.zip.

$ ./build/make/tools/releasetools/validate_target_files.py \
    --verity_key external/avb/test/data/testkey_rsa4096.pem \
    aosp_walleye-target_files-4627254.zip

Bug: 63706333
Bug: 65486807
Test: Run validate_target_files.py on target_files.zip files.
Test: PYTHONPATH=build/make/tools/releasetools python -m unittest \
          test_validate_target_files
Change-Id: I170f14d5828d15f3687d8af0a89a816968069057
2018-03-12 00:17:09 -07:00
Tao Bao cc8e2666ac releasetools: Create StreamingPropertyFiles class.
This CL breaks down ComputeStreamingMetadata() into mutiple member
functions of StreamingPropertyFiles class, which correspond to the
two-pass logic when generating streaming property files (aka streaming
metadata).

StreamingPropertyFiles.Compute() does the work for the first pass, by
putting placeholders before doing initial signing. Finalize()
corresponds to the second pass, where the placeholders get replaced with
actual data. Verify() can be optionally called to assert the correctness
of the work.

The separation between Compute() and Finalize() is to allow having
multiple StreamingPropertyFiles instances (in coming up CLs). This way
we can call Compute() multiple times for each instance, followed by only
one call to SignOutput(). And similarly for Finalize().

Bug: 74210298
Test: Generate an A/B OTA package. Check the METADATA entry.
Test: python -m unittest test_ota_from_target_files
Change-Id: I45be0372a4863c4405e6d8e20bcb9ccdc29e7e11
(cherry picked from commit ae5e4c30fe)
2018-03-09 13:44:13 -08:00
Tao Bao a49b0446d7 Merge "releasetools: Create StreamingPropertyFiles class." am: 32dfa4914d
am: 934e04d1a3

Change-Id: I2a1c65d09ffc1f4231cec45874557c3e3af74444
2018-03-09 20:13:30 +00:00
Tao Bao ae5e4c30fe releasetools: Create StreamingPropertyFiles class.
This CL breaks down ComputeStreamingMetadata() into mutiple member
functions of StreamingPropertyFiles class, which correspond to the
two-pass logic when generating streaming property files (aka streaming
metadata).

StreamingPropertyFiles.Compute() does the work for the first pass, by
putting placeholders before doing initial signing. Finalize()
corresponds to the second pass, where the placeholders get replaced with
actual data. Verify() can be optionally called to assert the correctness
of the work.

The separation between Compute() and Finalize() is to allow having
multiple StreamingPropertyFiles instances (in coming up CLs). This way
we can call Compute() multiple times for each instance, followed by only
one call to SignOutput(). And similarly for Finalize().

Bug: 74210298
Test: Generate an A/B OTA package. Check the METADATA entry.
Test: python -m unittest test_ota_from_target_files
Change-Id: I45be0372a4863c4405e6d8e20bcb9ccdc29e7e11
2018-03-08 14:59:42 -08:00
Tao Bao f6f13acf73 releasetools: Fix the broken validate_target_files.py.
The file was broken due to earlier touches:
 - Missing 'import zipfile';
 - Mismatching arguments when calling GetSparseImage().

Bug: 73996151
Test: Run validate_target_files.py with a walleye-target_files.zip.
Test: pylint --rcfile=pylintrc validate_target_files.py
Change-Id: I3692bd51fb27a3da698e06b75155e84502549f66
(cherry picked from commit c63626b4a3)
2018-03-08 08:10:28 -08:00
Baligh Uddin 8c7a2bb360 Merge "releasetools: Fix the broken validate_target_files.py." am: 1617db36af
am: 946752cb24

Change-Id: Id1f96e9c03a2154f707598ab0e9d13df1fe77136
2018-03-08 16:07:48 +00:00
Tao Bao c63626b4a3 releasetools: Fix the broken validate_target_files.py.
The file was broken due to earlier touches:
 - Missing 'import zipfile';
 - Mismatching arguments when calling GetSparseImage().

Bug: 73996151
Test: Run validate_target_files.py with a walleye-target_files.zip.
Test: pylint --rcfile=pylintrc validate_target_files.py
Change-Id: I3692bd51fb27a3da698e06b75155e84502549f66
2018-03-07 21:45:40 -08:00
Tao Bao b9ee298fc6 Merge "releasetools: Separate streaming metadata computation into functions." am: 737bc2b082
am: bfecf9b4cb

Change-Id: I47380d2479460fe9a18ff0b087a3a0a2523dc4c2
2018-03-07 04:40:32 +00:00
Tao Bao fe5b69a4ae releasetools: Separate streaming metadata computation into functions.
And add unittests for ComputeStreamingMetadata().

This prepares for the changes that add additional property-files (for
both of A/B and non-A/B).

Bug: 74210298
Bug: 72751683
Test: python -m unittest test_ota_from_target_files
Test: Generate A/B OTA package. Check the ota-streaming-property-files
      in the METADATA entry.
Change-Id: Ib4b069f61c2c06c035c0cff73a55112f3936b969
(cherry picked from commit f5110498c0)
2018-03-06 20:26:38 -08:00
Tao Bao f5110498c0 releasetools: Separate streaming metadata computation into functions.
And add unittests for ComputeStreamingMetadata().

This prepares for the changes that add additional property-files (for
both of A/B and non-A/B).

Bug: 74210298
Bug: 72751683
Test: python -m unittest test_ota_from_target_files
Test: Generate A/B OTA package. Check the ota-streaming-property-files
      in the METADATA entry.
Change-Id: Ib4b069f61c2c06c035c0cff73a55112f3936b969
2018-03-06 00:06:23 -08:00
Tao Bao 8e9155f236 Merge "releasetools: Fix a mismatching close." am: 395bf274b8 am: 06bc45b0c2
am: 6b93ed0c84

Change-Id: I74f9e797510bcecdb1f93824ce5881f92ff3b152
2018-03-03 00:10:31 +00:00
Tao Bao 6b93ed0c84 Merge "releasetools: Fix a mismatching close." am: 395bf274b8
am: 06bc45b0c2

Change-Id: I5e9a7d4a74e476b1ed86a9f7bc06e515998f81e2
2018-03-02 23:59:22 +00:00
Tao Bao 395bf274b8 Merge "releasetools: Fix a mismatching close." 2018-03-02 23:32:11 +00:00
Tao Bao 921c81f7c3 Merge "releasetools: Call Cleanup() in check_target_files_signatures.py." am: 903be914bb am: 5410d8a52a
am: 501b84236f

Change-Id: Ia4dfee212e3ff55fcdd9ce60faddab91c5e195c6
2018-03-02 23:05:20 +00:00
Tao Bao 501b84236f Merge "releasetools: Call Cleanup() in check_target_files_signatures.py." am: 903be914bb
am: 5410d8a52a

Change-Id: I3d015a9ffa1784c060d6a1d878feae9046090b5d
2018-03-02 22:55:11 +00:00
Tao Bao a652c00c8e releasetools: Fix a mismatching close.
There's a mismatch in WriteABOTAPackageWithBrilloScript().

  temp_zip_file = tempfile.NamedTemporaryFile()
  ...
  common.ZipClose(temp_zip_file)

It's benign since common.ZipClose() happens to be calling
"temp_zip_file.close()". This CL moves the use of tempfile to
common.MakeTempFile(), so that the tempfile will be cleaned up
automatically as part of the call to common.Cleanup(). (Not fixing the
close() directly, since the nearby lines will be refactored into another
function shortly.)

Also remove one assert in the same function, which trivially holds in
the current code.

Test: Generate an A/B OTA.
Change-Id: I53b375d1150820de718dec0ead55abf5f4951071
2018-03-02 13:20:43 -08:00
Tao Bao 150708b012 Merge "releasetools: Don't return ZipFile from common.UnzipTemp()." am: 5c93279fbe am: 0db979c6be
am: abe02acc48

Change-Id: Ie8d1d55d134a7aade1be643634641ef0eb80ff65
2018-03-01 22:44:21 +00:00
Tao Bao abe02acc48 Merge "releasetools: Don't return ZipFile from common.UnzipTemp()." am: 5c93279fbe
am: 0db979c6be

Change-Id: I9a4f2bc04851f048e168a11c039d781c2069b662
2018-03-01 22:38:46 +00:00
Tao Bao 767543a659 releasetools: Call Cleanup() in check_target_files_signatures.py.
Also minor cleanups to make it pylint clean.

Test: Run check_target_files_signatures.py with a target-files.zip.
Test: pylint --rcfile=pylintrc check_target_files_signatures.py
Change-Id: Ife3b54c7805c2f2562e87e91ab4b4de355782012
2018-03-01 10:12:39 -08:00
Tao Bao dba59eeabd releasetools: Don't return ZipFile from common.UnzipTemp().
In addition to the unzipping work, common.UnzipTemp() kindly bundles an
open ZipFile object as part of the return value. It doesn't look very
helpful to the callers though. It also looks less obvious that the
caller needs to properly close the handle (missing the close here is
benign though). This CL just removes the ZipFile object out of the
return value, and leaves the work to callers.

Test: `m dist` on both of A/B and non-A/B target.
Test: python -m unittest test_add_img_to_target_files
Test: python -m unittest test_common
Test: python -m unittest test_ota_from_target_files
Test: Check the callers to common.UnzipTemp() in code search.
Change-Id: Id47da3fd42a0e76d6ae8851f05780db319ee48cf
2018-03-01 10:03:31 -08:00
Tao Bao 7a08313a5e Merge "releasetools: Move one case of mkdtemp() to common.MakeTempDir()." am: a7a5e15f72 am: fc0bb788ed
am: e9f3ee0f80

Change-Id: Ie5cf8533ae02aab3f52c3652b2afc55c729e16f2
2018-02-28 21:37:09 +00:00
Tao Bao e9f3ee0f80 Merge "releasetools: Move one case of mkdtemp() to common.MakeTempDir()." am: a7a5e15f72
am: fc0bb788ed

Change-Id: Icb0ffb498e068417bdadabd4877731c63b776f37
2018-02-28 21:30:30 +00:00
Tao Bao 04bce3a696 releasetools: Move one case of mkdtemp() to common.MakeTempDir().
This should be the last case to be moved over.

Test: Generate an incremental BBOTA (which exercises the changed code).
Test: `rgrep mkdtemp` gives no more instance.
Change-Id: I76db069476201cdfaf3a2de9d9635dfe54507f7a
2018-02-28 11:13:29 -08:00
Tao Bao 9a90d8cae6 Merge "releasetools: Remove the unneeded 'prefix' parameters." am: 6e2a8ae094 am: 945e9c3e44
am: 5d74ac8574

Change-Id: I84554693ebd6be87eb0989d9b2d1df5f049c62f7
2018-02-27 22:36:18 +00:00
Tao Bao 5d74ac8574 Merge "releasetools: Remove the unneeded 'prefix' parameters." am: 6e2a8ae094
am: 945e9c3e44

Change-Id: Iee385b3b5d3a8f3848ccbfdd3773b7cf287f40cf
2018-02-27 22:31:24 +00:00
Tao Bao 6e2a8ae094 Merge "releasetools: Remove the unneeded 'prefix' parameters." 2018-02-27 22:13:06 +00:00
Tao Bao 886d883d61 releasetools: Remove the unneeded 'prefix' parameters.
... from the following functions in add_img_to_target_files.py.

  AddSystem
  AddSystemOther
  AddVendor
  AddProduct
  AddDtbo
  AddUserdata
  AddVBMeta
  AddPartitionTable
  AddCache

The last user of the parameter in img_from_target_files.py has been
removed in commit 2bb109709a (in O).

Test: pylint --rcfile=pylintrc add_img_to_target_files.py
Test: Check all the callers to the above functions.
Test: m dist
Change-Id: I551d1683def8f8535062fc90f68dafa0f4252822
2018-02-27 11:47:11 -08:00
Luis Hector Chavez 0fae5d0806 Merge "Make mksquashfs generate a fs_config-friendly root entry" am: 3d4fb4e335 am: 6128d7f30d
am: 21025515e4

Change-Id: I710cbaa8379c118e0c6a519f5d737a6395de2646
2018-02-27 16:41:46 +00:00
Luis Hector Chavez 21025515e4 Merge "Make mksquashfs generate a fs_config-friendly root entry" am: 3d4fb4e335
am: 6128d7f30d

Change-Id: I43a62ea7c4d6acb5adadfc488b94e05024499320
2018-02-27 16:36:39 +00:00
Treehugger Robot 3d4fb4e335 Merge "Make mksquashfs generate a fs_config-friendly root entry" 2018-02-27 16:24:21 +00:00
Dan Shi 9801d9b7b2 Merge "Default test to AndroidJUnitTest for auto-generated test config." am: ce1fa25d1b am: d91129eea9
am: 6dbd9e3df9

Change-Id: I84ee22fa20b08fb3db27d5e5d0e03f097a11463f
2018-02-26 23:26:06 +00:00
Dan Shi 6dbd9e3df9 Merge "Default test to AndroidJUnitTest for auto-generated test config." am: ce1fa25d1b
am: d91129eea9

Change-Id: Ib0e817e8d403ea130516dbc94201fd4a899c327f
2018-02-26 23:18:57 +00:00
Treehugger Robot ce1fa25d1b Merge "Default test to AndroidJUnitTest for auto-generated test config." 2018-02-26 23:00:28 +00:00
Tao Bao 47db251db0 Merge "releasetools: Remove the parsing of META/imagesizes.txt." am: 3aee4dfac1 am: 72f3c44243
am: ed98b94326

Change-Id: Id4f84a66faf7c06b4bbacae799d36db4a498182b
2018-02-26 22:46:52 +00:00
Tao Bao ed98b94326 Merge "releasetools: Remove the parsing of META/imagesizes.txt." am: 3aee4dfac1
am: 72f3c44243

Change-Id: I83af79b6d5a76099259a951585d92755b132af9e
2018-02-26 22:39:14 +00:00
Tao Bao 0835594a6f Merge "releasetools: Add unittests for GetCareMap() and AddCareMapTxtForAbOta()." am: 7998503ebe am: 5fed709d60
am: b8d0b5bd06

Change-Id: Iec88f7bdbf26047479d217d756d4847b1bac64f7
2018-02-26 22:32:18 +00:00
Tao Bao b8d0b5bd06 Merge "releasetools: Add unittests for GetCareMap() and AddCareMapTxtForAbOta()." am: 7998503ebe
am: 5fed709d60

Change-Id: I117d452b1a4d63b7665c3593b5b2167fe57be954
2018-02-26 22:24:13 +00:00
Treehugger Robot 3aee4dfac1 Merge "releasetools: Remove the parsing of META/imagesizes.txt." 2018-02-26 22:18:31 +00:00
Tao Bao 7998503ebe Merge "releasetools: Add unittests for GetCareMap() and AddCareMapTxtForAbOta()." 2018-02-26 22:06:32 +00:00
Tao Bao 41fb7d27cf releasetools: Remove the parsing of META/imagesizes.txt.
The file has been removed from target-files.zip since commit
c19a8d5590 (Gingerbread), whose info has
been consolidated into META/misc_info.txt.

Test: `m dist`
Change-Id: Ic144457954f5742ea082dcd9ffbea71df4afe46e
2018-02-23 12:06:07 -08:00
Tao Bao cf9c445ef9 Merge "releasetools: Allow skipping postinstall hooks when generating A/B OTAs." am: 6c0ddf2480 am: ff1288821d
am: a5baf56e6c

Change-Id: I59bf83ba415351ba07488086d8b812453b0ae61b
2018-02-22 20:49:01 +00:00
Tao Bao a5baf56e6c Merge "releasetools: Allow skipping postinstall hooks when generating A/B OTAs." am: 6c0ddf2480
am: ff1288821d

Change-Id: Iea818eb91dd5ff98d9b378a845ff507e1dee1e0c
2018-02-22 20:40:25 +00:00
Tao Bao 6c0ddf2480 Merge "releasetools: Allow skipping postinstall hooks when generating A/B OTAs." 2018-02-22 20:21:36 +00:00
Tao Bao 567fa9012b Merge "releasetools: Check for duplicate entries in ReplaceCerts()." am: 8bfde7c5fb am: 7032691417
am: 894cba2954

Change-Id: Ic89d331846654d23768e90cb4abab1b91d744271
2018-02-22 18:05:39 +00:00
Tao Bao 894cba2954 Merge "releasetools: Check for duplicate entries in ReplaceCerts()." am: 8bfde7c5fb
am: 7032691417

Change-Id: Icc23606930f1a2f3e0009b1ec225723ffc8a7617
2018-02-22 17:57:03 +00:00
Tao Bao 8bfde7c5fb Merge "releasetools: Check for duplicate entries in ReplaceCerts()." 2018-02-22 17:39:00 +00:00
Tao Bao 63c18fead8 releasetools: Add unittests for GetCareMap() and AddCareMapTxtForAbOta().
With test_utils.construct_sparse_image(), creating valid sparse image is
no longer a blocker for testing these functions.

Test: python -m unittest test_add_img_to_target_files
Change-Id: Iff1f15837cc744bad52e2120f9a9ad94d7db50d5
2018-02-22 00:10:05 -08:00
Tao Bao 15a146a7c4 releasetools: Allow skipping postinstall hooks when generating A/B OTAs.
This CL adds a new flag '--skip_postinstall' that allows skipping all
the postinstall hooks when generating an A/B OTA package (default:
False). Note that this discards ALL the hooks, including non-optional
ones. Should only be used if caller knows it's safe to do so (e.g. all
the postinstall work is to dexopt apps and a data wipe will happen
immediately after).

Bug: 73547992
Test: python -m unittest test_ota_from_target_files
Test: Generate a full OTA package for walleye. Examine the generated
      payload.
Change-Id: Ifc069e897b4019605051eabfd221230a6a37867c
2018-02-21 16:34:19 -08:00
Dan Shi 96068b77ed Default test to AndroidJUnitTest for auto-generated test config.
All new instrumentation tests shall use AndroidJUnitRunner, thus default
test class to AndroidJUnitTest.

Bug: 73679956
Test: unittest
Change-Id: I52df4d01b8459a4a6e5c2cbe15973dab079fb136
2018-02-21 11:31:06 -08:00
Tao Bao 039ccf14ca Merge "releasetools: Clean up test_sign_target_files_apks.py." am: 5ef78033d5 am: e1fee81bb5
am: 4e636f0441

Change-Id: I9133283350bd0d9c74d6d2b9b313a5fd1103aa48
2018-02-21 02:36:43 +00:00
Tao Bao 4e636f0441 Merge "releasetools: Clean up test_sign_target_files_apks.py." am: 5ef78033d5
am: e1fee81bb5

Change-Id: I3c504df0a047b88725433c68079c18495d536afe
2018-02-21 02:29:12 +00:00
Tao Bao 5ef78033d5 Merge "releasetools: Clean up test_sign_target_files_apks.py." 2018-02-21 02:06:00 +00:00
Tao Bao 90c3d2ebb5 Merge "releasetools: Add a testcase for common.ParseCertificate()." am: a0f55ee9eb am: 9773a2f7b2
am: 6f142ef096

Change-Id: Ic838ebf4e59d8d90eb102c0ae4399b639fcf64f6
2018-02-21 00:48:47 +00:00
Tao Bao 6f142ef096 Merge "releasetools: Add a testcase for common.ParseCertificate()." am: a0f55ee9eb
am: 9773a2f7b2

Change-Id: I3b00809f57c7f42f69fdbb44cdc5516e585ee987
2018-02-21 00:28:14 +00:00
Tao Bao a0f55ee9eb Merge "releasetools: Add a testcase for common.ParseCertificate()." 2018-02-21 00:04:24 +00:00
Tao Bao 4051a05b54 Merge "releasetools: Add unittest for common.GetSparseImage()." am: 35ee189898 am: 625b5b7a09
am: 0fa48c8c52

Change-Id: I12997ace8bfc5936a6db773b570c7ec2e26edb47
2018-02-20 21:27:34 +00:00
Tao Bao 0fa48c8c52 Merge "releasetools: Add unittest for common.GetSparseImage()." am: 35ee189898
am: 625b5b7a09

Change-Id: I494f253ab122f4098ec1e1fd814c915e618e2aa2
2018-02-20 21:21:31 +00:00
Treehugger Robot 35ee189898 Merge "releasetools: Add unittest for common.GetSparseImage()." 2018-02-20 21:04:58 +00:00
Tao Bao 66472637ad releasetools: Check for duplicate entries in ReplaceCerts().
testdata/{media,platform}.x509.pem files are copied from
build/target/product/security/.

Fixes: 69479366
Test: python -m unittest test_sign_target_files_apks
Change-Id: I8ba42b6f5b5432ee4b8786b241daff11db792c14
2018-02-20 10:22:30 -08:00
Tao Bao de1d479ace releasetools: Clean up test_sign_target_files_apks.py.
Since we have been carrying test certificates in testdata/ for other
tests, do the same for test_sign_target_files_apks.py. Copy
verity.x509.pem from build/target/product/security/ to testdata/ for
that purpose.

Also capture the stderr output in ReplaceVerityKeyId().

Test: python -m unittest test_sign_target_files_apks
Change-Id: Ie11e042086952e8a4a5a63950cb0b16cc436b7e6
2018-02-20 10:12:15 -08:00
Dan Willemsen dcdc1685c7 Merge "Add Docker instructions for older branches" am: c6620457d8 am: cfc82c4b33
am: aa582f7dda

Change-Id: I84b2706e169ecb4835cd1dd8193576a6001e750a
2018-02-17 01:27:40 +00:00
Dan Willemsen aa582f7dda Merge "Add Docker instructions for older branches" am: c6620457d8
am: cfc82c4b33

Change-Id: Ia6404fa198f502ad9be811a83ca19a7fee31bb17
2018-02-17 01:19:32 +00:00
Tao Bao 17e4e61691 releasetools: Add a testcase for common.ParseCertificate().
Also minor clean up to the function, to suppress the following lint
warning.
R:1813, 2: Redefinition of cert type from list to unicode (redefined-variable-type)

Test: python -m unittest -v test_common.CommonApkUtilsTest
Change-Id: Ib4b1dfa8b19e505fc70684b648efc36171c73bbf
2018-02-16 17:14:23 -08:00
Dan Willemsen 6dc4621afc Add Docker instructions for older branches
If you've upgraded away from Ubuntu 14.04 Trusty, you may find that
older branches (or certain code) doesn't build anymore. These
instructions can form a starting point to build these older branches
within Docker.

Bug: 72993575
Test: Follow instructions, build marshmallow-dev
Test: Follow instructions, build lollipop-mr1-dev
Change-Id: If4047b1bb5324d75a9d0947cb5280ff1cabccb6a
2018-02-15 18:17:46 -08:00
Tao Bao fc7e0e06b6 releasetools: Add unittest for common.GetSparseImage().
Add construct_sparse_image() to test_utils.py, which is a util function
to create sparse images. The new tests also partially cover the recent
changes that add 'incomplete' and 'uses_shard_blocks' tags.

Test: python -m unittest test_common
Change-Id: Ia15f5c4ad12423691216ebbad2c28f95c8427d7e
2018-02-14 15:12:52 -08:00
TreeHugger Robot 221d1f4bde Merge "releasetools: Specify max_timestamp for secondary payload." into oc-mr1-dev-plus-aosp 2018-02-14 00:26:57 +00:00
Tao Bao 9d0242d93e Merge "releasetools: Allow generating BBOTA for images with shared blocks." am: 9452d84b97 am: 6c453193a1
am: 7088b88a17

Change-Id: I123238db80af7d60e92da4aef0478f9f5f1a29a5
2018-02-13 23:13:15 +00:00
Tao Bao 072f2e4784 Merge "releasetools: Remove the unconditional fallback to bsdiff." am: 6392e05813 am: 61fc7f6746
am: 6b2854d44d

Change-Id: Iac4c743329de3db006ed4a2d3782835a5002490b
2018-02-13 23:10:28 +00:00
Tao Bao 7088b88a17 Merge "releasetools: Allow generating BBOTA for images with shared blocks." am: 9452d84b97
am: 6c453193a1

Change-Id: I422f6da0c524fa5329bc9dd11eb10eb27945ab92
2018-02-13 22:43:10 +00:00
Tao Bao 6b2854d44d Merge "releasetools: Remove the unconditional fallback to bsdiff." am: 6392e05813
am: 61fc7f6746

Change-Id: I3729f15bb532838ce2b9ec3f084ee5b81e97d3b1
2018-02-13 22:35:42 +00:00
Tao Bao db1fe41bbe releasetools: Specify max_timestamp for secondary payload.
This is a mandatory field in the generated payload.

Bug: 35724498
Test: Generate OTA with --include_secondary. Check the generation
      command for secondary payload.
Change-Id: Ib2e25cc020e294eae5a4be36d6dca297432804b8
Merged-In: Ib2e25cc020e294eae5a4be36d6dca297432804b8
(cherry picked from commit 6048121485)
2018-02-13 14:34:55 -08:00
Tao Bao 6048121485 releasetools: Specify max_timestamp for secondary payload.
This is a mandatory field in the generated payload.

Bug: 35724498
Test: Generate OTA with --include_secondary. Check the generation
      command for secondary payload.
Change-Id: Ib2e25cc020e294eae5a4be36d6dca297432804b8
2018-02-13 14:17:17 -08:00
Tao Bao d1239cd452 Merge "releasetools: Specify SWITCH_SLOT_ON_REBOOT for secondary payload." am: 4f0b725439 am: 78eef74e03
am: 55e571f0b6

Change-Id: I0bb607e9ad51dd8624493a82c3d5c2b2167abd43
2018-02-13 22:12:24 +00:00
Tao Bao 55e571f0b6 Merge "releasetools: Specify SWITCH_SLOT_ON_REBOOT for secondary payload." am: 4f0b725439
am: 78eef74e03

Change-Id: I02007c63b6a7398afe8b5304e48c01589842a876
2018-02-13 22:06:40 +00:00
Tao Bao 9452d84b97 Merge "releasetools: Allow generating BBOTA for images with shared blocks." 2018-02-13 21:53:25 +00:00
Tao Bao 6392e05813 Merge "releasetools: Remove the unconditional fallback to bsdiff." 2018-02-13 21:53:17 +00:00
Tao Bao 4f0b725439 Merge "releasetools: Specify SWITCH_SLOT_ON_REBOOT for secondary payload." 2018-02-13 20:20:30 +00:00