Commit Graph

111 Commits

Author SHA1 Message Date
Tao Bao ea4325baf8 Revert "releasetools: Generate streamable A/B OTA packages."
This reverts commit ef1bb4360f.

It has broken the builds that don't have care_map.txt entry.

Change-Id: I343455e66be5e79457968dfc2813c7c1c234b6dc
2017-01-25 01:41:13 +00:00
Tao Bao ef1bb4360f releasetools: Generate streamable A/B OTA packages.
In order to support streaming A/B OTA packages, we pack
payload_properties.txt and care_map.txt in ZIP_STORED mode. These two
entries along with payload.bin (already in ZIP_STORED prior to this CL)
can be fetched directly based on the offset and length info.

We write the offset and length info into the package metadata entry
(META-INF/com/android/metadata), which can be parsed by the OTA server.

payload_properties.txt and care_map.txt are usually less than 1-KiB. So
the change only incurs marginal size increase.

Bug: 33382114
Test: Generate an A/B OTA package. Verify the 'streaming-property-files'
      entry in the metadata file.

Change-Id: I04504e834eb36e18876c5f5a5a09289ee05c6f9a
2017-01-19 22:25:12 -08:00
Tao Bao 4a0b494eea Merge "releasetools: Replace print stmt with print()." 2017-01-14 15:36:41 +00:00
Tao Bao 89fbb0f6d5 releasetools: Replace print stmt with print().
So that it's compatible with Python 3.

Test: pylint --pylint=pylintrc

Change-Id: If06c135a492c94bedd713c8cbdf03155a502d5cd
2017-01-13 14:55:14 -08:00
Tao Bao 2a0d1da4d9 releasetools: Delete the obsolete --aslr_mode/-a flag.
It was added in commit 96be7205dc
("Working ASLR implementation.") in 2010, and removed in commit
1807e700a5 ("don't generate retouch
commands in OTA scripts") in 2012.

Remove the obsolete --aslr_mode flag.

Test: ota_from_target_files.py still works (by generating incremental
      and full OTAs respectively).

Change-Id: I6d8e62730ac192f3574d484c4a4b9b43b4ee0a9e
2017-01-13 22:46:27 +00:00
Tao Bao f9023856ad releasetools: Add prefix when dumping fingerprints.
We used to dump "Source: <fingerprint>" in update logs. The "Source: "
prefix was unintentionally dropped out.

Test: Check the generated incremental BBOTA script.
Change-Id: I4de62333aa38e3fb09a76df0e769b62af48e0313
2016-12-14 11:53:38 -08:00
Tao Bao d07f9995b8 Merge "Build recovery-two-step.img for two-step OTAs."
am: 06e14e438f

Change-Id: I9c5d6e5f71da88bc0f662373335b024a55ca193e
2016-12-02 15:19:00 +00:00
Tao Bao d42e97ebb4 Build recovery-two-step.img for two-step OTAs.
In two-step OTAs, we write recovery image to /boot as the first step so
that we can reboot from there and install a new recovery image to
/recovery. However, bootloader will show "Your device is corrupt"
message when booting /boot with the recovery image. Because the recovery
image encodes the path of "/recovery" as part of the signature metadata,
which fails the verified boot.

This CL generates a special "recovery-two-step.img" in addition to the
regular recovery.img. This image encodes "/boot" when being signed,
which will be flashed to /boot at stage 1/3 in a two-step OTA.

Here are the desired changes:

- 'IMAGES/recovery-two-step.img' exists in target_files.zip for non-A/B
targets (e.g. bullhead). The image should not exist for targets that
don't have a recovery partition (e.g. A/B devices like sailfish).

- <device>-img.zip should not contain 'recovery-two-step.img'.

- Nothing should change when building non-two-step OTAs. For two-step
OTAs, 'recovery-two-step.img' should be included in the OTA package;
'updater-script' should flash this image to /boot at stage 1/3.

- When building a two-step OTA with an input TF.zip that doesn't have
  IMAGES/recovery-two-step.img, it should use the existing
  IMAGES/recovery.img instead.

Bug: 32986477
Test: Tested the steps above on bullhead and sailfish.
Change-Id: I34e6c599bcf2011d4cd5c926999418b3975d6d0f
2016-12-01 17:47:59 -08:00
Tao Bao 9242e1f750 Merge "releasetools: compare size between compressed and patch while build otapackage"
am: 49758959f2

Change-Id: I17b988878cc7ffdfbe490f404c9cd5ab996a8c5f
2016-11-05 00:13:55 +00:00
YOUNG HO CHA ccc5c40523 releasetools: compare size between compressed and patch while build otapackage
Currently, whether contains patch or verbatim, compute with file size
and patch size.
But ota file must be compressed with zip, so it should be better with
compressed size than uncompressed.

Test: aosp_shamu-user build without proprietary blobs between MOB30P and NRD90S
$ du -k ota_shamu_old.zip ota_shamu_new.zip
217252	ota_shamu_old.zip
216520	ota_shamu_new.zip

Change-Id: If68cb1fbe2f7815067451915a0dcfe93ea5ba8d6
Signed-off-by: YOUNG HO CHA <ganadist@gmail.com>
2016-10-18 17:27:09 +09:00
Tianjie Xu 737afb98c9 resolve merge conflicts of 333df6c to nyc-mr1-dev-plus-aosp
Change-Id: Ibe66a7c78073a9c3e9f139bd6b00a0afab075190
2016-07-11 11:42:53 -07:00
Tao Bao 538ed261eb resolve merge conflicts of 4058a71 to stage-aosp-master
Change-Id: I0816bfc5d8adb20e175617d97651f1474ce4e916
2016-07-11 09:08:24 -07:00
Tao Bao 62a6c464a2 releasetools: Fix the detection of using squashfs.
We should disable using imgdiff if *any* of the source and target
partitions uses squashfs.

Bug: 30004734
Test: Create an incremental with two builds with one of them uses squashfs.
Change-Id: I826cd13d7b852c548e4b45e61f5ae00f6407cac3
(cherry picked from commit f8acad1480)
2016-07-10 23:16:14 -07:00
Tao Bao b6568cd4be releasetools: Disable using imgdiff for squashfs.
We use imgdiff to handle files in zip format (e.g. jar/zip/apk) for
higher compression ratio.

For system/vendor in squashfs, a) all files are compressed in LZ4
format; b) we use 4096-byte block size in their sparse images, but the
files in squashfs may not be laid out as 4K-aligned. So the blocks for
a given file as listed in block map may not form a valid zip file, which
may fail the patch generation with imgdiff.

Disable using imgdiff for squashfs images, and use bsdiff instead.

Bug: 22322817
Change-Id: Ie76aa4cece5c9d38cb1d1a34c505a4a8f37512d3
(cherry picked from commit 293fd135c7)
2016-07-10 23:15:48 -07:00
Tianjie Xu cfa86223d6 Add care_map to target_files package
Generate a new file containing care_data of system (and vendor)
partition, and add it under META/ of target file package. For
A/B update, copy this file to OTA package for later use by
update_verifier.

Bug: 27175949

Change-Id: I90bb972703afaeb94bc3efe718fd81b1cfbcabcc
2016-07-08 16:53:59 -07:00
Tao Bao ae974f289b releasetools: Fix the detection of using squashfs.
am: f8acad1480

Change-Id: I654730ef8566d12ad17a81261b450a695b79686d
2016-07-07 18:24:12 +00:00
Tao Bao f8acad1480 releasetools: Fix the detection of using squashfs.
We should disable using imgdiff if *any* of the source and target
partitions uses squashfs.

Bug: 30004734
Test: Create an incremental with two builds with one of them uses squashfs.
Change-Id: I826cd13d7b852c548e4b45e61f5ae00f6407cac3
2016-07-07 09:10:25 -07:00
Tao Bao c8338f1c35 Merge \\"Add ability to pass in payload_signer args\\" am: 8e6582b238
am: 8f9df9efb9

Change-Id: I3603ac55a8836406ca60c05959efffffb7ffca3a
2016-06-24 05:30:31 +00:00
Tao Bao 63cc08f93d Merge \\"releasetools: Support using payload_signer.\\" am: 983a42bdf1
am: 9322317ee7

Change-Id: I13c5eede68149c1dda6da9bb33a2527adc05dd8c
2016-06-24 05:30:31 +00:00
Baligh Uddin 8f9df9efb9 Merge \"Add ability to pass in payload_signer args\"
am: 8e6582b238

Change-Id: I83ab46e365fd09c7bf3d4a97fddf289e5d819ce0
2016-06-24 05:27:02 +00:00
Tao Bao 9322317ee7 Merge \"releasetools: Support using payload_signer.\"
am: 983a42bdf1

Change-Id: I027ae0f71c3a57ffad476791721b111605844f1d
2016-06-24 05:27:02 +00:00
Baligh Uddin 34eb000de5 Add ability to pass in payload_signer args
Bug: 28701652
Change-Id: I110d5fc14446e4a6a0f8e25dcb0d300decdf09a4
(cherry picked from commit 2abbbd0333)
2016-06-23 22:16:46 -07:00
Tao Bao 1a5e1d18a7 releasetools: Support using payload_signer.
For A/B OTAs, by default it calls 'openssl pkeyutl' to sign the payload
and metadata with the package private key. If the private key cannot be
accessed directly, a payload signer that knows how to do that should be
supplied via "--payload_signer <signer>".

The signer will be called with "-inkey <path_to_private_key>",
"-in <input_file>" and "-out <output_file>" parameters.

Test: Use a dummy signer, call 'ota_from_target_files.py --payload_signer <signer> <target_files.zip> <ota.zip>' and verify the signatures in the generated package.
Bug: 28701652
Change-Id: I26cfdd3fdba6fc90799221741b75426988e46fd3
(cherry picked from commit dea0f8bfed)
2016-06-23 22:15:30 -07:00
Tao Bao b79e31756b resolve merge conflicts of 2abbbd0 to nyc-mr1-dev-plus-aosp
Change-Id: I2d06f0cec793185533ef2fc78428dfd99815030c
2016-06-23 19:48:40 -07:00
Baligh Uddin 2abbbd0333 Add ability to pass in payload_signer args
Bug: 28701652
Change-Id: I110d5fc14446e4a6a0f8e25dcb0d300decdf09a4
2016-06-23 10:34:51 -07:00
Tao Bao 6047c24fae resolve merge conflicts of 9cbe37e to nyc-mr1-dev-plus-aosp
Change-Id: I9643386f714fe520e5732d0e6dbe821131ea6d02
2016-06-21 13:37:26 -07:00
Tao Bao dea0f8bfed releasetools: Support using payload_signer.
For A/B OTAs, by default it calls 'openssl pkeyutl' to sign the payload
and metadata with the package private key. If the private key cannot be
accessed directly, a payload signer that knows how to do that should be
supplied via "--payload_signer <signer>".

The signer will be called with "-inkey <path_to_private_key>",
"-in <input_file>" and "-out <output_file>" parameters.

Test: Use a dummy signer, call 'ota_from_target_files.py --payload_signer <signer> <target_files.zip> <ota.zip>' and verify the signatures in the generated package.
Bug: 28701652
Change-Id: I26cfdd3fdba6fc90799221741b75426988e46fd3
2016-06-20 20:32:37 -07:00
Elliott Hughes 38f9a845ca Merge \\"Remove --no_prereq flag from OTA script.\\" am: bc450fc9c8
am: bd30d5fa16

Change-Id: Ic083c9b038bd5a31f9b52ebd6fc498e0353239ca
2016-06-21 00:08:23 +00:00
Elliott Hughes bd30d5fa16 Merge \"Remove --no_prereq flag from OTA script.\"
am: bc450fc9c8

Change-Id: I35a4400afc829bf2b269135fa99f00979affd3bd
2016-06-21 00:05:51 +00:00
Elliott Hughes d8a52f9785 Remove --no_prereq flag from OTA script.
Bug: http://b/29393071
Test: aosp_flounder "make dist"
Change-Id: I9ec85210e118f7e525291e31ab4081a2bd10f998
2016-06-20 14:44:18 -07:00
Tao Bao 7e5205f929 Merge \\"Support wiping userdata for A/B OTA packages.\\" am: b727d5bb32
am: 4f2b227fae

Change-Id: Icf157b44ecda14a2c289d4826e5a55af67e805d5
2016-06-16 15:53:20 +00:00
Tao Bao 4f2b227fae Merge \"Support wiping userdata for A/B OTA packages.\"
am: b727d5bb32

Change-Id: I9d6e67c5e71b025f7cb9119b45f05c51f2cfd340
2016-06-16 15:50:47 +00:00
Tao Bao 7c5dc578b3 Support wiping userdata for A/B OTA packages.
update_engine now accepts POWERWASH=1 to schedule a factory reset in
the post-install phase. Hook up with the --wipe_user_data flag in the
OTA script.

Bug: 28700985
Change-Id: Ie73876a61db90d124d2af588d674757376e9aabc
(cherry picked from commit 38ca0be399)
2016-06-15 23:19:52 -07:00
Tao Bao 38ca0be399 Support wiping userdata for A/B OTA packages.
update_engine now accepts POWERWASH=1 to schedule a factory reset in
the post-install phase. Hook up with the --wipe_user_data flag in the
OTA script.

Bug: 28700985
Change-Id: Ie73876a61db90d124d2af588d674757376e9aabc
2016-06-15 17:07:03 -07:00
Tao Bao 46ff838582 Merge \"releasetools: Disable using imgdiff for squashfs.\" into nyc-dev
am: a8d3741f1a

Change-Id: If8392b72a865f53449a1d5ca7b52de7697c4b412
2016-06-15 16:12:52 +00:00
Tao Bao 293fd135c7 releasetools: Disable using imgdiff for squashfs.
We use imgdiff to handle files in zip format (e.g. jar/zip/apk) for
higher compression ratio.

For system/vendor in squashfs, a) all files are compressed in LZ4
format; b) we use 4096-byte block size in their sparse images, but the
files in squashfs may not be laid out as 4K-aligned. So the blocks for
a given file as listed in block map may not form a valid zip file, which
may fail the patch generation with imgdiff.

Disable using imgdiff for squashfs images, and use bsdiff instead.

Bug: 22322817
Change-Id: Ie76aa4cece5c9d38cb1d1a34c505a4a8f37512d3
2016-06-13 10:04:23 -07:00
Alex Deymo dbae67fac8 Merge \\"Redirect subprocess stderr to stdout in verbose mode.\\" am: 1318ec93e7
am: 74d7b31f7d

Change-Id: Ia9757f09dead577349b0cf4b97cb062f30d0f45d
2016-06-11 01:12:06 +00:00
Alex Deymo d8d96ecdae Redirect subprocess stderr to stdout in verbose mode.
This patch uses subprocess.communicate instead of subprocess.wait to
prevent deadlock if any of the child processes outputs too much data,
and redirects the subprocess output to stdout when running in verbose
mode.

With this patch `ota_from_target_files -v` prints the delta_generator
output in stdout, and no output if '-v' is not passed.

Bug: None
TEST=ota_from_target_files -v ...

Change-Id: Id66e4f3360a6f91d61a3ce96d53afbccdaa19da5
2016-06-10 16:38:31 -07:00
Tianjie Xu c201d28713 Add build.version.incremental to metadata
am: d06f07eef4

Change-Id: I4f0b89d69154ef32bbb04fa905fdc014c911d571
2016-06-10 17:24:52 +00:00
Tianjie Xu d06f07eef4 Add build.version.incremental to metadata
Add the build property "build.version.incremental" of the source (if
present) and target files to the metadata of the ota update package.

Example of metadata:
....
post-build-incremental=2951741
post-timestamp=1465345123
pre-build-incremental=2943039
pre-device=bullhead
...

Bug: 28658632
Change-Id: I889e8ccf39633b1b35590751001a42d1b05d5514
2016-06-09 17:54:46 -07:00
Tao Bao 24105eaab8 Merge "releasetools: Only verify the blocks to be touched." into nyc-dev
am: 21528c5

* commit '21528c5e053e28cd52d603eded53ffaf36d22637':
  releasetools: Only verify the blocks to be touched.

Change-Id: I053c7da789c44916456109c5153f6628fe38c849
2016-04-14 00:02:41 +00:00
Tao Bao d4caaae160 releasetools: Only verify the blocks to be touched.
For incremental BBOTAs, we used to verify the integrity of all the
blocks in the source partition. In order to reduce the time cost under
recovery, this CL changes to only verify the blocks that will be touched
in the given OTA package (BBOTA >= 3 only). This is a trade-off between
performance and reliability.

Bug: 27813356
Change-Id: I3975ae6f461f0f7e58d24f1df7df46a449d2988b
(cherry picked from commit d522bdc9ed)
2016-04-13 17:00:34 -07:00
Tao Bao d522bdc9ed releasetools: Only verify the blocks to be touched.
For incremental BBOTAs, we used to verify the integrity of all the
blocks in the source partition. In order to reduce the time cost under
recovery, this CL changes to only verify the blocks that will be touched
in the given OTA package (BBOTA >= 3 only). This is a trade-off between
performance and reliability.

Bug: 27813356
Change-Id: I3975ae6f461f0f7e58d24f1df7df46a449d2988b
2016-04-12 16:41:01 -07:00
Tao Bao 3e30d97dde Revert "Revert "releasetools: Support OTAs that have OEM properties changes.""
This CL fixes the bug in [1] (copy-paste error) and reenables it.

We need to handle a special case that an OTA goes from a source build
without OEM properties to a target build with those properties (or vice
versa). Add support in OTA scripts to deal the case properly, by a)
using two oem_props variables to handle source and target builds
respectively; b) adjusting the fingerprint/thumbprint assertions to
allow a mix of both.

[1] commit c086370440

Change-Id: I98118d77d5a0ff694fa1ee33602b5ee5e048599b
2016-03-15 13:28:49 -07:00
Tao Bao 3c37889299 Merge "Revert "releasetools: Support OTAs that have OEM properties changes."" 2016-03-15 19:16:33 +00:00
Tao Bao 838c68fa1a Revert "releasetools: Support OTAs that have OEM properties changes."
Broke some builds.

This reverts commit c086370440.

Change-Id: If07006db412fea6c04f8698776a2ab418125a469
2016-03-15 19:16:18 +00:00
Tao Bao 61cebe79f4 Merge "releasetools: Support OTAs that have OEM properties changes." 2016-03-15 18:33:20 +00:00
Tao Bao 00ed831a11 resolve merge conflicts of 2ad8b97577 to nyc-dev
Change-Id: Ifa7501e58dd25e73ed760b1383a1873e4d449885
2016-03-08 21:39:13 -08:00
Tao Bao 7302eb7dd6 Use the update-binary in the source build for downgrade OTAs.
am: fa41fb2705

* commit 'fa41fb27054185b3fba0519d01dabd21444fe772':
  Use the update-binary in the source build for downgrade OTAs.
2016-03-09 05:27:36 +00:00
Tao Bao fa41fb2705 Use the update-binary in the source build for downgrade OTAs.
We used to use the update-binary from the target build when creating
incremental OTAs. But for downgrade OTAs, we should use the one in the
source build instead, which is actually newer.

Bug: 27556903
Change-Id: Ib6415729b979dbffdebdda24902f7f560942801a
(cherry picked from commit 4996cf03d2)
2016-03-08 18:48:40 -08:00