Commit Graph

5559 Commits

Author SHA1 Message Date
Tao Bao cc8e2666ac releasetools: Create StreamingPropertyFiles class.
This CL breaks down ComputeStreamingMetadata() into mutiple member
functions of StreamingPropertyFiles class, which correspond to the
two-pass logic when generating streaming property files (aka streaming
metadata).

StreamingPropertyFiles.Compute() does the work for the first pass, by
putting placeholders before doing initial signing. Finalize()
corresponds to the second pass, where the placeholders get replaced with
actual data. Verify() can be optionally called to assert the correctness
of the work.

The separation between Compute() and Finalize() is to allow having
multiple StreamingPropertyFiles instances (in coming up CLs). This way
we can call Compute() multiple times for each instance, followed by only
one call to SignOutput(). And similarly for Finalize().

Bug: 74210298
Test: Generate an A/B OTA package. Check the METADATA entry.
Test: python -m unittest test_ota_from_target_files
Change-Id: I45be0372a4863c4405e6d8e20bcb9ccdc29e7e11
(cherry picked from commit ae5e4c30fe)
2018-03-09 13:44:13 -08:00
Tao Bao f6f13acf73 releasetools: Fix the broken validate_target_files.py.
The file was broken due to earlier touches:
 - Missing 'import zipfile';
 - Mismatching arguments when calling GetSparseImage().

Bug: 73996151
Test: Run validate_target_files.py with a walleye-target_files.zip.
Test: pylint --rcfile=pylintrc validate_target_files.py
Change-Id: I3692bd51fb27a3da698e06b75155e84502549f66
(cherry picked from commit c63626b4a3)
2018-03-08 08:10:28 -08:00
Tao Bao fe5b69a4ae releasetools: Separate streaming metadata computation into functions.
And add unittests for ComputeStreamingMetadata().

This prepares for the changes that add additional property-files (for
both of A/B and non-A/B).

Bug: 74210298
Bug: 72751683
Test: python -m unittest test_ota_from_target_files
Test: Generate A/B OTA package. Check the ota-streaming-property-files
      in the METADATA entry.
Change-Id: Ib4b069f61c2c06c035c0cff73a55112f3936b969
(cherry picked from commit f5110498c0)
2018-03-06 20:26:38 -08:00
Tao Bao 8e9155f236 Merge "releasetools: Fix a mismatching close." am: 395bf274b8 am: 06bc45b0c2
am: 6b93ed0c84

Change-Id: I74f9e797510bcecdb1f93824ce5881f92ff3b152
2018-03-03 00:10:31 +00:00
Tao Bao 6b93ed0c84 Merge "releasetools: Fix a mismatching close." am: 395bf274b8
am: 06bc45b0c2

Change-Id: I5e9a7d4a74e476b1ed86a9f7bc06e515998f81e2
2018-03-02 23:59:22 +00:00
Tao Bao 395bf274b8 Merge "releasetools: Fix a mismatching close." 2018-03-02 23:32:11 +00:00
Tao Bao 921c81f7c3 Merge "releasetools: Call Cleanup() in check_target_files_signatures.py." am: 903be914bb am: 5410d8a52a
am: 501b84236f

Change-Id: Ia4dfee212e3ff55fcdd9ce60faddab91c5e195c6
2018-03-02 23:05:20 +00:00
Tao Bao 501b84236f Merge "releasetools: Call Cleanup() in check_target_files_signatures.py." am: 903be914bb
am: 5410d8a52a

Change-Id: I3d015a9ffa1784c060d6a1d878feae9046090b5d
2018-03-02 22:55:11 +00:00
Tao Bao a652c00c8e releasetools: Fix a mismatching close.
There's a mismatch in WriteABOTAPackageWithBrilloScript().

  temp_zip_file = tempfile.NamedTemporaryFile()
  ...
  common.ZipClose(temp_zip_file)

It's benign since common.ZipClose() happens to be calling
"temp_zip_file.close()". This CL moves the use of tempfile to
common.MakeTempFile(), so that the tempfile will be cleaned up
automatically as part of the call to common.Cleanup(). (Not fixing the
close() directly, since the nearby lines will be refactored into another
function shortly.)

Also remove one assert in the same function, which trivially holds in
the current code.

Test: Generate an A/B OTA.
Change-Id: I53b375d1150820de718dec0ead55abf5f4951071
2018-03-02 13:20:43 -08:00
Tao Bao 150708b012 Merge "releasetools: Don't return ZipFile from common.UnzipTemp()." am: 5c93279fbe am: 0db979c6be
am: abe02acc48

Change-Id: Ie8d1d55d134a7aade1be643634641ef0eb80ff65
2018-03-01 22:44:21 +00:00
Tao Bao abe02acc48 Merge "releasetools: Don't return ZipFile from common.UnzipTemp()." am: 5c93279fbe
am: 0db979c6be

Change-Id: I9a4f2bc04851f048e168a11c039d781c2069b662
2018-03-01 22:38:46 +00:00
Tao Bao 767543a659 releasetools: Call Cleanup() in check_target_files_signatures.py.
Also minor cleanups to make it pylint clean.

Test: Run check_target_files_signatures.py with a target-files.zip.
Test: pylint --rcfile=pylintrc check_target_files_signatures.py
Change-Id: Ife3b54c7805c2f2562e87e91ab4b4de355782012
2018-03-01 10:12:39 -08:00
Tao Bao dba59eeabd releasetools: Don't return ZipFile from common.UnzipTemp().
In addition to the unzipping work, common.UnzipTemp() kindly bundles an
open ZipFile object as part of the return value. It doesn't look very
helpful to the callers though. It also looks less obvious that the
caller needs to properly close the handle (missing the close here is
benign though). This CL just removes the ZipFile object out of the
return value, and leaves the work to callers.

Test: `m dist` on both of A/B and non-A/B target.
Test: python -m unittest test_add_img_to_target_files
Test: python -m unittest test_common
Test: python -m unittest test_ota_from_target_files
Test: Check the callers to common.UnzipTemp() in code search.
Change-Id: Id47da3fd42a0e76d6ae8851f05780db319ee48cf
2018-03-01 10:03:31 -08:00
Tao Bao 7a08313a5e Merge "releasetools: Move one case of mkdtemp() to common.MakeTempDir()." am: a7a5e15f72 am: fc0bb788ed
am: e9f3ee0f80

Change-Id: Ie5cf8533ae02aab3f52c3652b2afc55c729e16f2
2018-02-28 21:37:09 +00:00
Tao Bao e9f3ee0f80 Merge "releasetools: Move one case of mkdtemp() to common.MakeTempDir()." am: a7a5e15f72
am: fc0bb788ed

Change-Id: Icb0ffb498e068417bdadabd4877731c63b776f37
2018-02-28 21:30:30 +00:00
Tao Bao 04bce3a696 releasetools: Move one case of mkdtemp() to common.MakeTempDir().
This should be the last case to be moved over.

Test: Generate an incremental BBOTA (which exercises the changed code).
Test: `rgrep mkdtemp` gives no more instance.
Change-Id: I76db069476201cdfaf3a2de9d9635dfe54507f7a
2018-02-28 11:13:29 -08:00
Tao Bao 9a90d8cae6 Merge "releasetools: Remove the unneeded 'prefix' parameters." am: 6e2a8ae094 am: 945e9c3e44
am: 5d74ac8574

Change-Id: I84554693ebd6be87eb0989d9b2d1df5f049c62f7
2018-02-27 22:36:18 +00:00
Tao Bao 5d74ac8574 Merge "releasetools: Remove the unneeded 'prefix' parameters." am: 6e2a8ae094
am: 945e9c3e44

Change-Id: Iee385b3b5d3a8f3848ccbfdd3773b7cf287f40cf
2018-02-27 22:31:24 +00:00
Tao Bao 6e2a8ae094 Merge "releasetools: Remove the unneeded 'prefix' parameters." 2018-02-27 22:13:06 +00:00
Tao Bao 886d883d61 releasetools: Remove the unneeded 'prefix' parameters.
... from the following functions in add_img_to_target_files.py.

  AddSystem
  AddSystemOther
  AddVendor
  AddProduct
  AddDtbo
  AddUserdata
  AddVBMeta
  AddPartitionTable
  AddCache

The last user of the parameter in img_from_target_files.py has been
removed in commit 2bb109709a (in O).

Test: pylint --rcfile=pylintrc add_img_to_target_files.py
Test: Check all the callers to the above functions.
Test: m dist
Change-Id: I551d1683def8f8535062fc90f68dafa0f4252822
2018-02-27 11:47:11 -08:00
Luis Hector Chavez 0fae5d0806 Merge "Make mksquashfs generate a fs_config-friendly root entry" am: 3d4fb4e335 am: 6128d7f30d
am: 21025515e4

Change-Id: I710cbaa8379c118e0c6a519f5d737a6395de2646
2018-02-27 16:41:46 +00:00
Luis Hector Chavez 21025515e4 Merge "Make mksquashfs generate a fs_config-friendly root entry" am: 3d4fb4e335
am: 6128d7f30d

Change-Id: I43a62ea7c4d6acb5adadfc488b94e05024499320
2018-02-27 16:36:39 +00:00
Treehugger Robot 3d4fb4e335 Merge "Make mksquashfs generate a fs_config-friendly root entry" 2018-02-27 16:24:21 +00:00
Dan Shi 9801d9b7b2 Merge "Default test to AndroidJUnitTest for auto-generated test config." am: ce1fa25d1b am: d91129eea9
am: 6dbd9e3df9

Change-Id: I84ee22fa20b08fb3db27d5e5d0e03f097a11463f
2018-02-26 23:26:06 +00:00
Dan Shi 6dbd9e3df9 Merge "Default test to AndroidJUnitTest for auto-generated test config." am: ce1fa25d1b
am: d91129eea9

Change-Id: Ib0e817e8d403ea130516dbc94201fd4a899c327f
2018-02-26 23:18:57 +00:00
Treehugger Robot ce1fa25d1b Merge "Default test to AndroidJUnitTest for auto-generated test config." 2018-02-26 23:00:28 +00:00
Tao Bao 47db251db0 Merge "releasetools: Remove the parsing of META/imagesizes.txt." am: 3aee4dfac1 am: 72f3c44243
am: ed98b94326

Change-Id: Id4f84a66faf7c06b4bbacae799d36db4a498182b
2018-02-26 22:46:52 +00:00
Tao Bao ed98b94326 Merge "releasetools: Remove the parsing of META/imagesizes.txt." am: 3aee4dfac1
am: 72f3c44243

Change-Id: I83af79b6d5a76099259a951585d92755b132af9e
2018-02-26 22:39:14 +00:00
Tao Bao 0835594a6f Merge "releasetools: Add unittests for GetCareMap() and AddCareMapTxtForAbOta()." am: 7998503ebe am: 5fed709d60
am: b8d0b5bd06

Change-Id: Iec88f7bdbf26047479d217d756d4847b1bac64f7
2018-02-26 22:32:18 +00:00
Tao Bao b8d0b5bd06 Merge "releasetools: Add unittests for GetCareMap() and AddCareMapTxtForAbOta()." am: 7998503ebe
am: 5fed709d60

Change-Id: I117d452b1a4d63b7665c3593b5b2167fe57be954
2018-02-26 22:24:13 +00:00
Treehugger Robot 3aee4dfac1 Merge "releasetools: Remove the parsing of META/imagesizes.txt." 2018-02-26 22:18:31 +00:00
Tao Bao 7998503ebe Merge "releasetools: Add unittests for GetCareMap() and AddCareMapTxtForAbOta()." 2018-02-26 22:06:32 +00:00
Tao Bao 41fb7d27cf releasetools: Remove the parsing of META/imagesizes.txt.
The file has been removed from target-files.zip since commit
c19a8d5590 (Gingerbread), whose info has
been consolidated into META/misc_info.txt.

Test: `m dist`
Change-Id: Ic144457954f5742ea082dcd9ffbea71df4afe46e
2018-02-23 12:06:07 -08:00
Tao Bao cf9c445ef9 Merge "releasetools: Allow skipping postinstall hooks when generating A/B OTAs." am: 6c0ddf2480 am: ff1288821d
am: a5baf56e6c

Change-Id: I59bf83ba415351ba07488086d8b812453b0ae61b
2018-02-22 20:49:01 +00:00
Tao Bao a5baf56e6c Merge "releasetools: Allow skipping postinstall hooks when generating A/B OTAs." am: 6c0ddf2480
am: ff1288821d

Change-Id: Iea818eb91dd5ff98d9b378a845ff507e1dee1e0c
2018-02-22 20:40:25 +00:00
Tao Bao 6c0ddf2480 Merge "releasetools: Allow skipping postinstall hooks when generating A/B OTAs." 2018-02-22 20:21:36 +00:00
Tao Bao 567fa9012b Merge "releasetools: Check for duplicate entries in ReplaceCerts()." am: 8bfde7c5fb am: 7032691417
am: 894cba2954

Change-Id: Ic89d331846654d23768e90cb4abab1b91d744271
2018-02-22 18:05:39 +00:00
Tao Bao 894cba2954 Merge "releasetools: Check for duplicate entries in ReplaceCerts()." am: 8bfde7c5fb
am: 7032691417

Change-Id: Icc23606930f1a2f3e0009b1ec225723ffc8a7617
2018-02-22 17:57:03 +00:00
Tao Bao 8bfde7c5fb Merge "releasetools: Check for duplicate entries in ReplaceCerts()." 2018-02-22 17:39:00 +00:00
Tao Bao 63c18fead8 releasetools: Add unittests for GetCareMap() and AddCareMapTxtForAbOta().
With test_utils.construct_sparse_image(), creating valid sparse image is
no longer a blocker for testing these functions.

Test: python -m unittest test_add_img_to_target_files
Change-Id: Iff1f15837cc744bad52e2120f9a9ad94d7db50d5
2018-02-22 00:10:05 -08:00
Tao Bao 15a146a7c4 releasetools: Allow skipping postinstall hooks when generating A/B OTAs.
This CL adds a new flag '--skip_postinstall' that allows skipping all
the postinstall hooks when generating an A/B OTA package (default:
False). Note that this discards ALL the hooks, including non-optional
ones. Should only be used if caller knows it's safe to do so (e.g. all
the postinstall work is to dexopt apps and a data wipe will happen
immediately after).

Bug: 73547992
Test: python -m unittest test_ota_from_target_files
Test: Generate a full OTA package for walleye. Examine the generated
      payload.
Change-Id: Ifc069e897b4019605051eabfd221230a6a37867c
2018-02-21 16:34:19 -08:00
Dan Shi 96068b77ed Default test to AndroidJUnitTest for auto-generated test config.
All new instrumentation tests shall use AndroidJUnitRunner, thus default
test class to AndroidJUnitTest.

Bug: 73679956
Test: unittest
Change-Id: I52df4d01b8459a4a6e5c2cbe15973dab079fb136
2018-02-21 11:31:06 -08:00
Tao Bao 039ccf14ca Merge "releasetools: Clean up test_sign_target_files_apks.py." am: 5ef78033d5 am: e1fee81bb5
am: 4e636f0441

Change-Id: I9133283350bd0d9c74d6d2b9b313a5fd1103aa48
2018-02-21 02:36:43 +00:00
Tao Bao 4e636f0441 Merge "releasetools: Clean up test_sign_target_files_apks.py." am: 5ef78033d5
am: e1fee81bb5

Change-Id: I3c504df0a047b88725433c68079c18495d536afe
2018-02-21 02:29:12 +00:00
Tao Bao 5ef78033d5 Merge "releasetools: Clean up test_sign_target_files_apks.py." 2018-02-21 02:06:00 +00:00
Tao Bao 90c3d2ebb5 Merge "releasetools: Add a testcase for common.ParseCertificate()." am: a0f55ee9eb am: 9773a2f7b2
am: 6f142ef096

Change-Id: Ic838ebf4e59d8d90eb102c0ae4399b639fcf64f6
2018-02-21 00:48:47 +00:00
Tao Bao 6f142ef096 Merge "releasetools: Add a testcase for common.ParseCertificate()." am: a0f55ee9eb
am: 9773a2f7b2

Change-Id: I3b00809f57c7f42f69fdbb44cdc5516e585ee987
2018-02-21 00:28:14 +00:00
Tao Bao a0f55ee9eb Merge "releasetools: Add a testcase for common.ParseCertificate()." 2018-02-21 00:04:24 +00:00
Tao Bao 4051a05b54 Merge "releasetools: Add unittest for common.GetSparseImage()." am: 35ee189898 am: 625b5b7a09
am: 0fa48c8c52

Change-Id: I12997ace8bfc5936a6db773b570c7ec2e26edb47
2018-02-20 21:27:34 +00:00
Tao Bao 0fa48c8c52 Merge "releasetools: Add unittest for common.GetSparseImage()." am: 35ee189898
am: 625b5b7a09

Change-Id: I494f253ab122f4098ec1e1fd814c915e618e2aa2
2018-02-20 21:21:31 +00:00