Commit Graph

34091 Commits

Author SHA1 Message Date
Tao Bao d10f5a71a3 Merge "releasetools: Fix the missing 'post-build' in full OTAs." am: 1e8364a8be
am: b538502353

Change-Id: I2facefd6c08e21966dd5793c2b4a57a011d27be2
2017-03-10 00:42:04 +00:00
Tao Bao b538502353 Merge "releasetools: Fix the missing 'post-build' in full OTAs."
am: 1e8364a8be

Change-Id: I663f53f36a7cbc29a6135b548af3371b54dfe688
2017-03-10 00:39:07 +00:00
Treehugger Robot 1e8364a8be Merge "releasetools: Fix the missing 'post-build' in full OTAs." 2017-03-10 00:30:55 +00:00
Tao Bao 39f3eaf221 releasetools: Fix the missing 'post-build' in full OTAs.
The line was unintentionally removed in commit 7f804ba.

Test: ota_from_target_files.py generates a full OTA. Check the package
      metadata.

Change-Id: Icae88e2a9bb2bfc450a3d0d7ab524d6a6eac9df5
2017-03-09 15:01:11 -08:00
Tao Bao dc880f5122 Merge "releasetools: Remove dead functions in EdifyGenerator." am: c87b38f547
am: debedc3343

Change-Id: I4a335ba2babcf30ff6f8549bc0c7f38cab17ec0c
2017-03-09 20:34:44 +00:00
Tao Bao debedc3343 Merge "releasetools: Remove dead functions in EdifyGenerator."
am: c87b38f547

Change-Id: Ia1eae3a4e401048c8cbfbf9bc85597dfc450fc4c
2017-03-09 20:31:19 +00:00
Tao Bao c87b38f547 Merge "releasetools: Remove dead functions in EdifyGenerator." 2017-03-09 20:24:19 +00:00
Tao Bao f388104eaa releasetools: Remove dead functions in EdifyGenerator.
Remove the following functions that are needed for file-based OTAs only:
 - SetPermissions()
 - SetPermissionsRecursive()
 - MakeSymlinks()
 - DeleteFiles()
 - DeleteFilesIfNotMatching()
 - RenameFiles()
 - SkipNextActionIfTargetExists()

Bug: 35853185
Test: Verified there's no reference to these functions.
Change-Id: Iff24a9d705476211effaef28eed2a147fa5fcbce
2017-03-08 16:37:07 -08:00
Steven Moreland 270f3a4778 Merge "hardware/interfaces: Android.mk always -Werror" am: df58de6303
am: 05a7a4da70

Change-Id: I3b6ef3cd6e4b741324f50c76cc910449428e76da
2017-03-08 23:23:29 +00:00
Steven Moreland 05a7a4da70 Merge "hardware/interfaces: Android.mk always -Werror"
am: df58de6303

Change-Id: I5bc41aee011d13f7579563fa0e8e6aa6988ddc04
2017-03-08 23:20:58 +00:00
Treehugger Robot df58de6303 Merge "hardware/interfaces: Android.mk always -Werror" 2017-03-08 23:16:35 +00:00
Tao Bao 3b848d039d Merge "releasetools: Drop the support for file-based OTA generation." am: 9bca43858f
am: aac8098d61

Change-Id: I795c0c7ff0342526e45ddfffa9f72e5e429b16b4
2017-03-08 21:10:19 +00:00
Tao Bao aac8098d61 Merge "releasetools: Drop the support for file-based OTA generation."
am: 9bca43858f

Change-Id: Ifffa94ab131ea25194517bad1c75a7ba504b1c7c
2017-03-08 21:07:19 +00:00
Tao Bao 9bca43858f Merge "releasetools: Drop the support for file-based OTA generation." 2017-03-08 21:00:03 +00:00
Dan Willemsen 53d64224ab Merge changes Ia5ce6870,I155654cd am: c619f99e2f
am: 6ffe13f8e7

Change-Id: Ie5f114fb34e95c189936d73472f502c82c292a9f
2017-03-08 20:54:48 +00:00
Dan Willemsen 6ffe13f8e7 Merge changes Ia5ce6870,I155654cd
am: c619f99e2f

Change-Id: I5006c4e87607a55f914b90f7218802bc29fc76c5
2017-03-08 20:51:19 +00:00
Dan Willemsen c619f99e2f Merge changes Ia5ce6870,I155654cd
* changes:
  Zip after add_img_to_target_files
  Allow add_img_to_target_files to work on a directory
2017-03-08 20:45:56 +00:00
Tao Bao a56e7c81e2 Merge "releasetools: Reduce memory footprint for BBOTA generation." am: 3b1f260205
am: cf82616047

Change-Id: I26b65079be3c6e72025e1b5705205bbabdc8fca9
2017-03-08 17:51:05 +00:00
Tao Bao cf82616047 Merge "releasetools: Reduce memory footprint for BBOTA generation."
am: 3b1f260205

Change-Id: I2b141db33f2a350d5280fb534999052c1ef439b2
2017-03-08 17:48:36 +00:00
Tao Bao 3b1f260205 Merge "releasetools: Reduce memory footprint for BBOTA generation." 2017-03-08 17:43:34 +00:00
Steven Moreland 324c436084 hardware/interfaces: Android.mk always -Werror
Bug: 35840847
Test: pass
Change-Id: I6ebf3f5d279ff405446592cdabaeca9d1fdf6cf9
2017-03-08 16:29:50 +00:00
Sen Jiang f1f1266fb1 Merge "Put system_version in target_files.zip" am: 3838ae7e9b
am: 75a7a591ba

Change-Id: I24c3d03dd9c30ec67dc7f9231da392f811650b20
2017-03-08 03:28:11 +00:00
Sen Jiang 75a7a591ba Merge "Put system_version in target_files.zip"
am: 3838ae7e9b

Change-Id: I357ce185442ec04c028858f187e42187c062a287
2017-03-08 03:25:41 +00:00
Treehugger Robot 3838ae7e9b Merge "Put system_version in target_files.zip" 2017-03-08 03:21:43 +00:00
Sen Jiang 9b8e4ca518 Put system_version in target_files.zip
Bug: 35364984
Test: m dist
Change-Id: I309acb41eb85de239c4c1718e14d9442ca1b0408
(cherry picked from commit cea4d52d2fee19281e660943129e1ad3bdc51fef)
2017-03-07 17:40:27 -08:00
Dan Willemsen 2a5a34f6bd Zip after add_img_to_target_files
For aosp_marlin (on AOSP), this speeds up target files creation from
~57 seconds to ~15 seconds.

For marlin-userdebug (on internal master), this speeds up target files
creation from ~300 seconds to ~95 seconds.

This does make some minor changes to the resulting target_files zip:

* All of the META files are actually at the beginning of the zip now,
  previously the ones created by add_img_to_target_files would be at
  the end.
* The images are slightly larger. Go's zip implementation in soong_zip
  isn't quite as good as the `zip` tool.
* vendor_filesystem_config.txt isn't present if we're not building a
  vendor image.

Test: aosp_marlin target_files zip changes look reasonable
Test: marlin target_files zip changes look reasonable
Test: bullhead target_files zip changes look reasonable
Change-Id: Ia5ce6870b85559d65107c3b94332391b4b43a0ea
2017-03-07 15:28:42 -08:00
Dan Willemsen 2ee00d5f66 Allow add_img_to_target_files to work on a directory
In addition to the current behavior of add_img_to_target_files working
on an existing zip file, allow passing in a directory where the target
files have already been extracted. When in this mode, it writes the
images out to that directory instead of the zip file.

This allows us to call add_img_to_target_files on the temp directory
used during the build to create the target files package, saving the
time and space of unzipping what we just zipped. This also allows us to
use the parallel soong_zip, which compresses the images much faster.

Test: aosp_marlin target_files zip is the same before/after this change
Test: marlin target_files zip is the same before/after this change
Test: bullhead target_files zip is the same before/after this change
Change-Id: I155654cdc7ad7754ba4ef76ec69c31f504a58198
2017-03-07 14:37:52 -08:00
Steven Moreland f2d838a52e Merge "Add vnod." am: 2b33e46593
am: 0e1913f4ff

Change-Id: Ibea28328b1267cbcf7a326ace2c448f8a2b6b792
2017-03-07 22:01:49 +00:00
Steven Moreland 0e1913f4ff Merge "Add vnod."
am: 2b33e46593

Change-Id: I006ef6b77e33f5b6e075fa392326e3e0eebb4610
2017-03-07 21:59:18 +00:00
Treehugger Robot 2b33e46593 Merge "Add vnod." 2017-03-07 21:54:07 +00:00
Steven Moreland a2e734d786 Add vnod.
Short name for vendorimage-nodeps like snod.

Bug: 33356516
Test: works
Change-Id: Iad62aa5a021d603fa9cebe90f4e28cf09d5d49ae
2017-03-07 12:17:51 -08:00
Tao Bao 457cbf6a8a releasetools: Drop the support for file-based OTA generation.
We have stopped shipping devices with file-based OTAs, and are not
actively maintaining the support. Devices using file-based OTAs
should be moved to block-based, if not A/B OTAs.

We will also need to clean up EdifyGenerator class, which will be
handled in follow-up CLs.

Bug: 35853185
Test: Generate full and incremental OTAs w/ and w/o the CL, and they
      give identical packages.
Test: Not specifying --block also generates block-base OTAs.
Change-Id: I3b0fc8ce5600e109f3251fe41f655534aaa298c7
2017-03-07 00:17:58 -08:00
Tao Bao 657ed318d8 Merge "releasetools: Remove the obsolete path in GetImage()." am: 7dc54e9761
am: 438753de24

Change-Id: I6edbb699689b39e263ea21ce875a0abf8dc7772c
2017-03-07 06:27:57 +00:00
Tao Bao 438753de24 Merge "releasetools: Remove the obsolete path in GetImage()."
am: 7dc54e9761

Change-Id: I75b60aeb1c8d9264179dd42fd9ce1f78e66c289c
2017-03-07 06:25:58 +00:00
Tao Bao 7dc54e9761 Merge "releasetools: Remove the obsolete path in GetImage()." 2017-03-07 06:20:04 +00:00
Tao Bao e319d34b21 Merge "releasetools: Fix two minor issues with _LoadOemDicts()." am: 0c9e8d87c4
am: cedbcb634f

Change-Id: Id80b3161d864392b249eb3d01ebbd28be16a8e61
2017-03-07 02:43:37 +00:00
Tao Bao cedbcb634f Merge "releasetools: Fix two minor issues with _LoadOemDicts()."
am: 0c9e8d87c4

Change-Id: I576ed85991a0e25ba2b05458d928bc38149c24b8
2017-03-07 02:41:38 +00:00
Tao Bao 0c9e8d87c4 Merge "releasetools: Fix two minor issues with _LoadOemDicts()." 2017-03-07 00:24:44 +00:00
Tao Bao 7e0f160f30 releasetools: Remove the obsolete path in GetImage().
We must have created the images (system.img, system.map etc) prior to
calling ota_from_target_files.py (added by commit
2e0d8fcf08, into Lollipop).

Remove the obsolete suppport for handling "old" target_files zips that
don't have such images. This simplies the logic for BuildSystem() and
BuildVendor(), which now would only be called by
add_img_to_target_files.py itself.

Test: Generating full and incremental OTAs give the same results, w/ and
      w/o this CL.
Change-Id: I0ea243d09d3378249d7982701ae4ec339b04b7b8
2017-03-06 16:22:44 -08:00
Mathieu Chartier de8546f31b Merge "Fix profile dependency" am: 4d42fef1db
am: 52e0317804

Change-Id: I6a51e3fc7b1f0c1a0187304be04f21bc1d1355fc
2017-03-06 20:30:20 +00:00
Mathieu Chartier 52e0317804 Merge "Fix profile dependency"
am: 4d42fef1db

Change-Id: Idd14fa4a1e0141558a301b80eb53da0442adf79b
2017-03-06 20:22:20 +00:00
Mathieu Chartier 4d42fef1db Merge "Fix profile dependency" 2017-03-06 19:45:00 +00:00
Mathieu Chartier fb6c6b38c3 Fix profile dependency
Otherwise it doesn't work for incremental builds.

Bug: 34929159
Bug: 35997300

Test: repo sync && m

Change-Id: I78ead57cf89a9d0f26dbf4f3a225a9357bdca1b3
2017-03-06 11:38:49 -08:00
Tao Bao ebce697429 releasetools: Fix two minor issues with _LoadOemDicts().
The first one in WriteVerifyPackage() is mismatching function parameters
that can be trivially fixed.

The other one is in WriteABOTAPackageWithBrilloScript(), where we don't
have edify script instance.

Test: `pylint --rcfile=pylintrc ota_from_target_files.py`.
Change-Id: Ie238ef5b296dfe9e725b61376992447b662d2376
2017-03-06 10:29:13 -08:00
Tao Bao 654e4f4c77 Merge "Revert "Revert "releasetools: Use pattern filter when unzipping.""" am: 8e02284362
am: 85f8de467a

Change-Id: I56d3dee99e7f33c5c1e5efd23776cf2106e7d63d
2017-03-06 16:54:06 +00:00
Tao Bao 85f8de467a Merge "Revert "Revert "releasetools: Use pattern filter when unzipping."""
am: 8e02284362

Change-Id: I9919e6c46d500523d096afc77f2bcc1fb4caabd4
2017-03-06 16:50:37 +00:00
Tao Bao 183e56e83d releasetools: Reduce memory footprint for BBOTA generation.
The major issue with the existing implementation is unnecessarily
holding too much data in memory, such as HashBlocks() which first reads
in *all* the data to a list before hashing. We can leverage generator
functions to stream such operations.

This CL makes the following changes to reduce the peak memory use.
 - Adding RangeSha1() and WriteRangeDataToFd() to Image classes. These
   functions perform the operations on-the-fly.
 - Caching the computed SHA-1 values for a Transfer instance.

As a result, this CL reduces the peak memory use by ~80% (e.g. reducing
from 5.85GB to 1.16GB for the same incremental, as shown by "Maximum
resident set size" from `/usr/bin/time -v`). It also effectively
improves the (package generation) performance by ~30%.

Bug: 35768998
Bug: 32312123
Test: Generating the same incremental w/ and w/o the CL give identical
      output packages.
Change-Id: Ia5c6314b41da73dd6fe1dbe2ca81bbd89b517cec
2017-03-06 08:46:30 -08:00
Tao Bao 8e02284362 Merge "Revert "Revert "releasetools: Use pattern filter when unzipping.""" 2017-03-06 16:45:26 +00:00
Tao Bao 6b0b2f9db0 Revert "Revert "releasetools: Use pattern filter when unzipping.""
This reverts commit a7316ce094.

This CL differs from the original CL by not unzipping RADIO/*. This is
because: a) AOSP targets don't have RADIO/ entries in the TF.zip; b)
we're not using the unzipped RADIO files (but reading them from the zip
files directly) - checked all the device-specific releasetools for
angler, bullhead, ryu, shamu, volantis, fugu, marlin and sailfish.

Test: `m dist` with AOSP targets (tested fugu and bullhead).
Change-Id: I4d0c67214ddd6202fc27c71bb79f52b5f4d40c64
2017-03-05 13:11:42 -08:00
Tao Bao a4170833d1 Merge "Revert "releasetools: Use pattern filter when unzipping."" am: 0c3e31c69b
am: 6c98ad6c4e

Change-Id: Ide956c794dee147ff5f6cfa6e8afc06f3c8b49a1
2017-03-05 17:52:48 +00:00