Commit Graph

43118 Commits

Author SHA1 Message Date
Adam Seaton ecebf9c349 Merge "[DO NOT MERGE]Update Security String to 2018-04-01 in oc-dev Bug:73240847" into oc-dev am: 656fd837e9 am: 05e66a80dc am: 7231e99020
am: a540c9466b  -s ours

Change-Id: Iee3445aeb218861e202583f1cc08d24dae902295
2018-02-14 00:06:45 +00:00
Adam Seaton a540c9466b Merge "[DO NOT MERGE]Update Security String to 2018-04-01 in oc-dev Bug:73240847" into oc-dev am: 656fd837e9 am: 05e66a80dc
am: 7231e99020

Change-Id: I9ed65a0bfddfcdd30873dff9cb683fc08d31f6dc
2018-02-14 00:01:05 +00:00
Adam Seaton 7231e99020 Merge "[DO NOT MERGE]Update Security String to 2018-04-01 in oc-dev Bug:73240847" into oc-dev am: 656fd837e9
am: 05e66a80dc

Change-Id: Ia769104e933025596c98b66598091eca3e3a860d
2018-02-13 23:54:31 +00:00
Adam Seaton 05e66a80dc Merge "[DO NOT MERGE]Update Security String to 2018-04-01 in oc-dev Bug:73240847" into oc-dev
am: 656fd837e9

Change-Id: Ic22c80e40cff1fe0afea3f26741d10c8a949b27b
2018-02-13 23:49:57 +00:00
TreeHugger Robot 656fd837e9 Merge "[DO NOT MERGE]Update Security String to 2018-04-01 in oc-dev Bug:73240847" into oc-dev 2018-02-13 23:35:05 +00:00
Adam Seaton 819b18263f Merge changes from topic "am-ddd6c5a0-3aa8-4d6c-a3b8-0460c103683f" into oc-dev am: 0d66f2b854 am: 24cbf1d542 am: 0940a73880
am: c4c2338c69

Change-Id: I98d88b80f0c1120db527a4b89f14286a61a8339f
2018-02-13 23:19:16 +00:00
Tao Bao 9d0242d93e Merge "releasetools: Allow generating BBOTA for images with shared blocks." am: 9452d84b97 am: 6c453193a1
am: 7088b88a17

Change-Id: I123238db80af7d60e92da4aef0478f9f5f1a29a5
2018-02-13 23:13:15 +00:00
Tao Bao 072f2e4784 Merge "releasetools: Remove the unconditional fallback to bsdiff." am: 6392e05813 am: 61fc7f6746
am: 6b2854d44d

Change-Id: Iac4c743329de3db006ed4a2d3782835a5002490b
2018-02-13 23:10:28 +00:00
Adam Seaton c4c2338c69 Merge changes from topic "am-ddd6c5a0-3aa8-4d6c-a3b8-0460c103683f" into oc-dev am: 0d66f2b854 am: 24cbf1d542
am: 0940a73880

Change-Id: I8f2a5550f1d886cbc789b5c41456cbb05aa72e8c
2018-02-13 23:09:40 +00:00
Adam Seaton 0940a73880 Merge changes from topic "am-ddd6c5a0-3aa8-4d6c-a3b8-0460c103683f" into oc-dev am: 0d66f2b854
am: 24cbf1d542

Change-Id: Ic71e186c3a18799af5666090107d70f037a51c37
2018-02-13 23:04:02 +00:00
Adam Seaton 04ad5c9520 [automerger] [DO NOT MERGE] Update Security String to 2018-04-01 on mnc-dev Bug:73240847 am: e4d86d9b06 am: 83709d3152 am: 70ea788dfa skipped: ac14372129 am: 83c448128c am: 50abbfdb53 am: 00a67ddab1 am: 2afc690751 skipped: 49f77737a2 am: b0aa7c342a am: a26a50ba69 am: 2d955bd781
am: 90b42aa3db  -s ours

Change-Id: I7a794b7d11e41d8e73b050f1292d9b6b0df6400b
2018-02-13 23:00:22 +00:00
Adam Seaton 24cbf1d542 Merge changes from topic "am-ddd6c5a0-3aa8-4d6c-a3b8-0460c103683f" into oc-dev
am: 0d66f2b854

Change-Id: Ia001a0d70252c2b19658e9a29c7e5c4a2a9d9cd7
2018-02-13 23:00:17 +00:00
TreeHugger Robot 0d66f2b854 Merge changes from topic "am-ddd6c5a0-3aa8-4d6c-a3b8-0460c103683f" into oc-dev
* changes:
  [automerger] [DO NOT MERGE] Update security string to 2018-04-01 in nyc-dev Bug: 73240847 am: b20e43bfbd am: e5018177a2 am: 9a781cc784 am: 82043e682d skipped: 6206e9c2c4
  [automerger] [DO NOT MERGE] Update security string to 2018-04-01 in nyc-dev Bug: 73240847 am: b20e43bfbd am: e5018177a2 am: 9a781cc784 am: 82043e682d
  [automerger] [DO NOT MERGE] Update security string to 2018-04-01 in nyc-dev Bug: 73240847 am: b20e43bfbd am: e5018177a2 am: 9a781cc784
  [automerger] [DO NOT MERGE] Update security string to 2018-04-01 in nyc-dev Bug: 73240847 am: b20e43bfbd am: e5018177a2
  [automerger] [DO NOT MERGE] Update security string to 2018-04-01 in nyc-dev Bug: 73240847 am: b20e43bfbd
  [DO NOT MERGE] Update security string to 2018-04-01 in nyc-dev Bug: 73240847
2018-02-13 22:48:46 +00:00
Tao Bao 7088b88a17 Merge "releasetools: Allow generating BBOTA for images with shared blocks." am: 9452d84b97
am: 6c453193a1

Change-Id: I422f6da0c524fa5329bc9dd11eb10eb27945ab92
2018-02-13 22:43:10 +00:00
Tao Bao 6b2854d44d Merge "releasetools: Remove the unconditional fallback to bsdiff." am: 6392e05813
am: 61fc7f6746

Change-Id: I3729f15bb532838ce2b9ec3f084ee5b81e97d3b1
2018-02-13 22:35:42 +00:00
Tao Bao 6c453193a1 Merge "releasetools: Allow generating BBOTA for images with shared blocks."
am: 9452d84b97

Change-Id: Iab1c1e4f25f67fbf9eebb9ec8a4ff0ad7e05fd13
2018-02-13 22:34:25 +00:00
Adam Seaton 90b42aa3db [automerger] [DO NOT MERGE] Update Security String to 2018-04-01 on mnc-dev Bug:73240847 am: e4d86d9b06 am: 83709d3152 am: 70ea788dfa skipped: ac14372129 am: 83c448128c am: 50abbfdb53 am: 00a67ddab1 am: 2afc690751 skipped: 49f77737a2 am: b0aa7c342a am: a26a50ba69
am: 2d955bd781

Change-Id: Ifec0f858474b1fe38bcd0bc32821cdcbb66c73b0
2018-02-13 22:32:04 +00:00
Tao Bao 61fc7f6746 Merge "releasetools: Remove the unconditional fallback to bsdiff."
am: 6392e05813

Change-Id: I1da2f7aedd14aaa5a19bfd8d83c7650272ed4214
2018-02-13 22:26:04 +00:00
Adam Seaton 2d955bd781 [automerger] [DO NOT MERGE] Update Security String to 2018-04-01 on mnc-dev Bug:73240847 am: e4d86d9b06 am: 83709d3152 am: 70ea788dfa skipped: ac14372129 am: 83c448128c am: 50abbfdb53 am: 00a67ddab1 am: 2afc690751 skipped: 49f77737a2 am: b0aa7c342a
am: a26a50ba69

Change-Id: Id6ac55b836818ace08eda63dc109524d78576c86
2018-02-13 22:26:01 +00:00
Adam Seaton a26a50ba69 [automerger] [DO NOT MERGE] Update Security String to 2018-04-01 on mnc-dev Bug:73240847 am: e4d86d9b06 am: 83709d3152 am: 70ea788dfa skipped: ac14372129 am: 83c448128c am: 50abbfdb53 am: 00a67ddab1 am: 2afc690751 skipped: 49f77737a2
am: b0aa7c342a

Change-Id: I4d5509274415bb8b27708cec7d7137b78e2c16aa
2018-02-13 22:21:23 +00:00
Tao Bao 6048121485 releasetools: Specify max_timestamp for secondary payload.
This is a mandatory field in the generated payload.

Bug: 35724498
Test: Generate OTA with --include_secondary. Check the generation
      command for secondary payload.
Change-Id: Ib2e25cc020e294eae5a4be36d6dca297432804b8
2018-02-13 14:17:17 -08:00
Tao Bao d1239cd452 Merge "releasetools: Specify SWITCH_SLOT_ON_REBOOT for secondary payload." am: 4f0b725439 am: 78eef74e03
am: 55e571f0b6

Change-Id: I0bb607e9ad51dd8624493a82c3d5c2b2167abd43
2018-02-13 22:12:24 +00:00
Tao Bao 55e571f0b6 Merge "releasetools: Specify SWITCH_SLOT_ON_REBOOT for secondary payload." am: 4f0b725439
am: 78eef74e03

Change-Id: I02007c63b6a7398afe8b5304e48c01589842a876
2018-02-13 22:06:40 +00:00
Tao Bao 78eef74e03 Merge "releasetools: Specify SWITCH_SLOT_ON_REBOOT for secondary payload."
am: 4f0b725439

Change-Id: Ic027da8aaab2f3d3034255a6726fd3a55063f647
2018-02-13 22:02:21 +00:00
Tao Bao 9452d84b97 Merge "releasetools: Allow generating BBOTA for images with shared blocks." 2018-02-13 21:53:25 +00:00
Tao Bao 6392e05813 Merge "releasetools: Remove the unconditional fallback to bsdiff." 2018-02-13 21:53:17 +00:00
Christopher Ferris f7352709ef Merge "Add libc_malloc_hook to list of libraries." 2018-02-13 21:21:32 +00:00
Pirama Arumuga Nainar 9c62fef547 Merge "Pass PGO_ADDITIONAL_PROFILE_DIRS to soong" am: 73e886ec62 am: 48fa1a2d87
am: 8aaf650a40

Change-Id: I9ef128d1598e5f614e2107b61721202f7f911483
2018-02-13 21:15:18 +00:00
Adam Seaton accee92300 [DO NOT MERGE]Update Security String to 2018-04-01 in oc-dev
Bug:73240847
2018-02-13 21:14:31 +00:00
Pirama Arumuga Nainar 8aaf650a40 Merge "Pass PGO_ADDITIONAL_PROFILE_DIRS to soong" am: 73e886ec62
am: 48fa1a2d87

Change-Id: I03b3f686736fa95d929fd3f44c65b1ab51e45ad6
2018-02-13 21:09:42 +00:00
Pirama Arumuga Nainar 48fa1a2d87 Merge "Pass PGO_ADDITIONAL_PROFILE_DIRS to soong"
am: 73e886ec62

Change-Id: I5f35840708b2716aa12df0198f8145da1afa1da5
2018-02-13 21:03:37 +00:00
Adam Seaton e44a1a8e88 Merge "Update Security String in Master to 2018-04-05 Bug:73240847" 2018-02-13 20:55:20 +00:00
Tao Bao 4f0b725439 Merge "releasetools: Specify SWITCH_SLOT_ON_REBOOT for secondary payload." 2018-02-13 20:20:30 +00:00
Treehugger Robot 73e886ec62 Merge "Pass PGO_ADDITIONAL_PROFILE_DIRS to soong" 2018-02-13 19:48:21 +00:00
Tao Bao e709b094e4 releasetools: Allow generating BBOTA for images with shared blocks.
When target defines 'BOARD_EXT4_SHARE_DUP_BLOCKS := true', the generated
system/vendor images may contain shared blocks (i.e. some blocks will
show up in multiple files' block list), which violates the current
assumptions in BBOTA script.

This CL allows generating BBOTAs by considering the first occurrence as
the "owner" of the shared blocks. All the later users of the shared
blocks will have an incomplete block list, whose RangeSet's will be
tagged with 'uses_shared_blocks'.

Files with 'uses_shared_blocks' tag will not be diff'd with imgdiff,
potentially with patch size penalty. Such files will be accounted for in
imgdiff stats report, where we can revisit for a better solution.

Bug: 64109868
Test: Generate BBOTA full and incremental package with targets defining
      'BOARD_EXT4_SHARE_DUP_BLOCKS := true'.
Change-Id: I87fbc22eef7fafe2a470a03fdcfa1babf088ea8d
2018-02-13 11:14:23 -08:00
Tao Bao 4ccea8549e releasetools: Remove the unconditional fallback to bsdiff.
This CL uses the 'incomplete' tag to skip applying imgdiff to files with
incomplete block list. It's not the ideal fix to address the holes in
ext4 images, but would unhide other imgdiff issues covered by the
unconditional fallback.

Bug: 68016761
Test: Generate an incremental OTA package from images with incomplete
      block list. Check the imgdiff stats report.
Test: `python -m unittest test_blockimgdiff`
Change-Id: Ice77686414e70f5e42de35c1757fb31cf02e4fd4
2018-02-13 11:06:20 -08:00
Tao Bao 3b67cdf291 Merge changes Ia7ecd0e1,I33982473 am: 4a066a2cd2 am: b25e43bfcb
am: 000274fda1

Change-Id: I360a5b5eeda4a3cfd9bd1145f36d030118eb8361
2018-02-13 17:36:22 +00:00
Pirama Arumuga Nainar 676fbe7e5e Pass PGO_ADDITIONAL_PROFILE_DIRS to soong
This variable can be set in BoardConfig.mk to specify a list of
additional paths that contain PGO profiles.  These directories are
searched before the predefined paths in soong/cc/pgo.go while finding
PGO profiles.

Test: Set this variable in a BoardConfig and verify that such profiles
are found and that these paths are searched before the predefined paths
in soong/cc/pgo.go.

Change-Id: Ied3ebf0ce6e6e2ead57f4828998e9617424e4573
2018-02-13 09:32:44 -08:00
Tao Bao 000274fda1 Merge changes Ia7ecd0e1,I33982473 am: 4a066a2cd2
am: b25e43bfcb

Change-Id: I52be4244cfb6faca30597a887a5abf7c5fed326f
2018-02-13 17:24:17 +00:00
Tao Bao b25e43bfcb Merge changes Ia7ecd0e1,I33982473
am: 4a066a2cd2

Change-Id: I07c077d4d174cdbdf6675e34680b3eb2705ee303
2018-02-13 17:20:19 +00:00
Tao Bao 4a066a2cd2 Merge changes Ia7ecd0e1,I33982473
* changes:
  releasetools: Make blockimgdiff.py pylint-clean.
  releasetools: Remove the global diff_done in blockimgdiff.py.
2018-02-13 17:13:17 +00:00
Chenbo Feng 53e5a438a8 Merge "Add bpfloader binary to system image" am: 5502f9958a am: 5880e5d7b6
am: 5443668bb0

Change-Id: Ia0112717e4a9dc55ae644f92fe071b9cbb18d2c7
2018-02-13 07:11:57 +00:00
Chenbo Feng 5443668bb0 Merge "Add bpfloader binary to system image" am: 5502f9958a
am: 5880e5d7b6

Change-Id: I0d28c3e36609f35cc187de50cb154095081d555e
2018-02-13 06:59:42 +00:00
Chenbo Feng 5880e5d7b6 Merge "Add bpfloader binary to system image"
am: 5502f9958a

Change-Id: Ifc233acb41850afb588d84551a8c30466fa9c4c6
2018-02-13 06:49:34 +00:00
Treehugger Robot 5502f9958a Merge "Add bpfloader binary to system image" 2018-02-13 05:49:46 +00:00
Christopher Ferris 8c70d4de97 Add libc_malloc_hook to list of libraries.
Bug: 30561479

Test: Build from scratch and verify the library is present.
Change-Id: I53f30d694cff202ffc93491485ed3c845de07862
2018-02-13 04:09:44 +00:00
Adam Seaton a9d30a23d3 Update Security String in Master to 2018-04-05
Bug:73240847
2018-02-12 13:13:41 -08:00
Tao Bao 508b087943 releasetools: Make blockimgdiff.py pylint-clean.
************* Module blockimgdiff
C:433, 0: Unnecessary parens after 'if' keyword (superfluous-parens)
C:687, 0: Wrong hanging indentation (add 4 spaces).
            max_stashed_blocks, self._max_stashed_size, max_allowed,
            ^   | (bad-continuation)
C:688, 0: Wrong hanging indentation (add 4 spaces).
            self._max_stashed_size * 100.0 / max_allowed))
            ^   | (bad-continuation)
C:691, 0: Wrong hanging indentation (remove 2 spaces).
            max_stashed_blocks, self._max_stashed_size))
          | ^ (bad-continuation)
C:898, 0: Wrong hanging indentation (add 4 spaces).
                  "imgdiff" if imgdiff else "bsdiff",
                  ^   | (bad-continuation)
C:899, 0: Wrong hanging indentation (add 4 spaces).
                  xf.tgt_name if xf.tgt_name == xf.src_name else
                  ^   | (bad-continuation)
C:901, 0: Wrong hanging indentation (add 4 spaces).
                  xf.tgt_ranges, xf.src_ranges, e.message))
                  ^   | (bad-continuation)
C:909, 0: Wrong hanging indentation (add 4 spaces).
                      xf.tgt_name,))
                      ^   | (bad-continuation)
C:917, 0: Wrong hanging indentation (add 4 spaces).
                      xf.tgt_name, e.message))
                      ^   | (bad-continuation)
C:961, 0: Wrong hanging indentation (remove 2 spaces).
                xf.patch_len, tgt_size, xf.patch_len * 100.0 / tgt_size,
              | ^ (bad-continuation)
C:962, 0: Wrong hanging indentation (remove 2 spaces).
                xf.style,
              | ^ (bad-continuation)
C:963, 0: Wrong hanging indentation (remove 2 spaces).
                xf.tgt_name if xf.tgt_name == xf.src_name else (
              | ^ (bad-continuation)
C:965, 0: Wrong hanging indentation (remove 2 spaces).
                xf.tgt_ranges, xf.src_ranges))
              | ^ (bad-continuation)
C:1422, 0: Wrong continued indentation (add 28 spaces).
                tgt_skipped.size() * 100.0 / tgt_size, tgt_name))
                ^                           | (bad-continuation)
C:1560, 0: Wrong continued indentation (add 8 spaces).
            split_src_ranges) in enumerate(split_info_list):
            ^       | (bad-continuation)
R:566, 6: Redefinition of src_str type from list to str (redefined-variable-type)
C:1198,25: More than one statement on a single line (multiple-statements)
C:1211,25: More than one statement on a single line (multiple-statements)
C:1220,16: More than one statement on a single line (multiple-statements)
C:1277,38: More than one statement on a single line (multiple-statements)
C:1284,19: More than one statement on a single line (multiple-statements)
C: 19, 0: standard import "import copy" comes before "import common" (wrong-import-order)
C: 20, 0: standard import "import functools" comes before "import common" (wrong-import-order)
C: 21, 0: standard import "import heapq" comes before "import common" (wrong-import-order)
C: 22, 0: standard import "import itertools" comes before "import common" (wrong-import-order)
C: 23, 0: standard import "import multiprocessing" comes before "import common" (wrong-import-order)
C: 24, 0: standard import "import os" comes before "import common" (wrong-import-order)
C: 25, 0: standard import "import os.path" comes before "import common" (wrong-import-order)
C: 26, 0: standard import "import re" comes before "import common" (wrong-import-order)
C: 27, 0: standard import "import subprocess" comes before "import common" (wrong-import-order)
C: 28, 0: standard import "import sys" comes before "import common" (wrong-import-order)
C: 29, 0: standard import "import threading" comes before "import common" (wrong-import-order)
C: 31, 0: standard import "from collections import deque, OrderedDict" comes before "import common" (wrong-import-order)
C: 32, 0: standard import "from hashlib import sha1" comes before "import common" (wrong-import-order)

Test: `pylint --rcfile=pylintrc blockimgdiff.py`
Test: Generate an incremental BBOTA package.
Change-Id: Ia7ecd0e1fa48daf4e43251bdcdfcd08fb316015d
2018-02-12 13:01:34 -08:00
Tao Bao 9739514769 releasetools: Remove the global diff_done in blockimgdiff.py.
pylint complains about undefined `diff_done`:

W:754, 8: Global variable 'diff_done' undefined at the module level (global-variable-undefined)
W:820,14: Global variable 'diff_done' undefined at the module level (global-variable-undefined)

It would still warn about using global statement after adding the
definition.

W:859, 8: Using the global statement (global-statement)
W:925,14: Using the global statement (global-statement)

This CL computes 'diff_done' via 'len(diff_queue)' instead. It also
moves the progress reporting _before_ the diff work. This way it avoids
showing 100% progress with still changing filenames (because multiple
workers could see an empty queue simultaneously upon finishing their own
works).

There're possible alternatives, such as using the 'nonlocal' keyword in
Python 3 (which we're not there yet), or by using mutable object instead
(e.g. 'diff_done = [0]'). This CL looks cleaner, since it just kills the
var.

Test: Generate a BBOTA incremental. Check the on-screen progress
      report.
Test: `pylint --rcfile=pylintrc blockimgdiff.py` no longer complains
      about the global diff_done.
Change-Id: I339824735527e1f794b5b1dc99ff3fdb2da85744
2018-02-12 12:55:25 -08:00
Tao Bao a3ee598230 Merge "releasetools: Add an ImgdiffStats class that reports imgdiff stats." am: 7eb2afb226 am: 0ced5030d8
am: e6188063f1

Change-Id: I34118e0e88042ef8477a071f6819483746b0038c
2018-02-12 20:48:54 +00:00