Commit Graph

42120 Commits

Author SHA1 Message Date
Adam Seaton 90b42aa3db [automerger] [DO NOT MERGE] Update Security String to 2018-04-01 on mnc-dev Bug:73240847 am: e4d86d9b06 am: 83709d3152 am: 70ea788dfa skipped: ac14372129 am: 83c448128c am: 50abbfdb53 am: 00a67ddab1 am: 2afc690751 skipped: 49f77737a2 am: b0aa7c342a am: a26a50ba69
am: 2d955bd781

Change-Id: Ifec0f858474b1fe38bcd0bc32821cdcbb66c73b0
2018-02-13 22:32:04 +00:00
Adam Seaton 2d955bd781 [automerger] [DO NOT MERGE] Update Security String to 2018-04-01 on mnc-dev Bug:73240847 am: e4d86d9b06 am: 83709d3152 am: 70ea788dfa skipped: ac14372129 am: 83c448128c am: 50abbfdb53 am: 00a67ddab1 am: 2afc690751 skipped: 49f77737a2 am: b0aa7c342a
am: a26a50ba69

Change-Id: Id6ac55b836818ace08eda63dc109524d78576c86
2018-02-13 22:26:01 +00:00
Adam Seaton a26a50ba69 [automerger] [DO NOT MERGE] Update Security String to 2018-04-01 on mnc-dev Bug:73240847 am: e4d86d9b06 am: 83709d3152 am: 70ea788dfa skipped: ac14372129 am: 83c448128c am: 50abbfdb53 am: 00a67ddab1 am: 2afc690751 skipped: 49f77737a2
am: b0aa7c342a

Change-Id: I4d5509274415bb8b27708cec7d7137b78e2c16aa
2018-02-13 22:21:23 +00:00
Tao Bao 55e571f0b6 Merge "releasetools: Specify SWITCH_SLOT_ON_REBOOT for secondary payload." am: 4f0b725439
am: 78eef74e03

Change-Id: I02007c63b6a7398afe8b5304e48c01589842a876
2018-02-13 22:06:40 +00:00
Tao Bao 78eef74e03 Merge "releasetools: Specify SWITCH_SLOT_ON_REBOOT for secondary payload."
am: 4f0b725439

Change-Id: Ic027da8aaab2f3d3034255a6726fd3a55063f647
2018-02-13 22:02:21 +00:00
Pirama Arumuga Nainar 8aaf650a40 Merge "Pass PGO_ADDITIONAL_PROFILE_DIRS to soong" am: 73e886ec62
am: 48fa1a2d87

Change-Id: I03b3f686736fa95d929fd3f44c65b1ab51e45ad6
2018-02-13 21:09:42 +00:00
Pirama Arumuga Nainar 48fa1a2d87 Merge "Pass PGO_ADDITIONAL_PROFILE_DIRS to soong"
am: 73e886ec62

Change-Id: I5f35840708b2716aa12df0198f8145da1afa1da5
2018-02-13 21:03:37 +00:00
Tao Bao 4f0b725439 Merge "releasetools: Specify SWITCH_SLOT_ON_REBOOT for secondary payload." 2018-02-13 20:20:30 +00:00
Treehugger Robot 73e886ec62 Merge "Pass PGO_ADDITIONAL_PROFILE_DIRS to soong" 2018-02-13 19:48:21 +00:00
Pirama Arumuga Nainar 676fbe7e5e Pass PGO_ADDITIONAL_PROFILE_DIRS to soong
This variable can be set in BoardConfig.mk to specify a list of
additional paths that contain PGO profiles.  These directories are
searched before the predefined paths in soong/cc/pgo.go while finding
PGO profiles.

Test: Set this variable in a BoardConfig and verify that such profiles
are found and that these paths are searched before the predefined paths
in soong/cc/pgo.go.

Change-Id: Ied3ebf0ce6e6e2ead57f4828998e9617424e4573
2018-02-13 09:32:44 -08:00
Tao Bao 000274fda1 Merge changes Ia7ecd0e1,I33982473 am: 4a066a2cd2
am: b25e43bfcb

Change-Id: I52be4244cfb6faca30597a887a5abf7c5fed326f
2018-02-13 17:24:17 +00:00
Tao Bao b25e43bfcb Merge changes Ia7ecd0e1,I33982473
am: 4a066a2cd2

Change-Id: I07c077d4d174cdbdf6675e34680b3eb2705ee303
2018-02-13 17:20:19 +00:00
Tao Bao 4a066a2cd2 Merge changes Ia7ecd0e1,I33982473
* changes:
  releasetools: Make blockimgdiff.py pylint-clean.
  releasetools: Remove the global diff_done in blockimgdiff.py.
2018-02-13 17:13:17 +00:00
Chenbo Feng 5443668bb0 Merge "Add bpfloader binary to system image" am: 5502f9958a
am: 5880e5d7b6

Change-Id: I0d28c3e36609f35cc187de50cb154095081d555e
2018-02-13 06:59:42 +00:00
Chenbo Feng 5880e5d7b6 Merge "Add bpfloader binary to system image"
am: 5502f9958a

Change-Id: Ifc233acb41850afb588d84551a8c30466fa9c4c6
2018-02-13 06:49:34 +00:00
Treehugger Robot 5502f9958a Merge "Add bpfloader binary to system image" 2018-02-13 05:49:46 +00:00
Tao Bao 508b087943 releasetools: Make blockimgdiff.py pylint-clean.
************* Module blockimgdiff
C:433, 0: Unnecessary parens after 'if' keyword (superfluous-parens)
C:687, 0: Wrong hanging indentation (add 4 spaces).
            max_stashed_blocks, self._max_stashed_size, max_allowed,
            ^   | (bad-continuation)
C:688, 0: Wrong hanging indentation (add 4 spaces).
            self._max_stashed_size * 100.0 / max_allowed))
            ^   | (bad-continuation)
C:691, 0: Wrong hanging indentation (remove 2 spaces).
            max_stashed_blocks, self._max_stashed_size))
          | ^ (bad-continuation)
C:898, 0: Wrong hanging indentation (add 4 spaces).
                  "imgdiff" if imgdiff else "bsdiff",
                  ^   | (bad-continuation)
C:899, 0: Wrong hanging indentation (add 4 spaces).
                  xf.tgt_name if xf.tgt_name == xf.src_name else
                  ^   | (bad-continuation)
C:901, 0: Wrong hanging indentation (add 4 spaces).
                  xf.tgt_ranges, xf.src_ranges, e.message))
                  ^   | (bad-continuation)
C:909, 0: Wrong hanging indentation (add 4 spaces).
                      xf.tgt_name,))
                      ^   | (bad-continuation)
C:917, 0: Wrong hanging indentation (add 4 spaces).
                      xf.tgt_name, e.message))
                      ^   | (bad-continuation)
C:961, 0: Wrong hanging indentation (remove 2 spaces).
                xf.patch_len, tgt_size, xf.patch_len * 100.0 / tgt_size,
              | ^ (bad-continuation)
C:962, 0: Wrong hanging indentation (remove 2 spaces).
                xf.style,
              | ^ (bad-continuation)
C:963, 0: Wrong hanging indentation (remove 2 spaces).
                xf.tgt_name if xf.tgt_name == xf.src_name else (
              | ^ (bad-continuation)
C:965, 0: Wrong hanging indentation (remove 2 spaces).
                xf.tgt_ranges, xf.src_ranges))
              | ^ (bad-continuation)
C:1422, 0: Wrong continued indentation (add 28 spaces).
                tgt_skipped.size() * 100.0 / tgt_size, tgt_name))
                ^                           | (bad-continuation)
C:1560, 0: Wrong continued indentation (add 8 spaces).
            split_src_ranges) in enumerate(split_info_list):
            ^       | (bad-continuation)
R:566, 6: Redefinition of src_str type from list to str (redefined-variable-type)
C:1198,25: More than one statement on a single line (multiple-statements)
C:1211,25: More than one statement on a single line (multiple-statements)
C:1220,16: More than one statement on a single line (multiple-statements)
C:1277,38: More than one statement on a single line (multiple-statements)
C:1284,19: More than one statement on a single line (multiple-statements)
C: 19, 0: standard import "import copy" comes before "import common" (wrong-import-order)
C: 20, 0: standard import "import functools" comes before "import common" (wrong-import-order)
C: 21, 0: standard import "import heapq" comes before "import common" (wrong-import-order)
C: 22, 0: standard import "import itertools" comes before "import common" (wrong-import-order)
C: 23, 0: standard import "import multiprocessing" comes before "import common" (wrong-import-order)
C: 24, 0: standard import "import os" comes before "import common" (wrong-import-order)
C: 25, 0: standard import "import os.path" comes before "import common" (wrong-import-order)
C: 26, 0: standard import "import re" comes before "import common" (wrong-import-order)
C: 27, 0: standard import "import subprocess" comes before "import common" (wrong-import-order)
C: 28, 0: standard import "import sys" comes before "import common" (wrong-import-order)
C: 29, 0: standard import "import threading" comes before "import common" (wrong-import-order)
C: 31, 0: standard import "from collections import deque, OrderedDict" comes before "import common" (wrong-import-order)
C: 32, 0: standard import "from hashlib import sha1" comes before "import common" (wrong-import-order)

Test: `pylint --rcfile=pylintrc blockimgdiff.py`
Test: Generate an incremental BBOTA package.
Change-Id: Ia7ecd0e1fa48daf4e43251bdcdfcd08fb316015d
2018-02-12 13:01:34 -08:00
Tao Bao 9739514769 releasetools: Remove the global diff_done in blockimgdiff.py.
pylint complains about undefined `diff_done`:

W:754, 8: Global variable 'diff_done' undefined at the module level (global-variable-undefined)
W:820,14: Global variable 'diff_done' undefined at the module level (global-variable-undefined)

It would still warn about using global statement after adding the
definition.

W:859, 8: Using the global statement (global-statement)
W:925,14: Using the global statement (global-statement)

This CL computes 'diff_done' via 'len(diff_queue)' instead. It also
moves the progress reporting _before_ the diff work. This way it avoids
showing 100% progress with still changing filenames (because multiple
workers could see an empty queue simultaneously upon finishing their own
works).

There're possible alternatives, such as using the 'nonlocal' keyword in
Python 3 (which we're not there yet), or by using mutable object instead
(e.g. 'diff_done = [0]'). This CL looks cleaner, since it just kills the
var.

Test: Generate a BBOTA incremental. Check the on-screen progress
      report.
Test: `pylint --rcfile=pylintrc blockimgdiff.py` no longer complains
      about the global diff_done.
Change-Id: I339824735527e1f794b5b1dc99ff3fdb2da85744
2018-02-12 12:55:25 -08:00
Tao Bao e6188063f1 Merge "releasetools: Add an ImgdiffStats class that reports imgdiff stats." am: 7eb2afb226
am: 0ced5030d8

Change-Id: I120d8d268fc4a5e1230e16d9b9ab85b2bef20193
2018-02-12 20:28:11 +00:00
Tao Bao 0ced5030d8 Merge "releasetools: Add an ImgdiffStats class that reports imgdiff stats."
am: 7eb2afb226

Change-Id: I9d87494c9a3e8784128f152b5f16bf67865bdd30
2018-02-12 20:14:37 +00:00
Android Build Merger (Role) b0aa7c342a [automerger] [DO NOT MERGE] Update Security String to 2018-04-01 on mnc-dev Bug:73240847 am: e4d86d9b06 am: 83709d3152 am: 70ea788dfa skipped: ac14372129 am: 83c448128c am: 50abbfdb53 am: 00a67ddab1 am: 2afc690751 skipped: 49f77737a2
Change-Id: I45b510b29678883a6d89fd63960df9df4c84f9f5
2018-02-12 19:29:30 +00:00
Android Build Merger (Role) 49f77737a2 [automerger] [DO NOT MERGE] Update Security String to 2018-04-01 on mnc-dev Bug:73240847 am: e4d86d9b06 am: 83709d3152 am: 70ea788dfa skipped: ac14372129 am: 83c448128c am: 50abbfdb53 am: 00a67ddab1 am: 2afc690751
Change-Id: I41804bc2f8485c6b89708a3a7e82cb8f49fd37be
2018-02-12 19:29:27 +00:00
Android Build Merger (Role) 2afc690751 [automerger] [DO NOT MERGE] Update Security String to 2018-04-01 on mnc-dev Bug:73240847 am: e4d86d9b06 am: 83709d3152 am: 70ea788dfa skipped: ac14372129 am: 83c448128c am: 50abbfdb53 am: 00a67ddab1
Change-Id: I5bda5bccef672c7e713545801701481163d81734
2018-02-12 19:29:23 +00:00
Android Build Merger (Role) 00a67ddab1 [automerger] [DO NOT MERGE] Update Security String to 2018-04-01 on mnc-dev Bug:73240847 am: e4d86d9b06 am: 83709d3152 am: 70ea788dfa skipped: ac14372129 am: 83c448128c am: 50abbfdb53
Change-Id: I3afc37971e979b561cd4ead55a7f04d456526600
2018-02-12 19:29:20 +00:00
Android Build Merger (Role) 50abbfdb53 [automerger] [DO NOT MERGE] Update Security String to 2018-04-01 on mnc-dev Bug:73240847 am: e4d86d9b06 am: 83709d3152 am: 70ea788dfa skipped: ac14372129 am: 83c448128c
Change-Id: I2bd8d8265cf136682fa5faba753979ad6ff2dda8
2018-02-12 19:29:17 +00:00
Android Build Merger (Role) 83c448128c [automerger] [DO NOT MERGE] Update Security String to 2018-04-01 on mnc-dev Bug:73240847 am: e4d86d9b06 am: 83709d3152 am: 70ea788dfa skipped: ac14372129
Change-Id: I47eefa3ba40b22e9f03a510bb6f03b04f0d18aef
2018-02-12 19:29:13 +00:00
Android Build Merger (Role) ac14372129 [automerger] [DO NOT MERGE] Update Security String to 2018-04-01 on mnc-dev Bug:73240847 am: e4d86d9b06 am: 83709d3152 am: 70ea788dfa
Change-Id: Ia9c78b05a58e5dddd2e90a6bf53b638ddbccecf1
2018-02-12 19:29:10 +00:00
Android Build Merger (Role) 70ea788dfa [automerger] [DO NOT MERGE] Update Security String to 2018-04-01 on mnc-dev Bug:73240847 am: e4d86d9b06 am: 83709d3152
Change-Id: I970dd7c49833638b34d415ee962b60a13d382c0c
2018-02-12 19:29:07 +00:00
Android Build Merger (Role) 83709d3152 [automerger] [DO NOT MERGE] Update Security String to 2018-04-01 on mnc-dev Bug:73240847 am: e4d86d9b06
Change-Id: I8dd3706df6cb68d4e7c7dc47bb218e3bc0e5afec
2018-02-12 19:29:03 +00:00
Adam Seaton e4d86d9b06 [DO NOT MERGE] Update Security String to 2018-04-01 on mnc-dev
Bug:73240847
2018-02-12 11:27:35 -08:00
Tao Bao 7eb2afb226 Merge "releasetools: Add an ImgdiffStats class that reports imgdiff stats." 2018-02-12 19:01:24 +00:00
Tom Cherry c2ab6232d9 Merge "Place TARGET_FS_CONFIG_GEN passwd/group files in /vendor/etc" am: 3d9fa3615d
am: 57dd54f623

Change-Id: Ica4d341247fc0f770ccdfafd165df05d70f39772
2018-02-12 16:38:43 +00:00
Tom Cherry 57dd54f623 Merge "Place TARGET_FS_CONFIG_GEN passwd/group files in /vendor/etc"
am: 3d9fa3615d

Change-Id: I418fa2dd55aadcbb4eb013c83559ac02a0a29f22
2018-02-12 16:33:01 +00:00
Tom Cherry 3d9fa3615d Merge "Place TARGET_FS_CONFIG_GEN passwd/group files in /vendor/etc" 2018-02-12 16:27:20 +00:00
Tao Bao 667ff57272 releasetools: Specify SWITCH_SLOT_ON_REBOOT for secondary payload.
The secondary payload should always be applied with
SWITCH_SLOT_ON_REBOOT=0. This CL moves the 'secondary' parameter from
Payload.WriteToZip() to Payload.__init__(). So it can append the flag to
secondary/payload_properties.txt.

Bug: 35724498
Test: Generate an A/B OTA with --include_secondary. Check
      secondary/payload_properties.txt entry in the generated ZIP.
Test: `python -m unittest test_ota_from_target_files`
Change-Id: I816c07ab57a1c8a52eff785801634b8b1cb134d4
2018-02-10 00:13:07 -08:00
Tao Bao 294651d7b4 releasetools: Add an ImgdiffStats class that reports imgdiff stats.
We have a couple of active imgdiff workarounds (and likely with one more
inbounding that allows having shared blocks in ext4 image). Most of
these workarounds need extending imgdiff's capability. While us not
getting there anytime soon, collect the stats to better understand the
impact of each kind so we can prioritize accordingly.

A sample report is as follows.

  Imgdiff Stats Report
========================

 APK files diff'd with imgdiff (count: 88)
-------------------------------------------

  /system/priv-app/Shell/Shell.apk
  ...

 Large APK files split and diff'd with imgdiff (count: 4)
----------------------------------------------------------

  /system/priv-app/Settings/Settings.apk
  ...

Bug: 68016761
Test: Generate an incremental BBOTA package. Check the stats report.
Test: python -m unittest test_blockimgdiff
Change-Id: I27ad862cde472ab2806db877632ce5a0607420f2
2018-02-09 23:02:32 -08:00
Tao Bao 194e49e06c Merge "releasetools: Refactor the condition checking for using imgdiff." am: bec8be51a5
am: 4a79eb0d4e

Change-Id: Ia050150657bb373e92bf9797764ef987493e9c96
2018-02-10 02:16:54 +00:00
Tao Bao 4a79eb0d4e Merge "releasetools: Refactor the condition checking for using imgdiff."
am: bec8be51a5

Change-Id: I4cabec977a8a6d963c707d95dff398be2aff3140
2018-02-10 02:00:38 +00:00
Tao Bao bec8be51a5 Merge "releasetools: Refactor the condition checking for using imgdiff." 2018-02-10 00:47:42 +00:00
Chenbo Feng 2b948a2259 Add bpfloader binary to system image
The bpfloader is a executable binary used by netd to load and attach
networking related bpf program at run time. It need to be flashed to
device when updating the system image. Currently this binary is only
used by devices with kernel version 4.9 or above.

Test: hikey boot and bpf program show up under sys/fs/bpf
Bug: 30950746
Change-Id: I3d14d270a04ea57319ad4292ff51b1cf933fc975
2018-02-09 15:17:22 -08:00
Tao Bao cb73aed1f0 releasetools: Refactor the condition checking for using imgdiff.
In Transfer class, unbundle 'intact' with the monotonicity of the input
ranges. Negate the logic of 'intact', and thus rename it to 'trimmed'.
Move this property from an attribute of Transfer class as the one in
RangeSet.extra. 'trimmed' indicates whether the source / target ranges
have been modified after creating the Transfer() instance.

The logic that determines whether we can apply imgdiff has been
refactored and consolidated into BlockImageDiff.CanUseImgdiff(). Now
both of the two paths call this single copy, i.e. the one that detects
large APKs (before creating Transfer()'s), and the other that's about to
generate the patch for a given Transfer instance.

Bug: 68016761
Test: python -m unittest test_blockimgdiff
Test: Generate an incremental BBOTA package.
Change-Id: Id07195f63f1fa6c3af6e9091940d251cf09fa104
2018-02-09 12:46:01 -08:00
Dan Willemsen ea121f2a9a Merge "Revert "Propagate proguard rules from LOCAL_STATIC_ANDROID_LIBRARIES"" am: 2744cbf190
am: 22eb731606

Change-Id: Ibfa7d19f060c28bf3f44ca284275f2d6c7c7e904
2018-02-09 19:44:15 +00:00
Dan Willemsen 22eb731606 Merge "Revert "Propagate proguard rules from LOCAL_STATIC_ANDROID_LIBRARIES""
am: 2744cbf190

Change-Id: I276e0f9be16d88d4c39d1c4a5dd177cf5aae9f32
2018-02-09 19:38:38 +00:00
Dan Willemsen 2744cbf190 Merge "Revert "Propagate proguard rules from LOCAL_STATIC_ANDROID_LIBRARIES"" 2018-02-09 19:22:13 +00:00
Dan Willemsen 30d6869faf Revert "Propagate proguard rules from LOCAL_STATIC_ANDROID_LIBRARIES"
This reverts commit df5d7da7d8.

Reason for revert: breaking builds

Bug: 72969557
Change-Id: I27aad65587933c3f760a89c541d27469c916abe8
2018-02-09 19:21:11 +00:00
Tom Cherry af0a12eded Merge "Add compile time check that friendly AID names are < 32 characters" am: 9f25d873bf
am: cfac68eb9c

Change-Id: I52c488b8fc0b6e42ef379b517cc816fc8e759738
2018-02-09 19:00:04 +00:00
Tom Cherry cfac68eb9c Merge "Add compile time check that friendly AID names are < 32 characters"
am: 9f25d873bf

Change-Id: Iace2d0294a7ca5318d568614b021a13713a72a78
2018-02-09 18:53:56 +00:00
Tom Cherry 9f25d873bf Merge "Add compile time check that friendly AID names are < 32 characters" 2018-02-09 18:46:27 +00:00
Tom Cherry ee0610e86c Add compile time check that friendly AID names are < 32 characters
There is an internal buffer in bionic for user/group names that is 32
characters long including the trailing null, so we must restrict the
length of these names during compile time.

Bug: 27999086
Test: Successfully compile a valid config.fs
Test: Fail to compile a config.fs with AID name > 31 characters
Change-Id: I7fe887c630dd4d1033b86a5d8332480eb3b0fa07
2018-02-09 18:46:13 +00:00
Tom Cherry 55d0104f1b Place TARGET_FS_CONFIG_GEN passwd/group files in /vendor/etc
These entries are vendor provided and belong on the /vendor partition.

Bug: 27999086
Test: end to end user/group check via config.fs and getpwnam, etc.
Change-Id: I9a5d56da594bf0d04de2b9ce7fd7d9a8151d4682
2018-02-09 10:44:52 -08:00