Commit Graph

38641 Commits

Author SHA1 Message Date
Elliott Hughes c8a8771b6b Merge "Make ziparchive-tests run standalone." 2018-04-25 21:38:13 +00:00
Tim Murray 419ba9e2aa cutils: add restricted cpuset
(cherrypick of 658ae90f69 without the init.rc change)

Bug: 78197570
Test: CTS
Change-Id: I6df972950b75a839caa463ae282ad000b959e8ae
2018-04-25 14:30:37 -07:00
Elliott Hughes bde0ec9e00 Make ziparchive-tests run standalone.
We need to (a) tell soong to copy our data and (b) automatically find
our data relative to our executable.

The real point of this is to be able to run these tests in APCT and
presubmit.

Bug: N/A
Test: ran tests on host and device, from a variety of directories
Change-Id: I4c0be1ac60f03953fdd5ba6e3d15b1aaa37ed019
2018-04-25 12:50:17 -07:00
Paul Crowley f88905d231 Merge "Wipe metadata when we wipe userdata and cache." 2018-04-25 19:42:56 +00:00
Paul Crowley 4d17006b57 Wipe metadata when we wipe userdata and cache.
Bug: 78469699
Test: Run "fastboot -w" and see that metadata is wiped
Change-Id: I2d795281c48b59c87d4f8f010037d95186f55684
2018-04-25 10:51:05 -07:00
Tao Bao 7d27ffdd45 fastboot: Track the libsparse API change.
Bug: 78432315
Test: Successfully flash a previously failing (huge) sparse image.
Test: `fastboot update` existing marlin-img.zip.
Change-Id: I09c9a06109769882d26be56d4a0d2a2b7b62cb5f
2018-04-25 10:37:01 -07:00
Tao Bao e18c03165b libsparse: Use 'size_t' for the 'len' parameter in callbacks.
This CL updates the callback function signature in
sparse_file_callback() and sparse_file_foreach_chunk().

Before:
int sparse_file_callback(
    struct sparse_file *s, bool sparse, bool crc,
    int (*write)(void *priv, const void *data, int len), void *priv);

int sparse_file_foreach_chunk(
    struct sparse_file *s, bool sparse, bool crc,
    int (*write)(
        void *priv, const void *data, int len, unsigned int block,
        unsigned int nr_blocks),
    void *priv);

After:
int sparse_file_callback(
    struct sparse_file *s, bool sparse, bool crc,
    int (*write)(void *priv, const void *data, size_t len), void *priv);

int sparse_file_foreach_chunk(
    struct sparse_file *s, bool sparse, bool crc,
    int (*write)(
        void *priv, const void *data, size_t len, unsigned int block,
        unsigned int nr_blocks),
    void *priv);

The length (i.e. 'len') comes from the size of a chunk, which could be
legitimately larger than INT_MAX. Prior to this CL, callers (e.g.
write_sparse_data_chunk()) were already passing unsigned int to the
callbacks. When a chunk size exceeds INT_MAX, the callback would see a
negative value, which could lead to undesired behavior. For example,
out_counter_write(), as one of the internal callbacks in libsparse,
gives a wrong sum of chunk sizes, which in turn fails the fastboot
flashing when given a huge sparse image.

It also defines SPARSE_CALLBACK_USES_SIZE_T that allows clients to keep
their codes compatibile with both versions.

Bug: 78432315
Test: `m dist` (with matching changes to all the clients)
Test: Build fastboot and successfully flash a previously failing (huge)
      sparse image.
Change-Id: Iac4bcf7b57039d08af3c57f4be00d75f6b693d93
2018-04-25 10:29:22 -07:00
Tom Cherry 6a2686b0f3 Merge "logd: notify kernel log readers of new messages" 2018-04-25 16:40:59 +00:00
Tao Bao 9249f4bc41 Merge "fastboot: sparse_file_len() returns int64_t." 2018-04-25 13:32:43 +00:00
Tao Bao 41cf35f8b2 fastboot: sparse_file_len() returns int64_t.
Check that the value fits in uint32_t that's supported by the current
protocol.

Also fix and sanity check the max_size before passing it to
sparse_file_resparse(), which accepts `unsigned int`. This shouldn't
happen in practice because of RESPARSE_LIMIT (1 GiB).

Test: `fastboot flash` with small and large images.
Change-Id: I0a8279fc14c54c40a70ddce65c3b25173c0d0a40
2018-04-24 18:10:42 -07:00
Treehugger Robot 9feca7c249 Merge "init: clarify documentation for disabled" 2018-04-25 01:00:54 +00:00
Steven Moreland 8cb36868a4 init: clarify documentation for disabled
Services can be started by name or interface name now.

Bug: N/A
Test: N/A
Change-Id: I9fef619e7c27d458193311f7cd7fca4dcf8c8e72
2018-04-24 16:24:58 -07:00
Tom Cherry cbfebdbadb logd: notify kernel log readers of new messages
LogBuffer::log() returns either a negative number on error or a
positive number indicating the length of the message written.
Therefore, the check to notify kernel log readers of a new message
should be that this function's return value is > 0.

Bug: 78209416
Test: `adb logcat -b kernel` updates when new log messages are present
Change-Id: Icc18c0c22e62340994e5c26aedb72282d61c1541
2018-04-24 14:19:32 -07:00
Elliott Hughes 9aca01907d Merge "Switch to PCRE grep." 2018-04-24 15:08:57 +00:00
Elliott Hughes a366494fc3 Remove misleading comment.
Bug: N/A
Test: N/A
Change-Id: Ie93d9d0abb13bf204c62f6951d3a7bae650f8e08
2018-04-23 21:50:43 -07:00
Elliott Hughes 90a018a87a Switch to PCRE grep.
Bug: http://b/77150016
Test: boots, manual testing
Change-Id: I377ca42ed9783edcceab4daabe43e993ef74e837
2018-04-23 17:00:54 -07:00
Treehugger Robot ca07de4214 Merge "Add a public.libraries.txt file for iot devices." 2018-04-23 23:20:42 +00:00
Treehugger Robot a3721db333 Merge "Document the switch to toybox dd." 2018-04-21 19:25:42 +00:00
Elliott Hughes 9edec24a2f Document the switch to toybox dd.
Bug: N/A
Test: N/A
Change-Id: Ia1fd8a14ceb9d9e8661804a126b3818cc066b0ff
2018-04-21 10:49:17 -07:00
Treehugger Robot 75d3592cb9 Merge "Remove BSD dd." 2018-04-21 02:31:56 +00:00
Treehugger Robot eb1b202acb Merge "libdebuggerd: fix out of bounds write." 2018-04-21 02:21:55 +00:00
Treehugger Robot 3e378800b8 Merge changes I4b017701,I28aff510
* changes:
  libdebuggerd: clone registers before we Unwind with them.
  libunwindstack: add Regs::Clone.
2018-04-21 02:15:45 +00:00
Treehugger Robot a7112f4c9b Merge "Mark libziparchive as double_loadable" 2018-04-21 02:14:51 +00:00
Elliott Hughes 21ceafbaac Remove BSD dd.
Bug: N/A
Test: builds
Change-Id: Id80364241434abfc216149b4ce7440cef700abef
2018-04-20 17:55:44 -07:00
Josh Gao 83b8ac24b6 libdebuggerd: fix out of bounds write.
Bug: http://b/78363964
Test: treehugger
Change-Id: I79ae818e4e0c29f064335d59789afc3b1bf87c69
2018-04-20 17:46:50 -07:00
Treehugger Robot b1677c15c2 Merge "Lose dd to toybox." 2018-04-20 22:33:27 +00:00
Treehugger Robot 481a8125a6 Merge "llkd: bootstat: propagate detailed livelock canonical boot reason" 2018-04-20 21:09:04 +00:00
Treehugger Robot 1803782064 Merge "llkd: default enabled for userdebug" 2018-04-20 21:09:00 +00:00
Ralph Nathan ab39ce88e4 Add a public.libraries.txt file for iot devices.
Previously on Android Things, we used a vendor public.libraries.txt
file. This cl enables us to tag our library correctly.

Bug: 78226207
Test: test app works
Change-Id: I9e69717e5968a903e84f59c1d889c8e4cf9bcc35
2018-04-20 13:46:55 -07:00
Treehugger Robot 656bc24630 Merge "llkd: add live-lock daemon" 2018-04-20 20:46:41 +00:00
Elliott Hughes 8591440fe2 Lose dd to toybox.
Just the symlink for now. If this sticks, I'll come back to remove the code.

Bug: N/A
Test: `toolbox dd --help`
Change-Id: I9b967e9246f42db0d1f48d9d147c538d57fb3bc8
2018-04-20 13:34:20 -07:00
Josh Gao 5d1c14f41b libdebuggerd: clone registers before we Unwind with them.
Bug: http://b/77296294
Test: manual inspection of tombstone generated by crasher
Change-Id: I4b017701d7d2041db4aefbbb90977e99b844c328
2018-04-20 13:23:20 -07:00
Josh Gao 2f37a15472 libunwindstack: add Regs::Clone.
Add a way to copy a Regs object.

Bug: http://b/77296294
Test: libunwindstack_test
Change-Id: I28aff510aa3e1b677d5ae46dc3bfe7652817ce52
2018-04-20 13:23:20 -07:00
Elliott Hughes 395131df7b Merge changes I1f55c6e1,Ib2adcf0a
* changes:
  Move fastboot over to ParseByteCount.
  Add android::base::ParseByteCount.
2018-04-20 20:15:30 +00:00
Elliott Hughes 542370dead Move fastboot over to ParseByteCount.
Bug: N/A
Test: `fastboot flashall` and `fastboot -S 100m flashall`
Change-Id: I1f55c6e10be74780bbf8dc30b2d4be3f362ac662
2018-04-19 19:49:44 -07:00
Elliott Hughes 955b6a4792 Add android::base::ParseByteCount.
Bug: N/A
Test: ran tests
Change-Id: Ib2adcf0a5b9494fcf8259b29974303e8516a9ad9
2018-04-19 19:49:16 -07:00
Treehugger Robot c4ad6b6adc Merge changes I722d2c41,Ic216ddef
* changes:
  Improve test_adb.py
  adb: Distinguish betwen already-connected and connection failure
2018-04-20 02:13:09 +00:00
Treehugger Robot 224d965be0 Merge "adb: don't error in handle_host_request if we can't acquire a transport." 2018-04-19 21:40:57 +00:00
Josh Gao 48383c806a tombstoned: don't create tombstones for failed dumps.
Instead of creating tombstone FDs in place and passing them out to
crash_dump directly, create them as O_TMPFILEs and link them into place
when crash_dump reports success, to avoid creating empty tombstones
in cases like an aborting thread racing with another thread that
manages to cleanly exit_group before the dump finishes.

Bug: http://b/77729983
Test: debuggerd_test
Test: adb shell 'for x in `seq 0 50`; do crasher; done'
Change-Id: I31ce4fd4a524abf8bde57152450209483d9d0ba9
2018-04-19 14:33:18 -07:00
Josh Gao 78f133d7d4 adb: don't error in handle_host_request if we can't acquire a transport.
Host services are attempted after handle_host_request, which means that
failing to find a transport to give to handle_forward_request shouldn't
send an error over to the other end.

Bug: http://b/78294734
Test: `adb track-devices` with multiple devices connected
Change-Id: I46c89cc1894b51d48fea7d4e629b1d57f73e3fd6
2018-04-19 13:09:18 -07:00
Tom Cherry 3152c540b4 Merge "Add a dummy entry for the 70220 event tag" 2018-04-19 18:02:30 +00:00
Treehugger Robot 8e73e6f242 Merge "init: Use android::base::boot_clock instead of /proc/uptime" 2018-04-19 17:43:07 +00:00
Treehugger Robot def249956c Merge "Set property for metadata encryption on first boot" 2018-04-19 00:17:42 +00:00
Tom Cherry dcc4b2bb4a Add a dummy entry for the 70220 event tag
This will prevent services from reaching out to logd if this tag is
present in the event log.

Bug: 64734187
Test: tree-hugger
Change-Id: If117e1c0cfa678af4190913f0ca87f4e92c54373
2018-04-18 16:25:56 -07:00
Treehugger Robot c629131c26 Merge "init.rc: create /data/system/dropbox" 2018-04-18 21:48:39 +00:00
Paul Crowley e383334457 Set property for metadata encryption on first boot
Bug: 77335096
Test: device boots twice with and without metadata encryption
Change-Id: Iaed78288cb37865ba23833721b73b11414e7e862
2018-04-18 14:42:45 -07:00
Mark Salyzyn afd66f2fd3 llkd: bootstat: propagate detailed livelock canonical boot reason
Report kernel_panic,sysrq,livelock,<state> reboot reason via last
dmesg (pstore console).  Add ro.llk.killtest property, which will
allow reliable ABA platforms to drop kill test and go directly
to kernel panic.  This should also allow some manual unit testing
of the canonical boot reason report.

New canonical boot reasons from llkd are:
- kernel_panic,sysrq,livelock,alarm llkd itself locked up (Hail Mary)
- kernel_panic,sysrq,livelock,driver uninterrruptible D state
- kernel_panic,sysrq,livelock,zombie uninterrruptible Z state

Manual test assumptions:
- llkd is built by the platform and landed on system partition
- unit test is built and landed in /data/nativetest (could
  land in /data/nativetest64, adjust test correspondingly)
- llkd not enabled, ro.llk.enable and ro.llk.killtest
  are not set by platform allowing test to adjust all the
  configuration properties and start llkd.
- or, llkd is enabled, ro.llk.enable is true, and killtest is
  disabled, ro.llk.killtest is false, setup by the platform.
  This breaks the go/apct generic operations of the unit test
  for llk.zombie and llk.driver as kernel panic results
  requiring manual intervention otherwise.  If test moves to
  go/apct, then we will be forced to bypass these tests under
  this condition (but allow them to run if ro.llk.killtest
  is "off" so specific testing above/below can be run).

for i in driver zombie; do
        adb shell su root setprop ro.llk.killtest off
        adb shell /data/nativetest/llkd_unit_test/llkd_unit_test --gtest_filter=llkd.${i}
        adb wait-for-device
        adb shell su root setprop ro.llk.killtest off
        sleep 60
        adb shell getprop sys.boot.reason
        adb shell /data/nativetest/llkd_unit_test/llkd_unit_test --gtest_filter=llkd.${i}
done

Test: llkd_unit_test (see test assumptions)
Bug: 33808187
Bug: 72838192
Change-Id: I2b24875376ddfdbc282ba3da5c5b3567de85dbc0
2018-04-18 14:02:16 -07:00
Mark Salyzyn d035dbbecf llkd: default enabled for userdebug
If LLK_ENABLE_DEFAULT is false, then check "ro.llk.enable" for "eng",
also the default value if not set, and then check if userdebug build
to establish a default of true for enable.  Same for
ro.khungtask.enable.

Test: llkd_unit_test report eng status on "userdebug" or "user" builds
Bug: 33808187
Bug: 72838192
Change-Id: I2adb23c7629dccaa2856c50bccbf4e363703c82c
2018-04-18 14:02:05 -07:00
Mark Salyzyn f089e1403b llkd: add live-lock daemon
Introduce a standalone live-lock daemon (llkd), to catch kernel
or native user space deadlocks and take mitigating actions.  Will
also configure [khungtaskd] to fortify the actions.

If a thread is in D or Z state with no forward progress for longer
than ro.llk.timeout_ms, or ro.llk.[D|Z].timeout_ms, kill the process
or parent process respectively.  If another scan shows the same
process continues to exist, then have a confirmed live-lock condition
and need to panic.  Panic the kernel in a manner to provide the
greatest bugreporting details as to the condition.  Add a alarm self
watchdog should llkd ever get locked up that is double the expected
time to flow through the mainloop.  Sampling is every
ro.llk_sample_ms.

Default will not monitor init, or [kthreadd] and all that [kthreadd]
spawns.  This reduces the effectiveness of llkd by limiting its
coverage.  If in the future, if value in covering kthreadd spawned
threads, the requirement will be to code drivers so that they do not
remain in a persistent 'D' state, or that they have mechanisms to
recover the thread should it be killed externally.  Then the
blacklists can be adjusted accordingly if these conditions are met.

An accompanying gTest set have been added, and will setup a persistent
D or Z process, with and without forward progress, but not in a
live-lock state because that would require a buggy kernel, or a module
or kernel modification to stimulate.

Android Properties llkd respond to (*_ms parms are in milliseconds):
- ro.config.low_ram default false, if true do not sysrq t (dump
  all threads).
- ro.llk.enable default false, allow live-lock daemon to be enabled.
- ro.khungtask.enable default false, allow [khungtaskd] to be enabled.
- ro.llk.mlockall default true, allow mlock'd live-lock daemon.
- ro.khungtask.timeout default 12 minutes.
- ro.llk.timeout_ms default 10 minutes, D or Z maximum timelimit,
  double this value and it sets the alarm watchdog for llkd.
- ro.llk.D.timeout_ms default ro.llk.timeout_ms, D maximum timelimit.
- ro.llk.Z.timeout_ms default ro.llk.timeout_ms, Z maximum timelimit.
- ro.llk.check_ms default 2 minutes sampling interval
  (ro.llk.timeout_ms / 5) for threads in D or Z state.
- ro.llk.blacklist.process default 0,1,2 (kernel, init and
  [kthreadd]), and process names (/comm or /cmdline) init,[kthreadd],
  lmkd,lmkd.llkd,llkd,[khungtaskd],watchdogd,[watchdogd],
  [watchdogd/0] ...
- ro.llk.blacklist.parent default 0,2 (kernel and [kthreadd]) and
  "[kthreadd]".  A comma separated lists of process ids, /comm names
  or /cmdline names.
- ro.llk.blacklist.uid default <empty>, comma separated list of
  uid numbers or names from getpwuid/getpwnam.

Test: llkd_unit_test
Bug: 33808187
Bug: 72838192
Change-Id: I32e8aa78aef10834e093265d0f3ed5b4199807c6
2018-04-18 14:01:56 -07:00
Tri Vo 4640149fd5 Merge "Add /mnt/vendor rw mount point for vendor partitions." 2018-04-18 19:32:32 +00:00