Report kernel_panic,sysrq,livelock,<state> reboot reason via last
dmesg (pstore console). Add ro.llk.killtest property, which will
allow reliable ABA platforms to drop kill test and go directly
to kernel panic. This should also allow some manual unit testing
of the canonical boot reason report.
New canonical boot reasons from llkd are:
- kernel_panic,sysrq,livelock,alarm llkd itself locked up (Hail Mary)
- kernel_panic,sysrq,livelock,driver uninterrruptible D state
- kernel_panic,sysrq,livelock,zombie uninterrruptible Z state
Manual test assumptions:
- llkd is built by the platform and landed on system partition
- unit test is built and landed in /data/nativetest (could
land in /data/nativetest64, adjust test correspondingly)
- llkd not enabled, ro.llk.enable and ro.llk.killtest
are not set by platform allowing test to adjust all the
configuration properties and start llkd.
- or, llkd is enabled, ro.llk.enable is true, and killtest is
disabled, ro.llk.killtest is false, setup by the platform.
This breaks the go/apct generic operations of the unit test
for llk.zombie and llk.driver as kernel panic results
requiring manual intervention otherwise. If test moves to
go/apct, then we will be forced to bypass these tests under
this condition (but allow them to run if ro.llk.killtest
is "off" so specific testing above/below can be run).
for i in driver zombie; do
adb shell su root setprop ro.llk.killtest off
adb shell /data/nativetest/llkd_unit_test/llkd_unit_test --gtest_filter=llkd.${i}
adb wait-for-device
adb shell su root setprop ro.llk.killtest off
sleep 60
adb shell getprop sys.boot.reason
adb shell /data/nativetest/llkd_unit_test/llkd_unit_test --gtest_filter=llkd.${i}
done
Test: llkd_unit_test (see test assumptions)
Bug: 33808187
Bug: 72838192
Change-Id: I2b24875376ddfdbc282ba3da5c5b3567de85dbc0
This change allows bootstat to read the ro.boot.boottime_offset
property, which is set on devices where Android runs in a container.
This is because the CLOCK_BOOTTIME clock is not reset when (from
Android's perspective) the device restarts, since the host OS does not
restart itself.
Bug: 77273909
Test: CtsBootStatsTestCases
Change-Id: Ifb792864394be0b4686cc9d555c0afced856f4b4
Merged-In: Ifb792864394be0b4686cc9d555c0afced856f4b4
This change allows bootstat to read the ro.boot.boottime_offset
property, which is set on devices where Android runs in a container.
This is because the CLOCK_BOOTTIME clock is not reset when (from
Android's perspective) the device restarts, since the host OS does not
restart itself.
Bug: 77273909
Test: CtsBootStatsTestCases
Change-Id: Ifb792864394be0b4686cc9d555c0afced856f4b4
Provide some easy kernel panic subreasons mined from last kmesg,
generates a canonical boot reason (system boot reason) that
may aid triage.
Notably report kernel_panic,hung if [khungtaskd] triggers on
a livelock condition, forms a signals on the dashboards.
Helper function getSubreason modified to optionally enable checking
for a single quote resulting in a refactoring to ease maintenance of
the termination detection in the face of single bit errors heuristics.
Test: boot_reason_test.sh
Bug: 63736262
Bug: 33808187
Change-Id: I7fdd1e57e7a26095738175074306f0d2d59b1d69
Allow for a daemon to write to last kmsg to propagate a detailed
subreason to kernel_panic,sysrq actions. A minor refactor moves
common code into a helper function getSubreason for retrieval and
bit error correction operations.
A sysrq crash generally produces a kernel-provided message:
SysRq : Trigger a crash
which is used to generate a canonical boot reason kernel_panic,sysrq.
A user daemon could write to /dev/kmsg just prior to the sysrq with
SysRq : Trigger a crash : '<subreason>'
to change the canonical boot reason to kernel_panic,sysrq,<subreason>.
Administration added pending kBootReasonMap entries present in TRON.
Test: manual echo into /dev/kmsg and /proc/sysrq-trigger and check
Test: boot_reason_test.sh
Bug: 33808187
Bug: 63736262
Change-Id: Ibf5432737e5a3449ebe40a8c6cf2d3e912ed6bbc
For aliasReasons allow one to optionally suppress needle for output
member using a <bang> (!) character prefix.
Test: boot_reason_test.sh
Bug: 63736262
Bug: 74595769
Change-Id: If35518c08cf909c6c78a16275e9d8dfd0ff839a9
For all known cases, if usb is present in the bootloader reason,
then it is actually reporting a cold,charger canonical boot reason.
This signifies that the device was powered down, and was woken
up by the charger being connected.
For all known cases, if rtc is present in the bootloader reason,
then it is actually reporting a cold,rtc canonical boot reason.
This signifies that the device was powered down, and was woken
up by the rtc clock.
Test: boot_reason_test.sh
Bug: 74595769
Bug: 63736262
Change-Id: I04bede0b7ccaa1b859943f7def93521a8f7b25c6
Add support for regex in aliasReasons for the alias member. Use this
new feature to check powerkey|power_key|PowerKey for a single entry.
Test: boot_reason_test.sh
Bug: 63736262
Change-Id: Ia6add99b9e33f3197643dbaab88dde20aa726f90
When we are matching existing known boot reasons, we should try with
compliant underlines first, then again with underlines replaced with
spaces. Replace references to Ber with BitError for maintenance
clarity. Replace helper functions with C++11 lambdas.
Test: boot_reason_test.sh
Bug: 63736262
Change-Id: I91b865013513526a55a85523080c7127f198968c
Two entries can be reused. The third "unknown" entry is not really
a duplicate since the kUnknownBootReason is not checked. Duplicate
entries reused in the future, should have
analysis/uma/configs/clearcut/TRON/histograms.xml updated first.
Test: boot_reason_test.sh
Change-Id: If2071a18160dc2c93e851fecc6b8c11fc76c9845
Use an alternate means to determine that the sysrq crash was
requested. Also, to be CTS compliant, the kernel_panic subreason
must be in lower case.
Test: boot_reason_test.sh
Bug: 74595769
Bug: 63736262
Change-Id: Ica06960ce62d220a909006e365951376d672b7e6
Adding the boot sequence reported atom in ag/3518079 caused the duration
of bootstat to increase, as seen in b/72864061. I isolated the cause
down to calling BootReasonStrToReason. However, this function also gets
called in ReportBootReason, so I created another function that does the
parsing and sets the system boot reason property, and made
RecordBootReason and statsd logging get that property.
Bug: 72864061
Test: rebooted phone, verified boot events were received in adb shell
logcat -b stats and verified adb shell bootstat -p printed correct
values. Ran timing tests as well on walleye with 20 boots: before this
change, the average was ~150-160ms. After, it was ~80ms.
Change-Id: I92dbc9880328835647be7d9d50c7861b42f27bdb
Logs information about boot time and reason to statsd.
Specifically: bootloader boot reason, system boot reason, bootloader
boot time, total boot time, time that the boot finished, and time since
last boot.
Test: booted the phone and verified "adb logcat -b stats" received the
event
Change-Id: I769df9a09263ed3667f7085c81b3d072e868cbda
Integer overflow sanitized builds are throwing an error on the while
loop decrement in the rfind function. This refactors the loop to prevent
decrementing the value on the final iteration.
Test: Compiled and device boots without runtime error.
Bug: 30969751
Change-Id: Ice4532cce933062b3c14adf2d9749cfdea4ad84c
Merged-In: Ice4532cce933062b3c14adf2d9749cfdea4ad84c
We should have done this from the beginning. Thanks to Windows, we're not
going to be able to switch libbase over to std::string_view any time soon.
Bug: N/A
Test: ran tests
Change-Id: Iff2f56986e39de53f3ac484415378af17dacf26b
Create a private rfind that allows a fuzzy match based on a bit error
rate (BER) of 1 every 8 bits. last kmsg is affected by pstore ramoops
backing that suffers from data corruption. Add some additional
validation based on possible data corruption scenarios, as a noisy
match means higher chance of noisy data.
Noisy data notably can affect the battery level detection, but do not
typically result in false positives. Battery level, or failure, is
the responsibility of the BatteryStats service, providing a positive
signal and strong device-independent algorithm. The checking done in
bootstat is likely to be deprecated in favour of an API request to
BatteryStats once their algorithms deal with surprise outages due to
aging.
The kernel logging heuristic and BER fixup handily deals with a
prevalent issue where some bootloaders failure to properly notify us
of panics. This is where the gains are noticed with this improvement.
Test: system/core/bootstat/boot_reason_test.sh
Bug: 63736262
Change-Id: I93b4210f12fb47c5c036f4d6eb4cafeee4896d35
Replace simple strtoull with loop that ensures no leading zeros.
Restrict size of value buffer being checked as allocation was
going to end of retrieved buffer, which can cause unnecessary
memory pressure during boot.
Test: system/core/bootstat/boot_reason_test.sh
Bug: 63736262
Change-Id: Ifdc1d4fd3a73794c001577024ce7cbfde9c25028
If we sniff an unknown boot reason from last kmsg, make sure it
has a "reboot," prefix.
Test: system/core/bootstat/boot_reason_test.sh
Bug: 63736262
Change-Id: Ia1c401b8899d1f0c56bd4f5d8d2d19b7fc889a30
Add the test injection to known list, and deal with an error
propagation issue.
Test: system/core/bootstat/boot_reason_test.sh Its_Just_So_Hard_reboot
Bug: 63736262
Change-Id: I4799956978a8884c69c830fcedd7febd143651fd
Add cold,powerkey, warm,s3_wakeup and hard,hw_reset so that
sys.boot.reason values can also be enumerated. Also add
some reserved speculative entries associated with forced
suspend to RAM and DISK; shutdown,suspend and shutdown,hibernate
respectively.
Test: system/core/bootstat/boot_reason_test.sh
Bug: 67636061
Change-Id: Ic43523748e6006aaca882f8eec7c1f0c08431bd8
Empty boot reason is mostly unexpected but may take up the bulk of
unknown reported boot reason values.
Bug: none
Test: none
Change-Id: I9978658c2b052d5cf5d28299861b0d1125ba9fc0
To make parsing easier for last reboot reason. This also ensures that
last boot reason matches the content that is typically returned by the
bootloader or in turn landed in the canonical system boot reason.
Simplify parsing in bootstat. Adjust and fix boot_reason_test.sh for
new reality. Allow boot reason tests battery and kernel_panic to pass
if device does not support pstore (empty before and after the test).
If device somehow landed in fastboot mode while waiting for the
display, issue a fastboot reboot to move the test along. Some cleanup
and standardization changes to the test script.
Test: system/core/bootstat/boot_reason_test.sh
Bug: 63736262
Change-Id: I97d5467c0b4a6d65df3525f1a2d0051db813d5ad
Mark Salyzyn