For the gdb command lx-dmesg, the entire descriptor, info, and text
data regions are read into memory before printing any records. For
large kernel log buffers, this not only causes a huge delay before
seeing any records, but it may also lead to python errors of too
much memory allocation.
Rather than reading in all these regions in advance, read them as
needed and only read the regions for the particular record that is
being printed.
The gdb macro "dmesg" in Documentation/admin-guide/kdump/gdbmacros.txt
already prints out the kernel log buffer like this.
Signed-off-by: John Ogness <john.ogness@linutronix.de>
Signed-off-by: Petr Mladek <pmladek@suse.com>
Link: https://lore.kernel.org/r/874k79c3a9.fsf@jogness.linutronix.de
Dictionaries are only used for SUBSYSTEM and DEVICE properties. The
current implementation stores the property names each time they are
used. This requires more space than otherwise necessary. Also,
because the dictionary entries are currently considered optional,
it cannot be relied upon that they are always available, even if the
writer wanted to store them. These issues will increase should new
dictionary properties be introduced.
Rather than storing the subsystem and device properties in the
dict ring, introduce a struct dev_printk_info with separate fields
to store only the property values. Embed this struct within the
struct printk_info to provide guaranteed availability.
Signed-off-by: John Ogness <john.ogness@linutronix.de>
Reviewed-by: Petr Mladek <pmladek@suse.com>
Signed-off-by: Petr Mladek <pmladek@suse.com>
Link: https://lore.kernel.org/r/87mu1jl6ne.fsf@jogness.linutronix.de
Add support for extending the newest data block. For this, introduce
a new finalization state (desc_finalized) denoting a committed
descriptor that cannot be extended.
Until a record is finalized, a writer can reopen that record to
append new data. Reopening a record means transitioning from the
desc_committed state back to the desc_reserved state.
A writer can explicitly finalize a record if there is no intention
of extending it. Also, records are automatically finalized when a
new record is reserved. This relieves writers of needing to
explicitly finalize while also making such records available to
readers sooner. (Readers can only traverse finalized records.)
Four new memory barrier pairs are introduced. Two of them are
insignificant additions (data_realloc:A/desc_read:D and
data_realloc:A/data_push_tail:B) because they are alternate path
memory barriers that exactly match the purpose, pairing, and
context of the two existing memory barrier pairs they provide an
alternate path for. The other two new memory barrier pairs are
significant additions:
desc_reopen_last:A / _prb_commit:B - When reopening a descriptor,
ensure the state transitions back to desc_reserved before
fully trusting the descriptor data.
_prb_commit:B / desc_reserve:D - When committing a descriptor,
ensure the state transitions to desc_committed before checking
the head ID to see if the descriptor needs to be finalized.
Signed-off-by: John Ogness <john.ogness@linutronix.de>
Reviewed-by: Petr Mladek <pmladek@suse.com>
Signed-off-by: Petr Mladek <pmladek@suse.com>
Link: https://lore.kernel.org/r/20200914123354.832-6-john.ogness@linutronix.de
Rather than deriving the state by evaluating bits within the flags
area of the state variable, assign the states explicit values and
set those values in the flags area. Introduce macros to make it
simple to read and write state values for the state variable.
Although the functionality is preserved, the binary representation
for the states is changed.
Signed-off-by: John Ogness <john.ogness@linutronix.de>
Reviewed-by: Petr Mladek <pmladek@suse.com>
Signed-off-by: Petr Mladek <pmladek@suse.com>
Link: https://lore.kernel.org/r/20200914123354.832-5-john.ogness@linutronix.de
With the introduction of the lockless printk ringbuffer, the data
structure for the kernel log buffer was changed. Update the gdb
scripts to be able to parse/print the new log buffer structure.
Fixes: 896fbe20b4 ("printk: use the lockless ringbuffer")
Signed-off-by: John Ogness <john.ogness@linutronix.de>
Reported-by: Nick Desaulniers <ndesaulniers@google.com>
Tested-by: Nick Desaulniers <ndesaulniers@google.com>
Tested-by: Petr Mladek <pmladek@suse.com>
[akpm@linux-foundation.org: A typo fix.]
Signed-off-by: Petr Mladek <pmladek@suse.com>
Link: https://lore.kernel.org/r/20200814212525.6118-3-john.ogness@linutronix.de
When CONFIG_PRINTK_CALLER is set, struct printk_log contains an
additional member caller_id. This affects the offset of the log text.
Account for this by using the type information from gdb to determine all
the offsets instead of using hardcoded values.
This fixes following error:
(gdb) lx-dmesg
Python Exception <class 'ValueError'> embedded null character:
Error occurred in Python command: embedded null character
The read_u* utility functions now take an offset argument to make them
easier to use.
Link: http://lkml.kernel.org/r/20191011142500.2339-1-joel.colledge@linbit.com
Signed-off-by: Joel Colledge <joel.colledge@linbit.com>
Reviewed-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Kieran Bingham <kbingham@kernel.org>
Cc: Leonard Crestez <leonard.crestez@nxp.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Use errors=replace because it is never desirable for lx-dmesg to fail on
string decoding errors, not even if the log buffer is corrupt and we
show incorrect info.
The kernel will sometimes print utf8, for example the copyright symbol
from jffs2. In order to make this work specify 'utf8' everywhere
because python2 otherwise defaults to 'ascii'.
In theory the second errors='replace' is not be required because
everything that can be decoded as utf8 should also be encodable back to
utf8. But it's better to be extra safe here. It's worth noting that
this is definitely not true for encoding='ascii', unknown characters are
replaced with U+FFFD REPLACEMENT CHARACTER and they fail to encode back
to ascii.
Link: http://lkml.kernel.org/r/acee067f3345954ed41efb77b80eebdc038619c6.1498481469.git.leonard.crestez@nxp.com
Signed-off-by: Leonard Crestez <leonard.crestez@nxp.com>
Acked-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Jason Wessel <jason.wessel@windriver.com>
Cc: Kieran Bingham <kieran@ksquared.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
In some cases it is possible for the str() conversion here to throw
encoding errors because log_buf might not point to valid ascii. For
example:
(gdb) python print str(gdb.parse_and_eval("log_buf"))
Traceback (most recent call last):
File "<string>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0303' in
position 24: ordinal not in range(128)
Avoid this by explicitly casting to (void *) inside the gdb expression.
Link: http://lkml.kernel.org/r/ba6f85dbb02ca980ebd0e2399b0649423399b565.1498481469.git.leonard.crestez@nxp.com
Signed-off-by: Leonard Crestez <leonard.crestez@nxp.com>
Reviewed-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Jason Wessel <jason.wessel@windriver.com>
Cc: Kieran Bingham <kieran@ksquared.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
lx-dmesg needs access to the log_buf symbol from printk.c.
Unfortunately, the symbol log_buf also exists in BPF's verifier.c and
hence gdb can pick one or the other. If it happens to pick BPF's
log_buf, lx-dmesg doesn't work:
(gdb) lx-dmesg
Python Exception <class 'gdb.MemoryError'> Cannot access memory at address 0x0:
Error occurred in Python command: Cannot access memory at address 0x0
(gdb) p log_buf
$15 = 0x0
Luckily, GDB has a way to deal with this, see
https://sourceware.org/gdb/onlinedocs/gdb/Symbols.html
(gdb) info variables ^log_buf$
All variables matching regular expression "^log_buf$":
File <linux.git>/kernel/bpf/verifier.c:
static char *log_buf;
File <linux.git>/kernel/printk/printk.c:
static char *log_buf;
(gdb) p 'verifier.c'::log_buf
$1 = 0x0
(gdb) p 'printk.c'::log_buf
$2 = 0x811a6aa0 <__log_buf> ""
(gdb) p &log_buf
$3 = (char **) 0x8120fe40 <log_buf>
(gdb) p &'verifier.c'::log_buf
$4 = (char **) 0x8120fe40 <log_buf>
(gdb) p &'printk.c'::log_buf
$5 = (char **) 0x8048b7d0 <log_buf>
By being explicit about the location of the symbol, we can make lx-dmesg
work again. While at it, do the same for the other symbols we need from
printk.c
Link: http://lkml.kernel.org/r/20170526112222.3414-1-git@andred.net
Signed-off-by: André Draszik <git@andred.net>
Tested-by: Kieran Bingham <kieran@bingham.xyz>
Acked-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
The recent fixes to lx-dmesg, now allow the command to print
successfully on Python3, however the python interpreter wraps the bytes
for each line with a b'<text>' marker.
To remove this, we need to decode the line, where .decode() will default
to 'UTF-8'
Link: http://lkml.kernel.org/r/d67ccf93f2479c94cb3399262b9b796e0dbefcf2.1462865983.git.jan.kiszka@siemens.com
Signed-off-by: Kieran Bingham <kieran@bingham.xyz>
Acked-by: Dom Cote <buzdelabuz2@gmail.com>
Tested-by: Dom Cote <buzdelabuz2@gmail.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
When built against Python 3, GDB differs in the return type for its
read_memory function, causing the lx-dmesg command to fail.
Now that we have an improved read_16() we can use the new
read_memoryview() abstraction to make lx-dmesg return valid data on both
current Python APIs
Tested with python 3.4 and 2.7
Tested with gdb 7.7
Link: http://lkml.kernel.org/r/28477b727ff7fe3101fd4e426060e8a68317a639.1462865983.git.jan.kiszka@siemens.com
Signed-off-by: Dom Cote <buzdelabuz2+git@gmail.com>
[kieran@bingham.xyz: Adjusted commit log to better reflect code changes]
Tested-by: Kieran Bingham <kieran@bingham.xyz> (Py2.7,Py3.4,GDB10)
Signed-off-by: Kieran Bingham <kieran@bingham.xyz>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
I tried to use these scripts in an ubuntu 14.04 host (gdb 7.7 compiled
against python 3.3) but there were several errors.
I believe this patch fixes these issues so that the commands now work (I
tested lx-symbols, lx-dmesg, lx-lsmod).
Main issues that needed to be resolved:
* In python 2 iterators have a "next()" method. In python 3 it is
__next__() instead (so let's just add both).
* In older python versions there was an implicit conversion
in object.__format__() (used when an object is in string.format())
where it was converting the object to str first and then
calling str's __format__(). This has now been removed so
we must explicitly convert to str the objects for which
we need to keep this behavior.
* In dmesg.py: in python 3 log_buf is now a "memoryview" object
which needs to be converted to a string in order to use string
methods like "splitlines()". Luckily memoryview exists in
python 2.7.6 as well, so we can convert log_buf to memoryview
and use the same code in both python 2 and python 3.
This version of the patch has now been tested with gdb 7.7 and both python
3.4 and python 2.7.6 (I think asking for at least python 2.7.6 is a
reasonable requirement instead of complicating the code with version
checks etc).
Signed-off-by: Pantelis Koukousoulas <pktoss@gmail.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Wessel <jason.wessel@windriver.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Ben Widawsky <ben@bwidawsk.net>
Cc: Borislav Petkov <bp@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This pokes into the log buffer of the debugged kernel, dumping it to the
gdb console. Helping in case the target should or can no longer execute
dmesg itself.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Kay Sievers <kay@vrfy.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Wessel <jason.wessel@windriver.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Ben Widawsky <ben@bwidawsk.net>
Cc: Borislav Petkov <bp@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
John Ogness