There could be different types of memory in the system. E.g normal
System Memory, Persistent Memory. To understand how the workload maps to
those memories, it's important to know the I/O statistics of them. Perf
can collect physical addresses, but those are raw data. It still needs
extra work to resolve the physical addresses. Provide a script to
facilitate the physical addresses resolving and I/O statistics.
Profile with MEM_INST_RETIRED.ALL_LOADS or MEM_UOPS_RETIRED.ALL_LOADS
event if any of them is available.
Look up the /proc/iomem and resolve the physical address. Provide
memory type summary.
Here is an example output:
# perf script report mem-phys-addr
Event: mem_inst_retired.all_loads:P
Memory type count percentage
---------------------------------------- ----------- -----------
System RAM 74 53.2%
Persistent Memory 55 39.6%
N/A
---
Changes since V2:
- Apply the new license rules.
- Add comments for globals
Changes since V1:
- Do not mix DLA and Load Latency. Do not compare the loads and stores.
Only profile the loads.
- Use event name to replace the RAW event
Signed-off-by: Kan Liang <Kan.liang@intel.com>
Reviewed-by: Andi Kleen <ak@linux.intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Philippe Ombredanne <pombredanne@nexb.com>
Cc: Stephane Eranian <eranian@google.com>
Link: https://lkml.kernel.org/r/1515099595-34770-1-git-send-email-kan.liang@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add support for exporting to SQLite 3 the same data as the PostgreSQL
export.
Committer note:
Tested on RHEL 7.4 using the 1.2.2-4el python-pyside packages from EPEL.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/r/1501749090-20357-4-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add script intel-pt-events.py that provides an example of how to unpack the
raw data for power events and PTWRITE.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Link: http://lkml.kernel.org/r/1495786658-18063-35-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add stackcollapse.py script as an example of parsing call chains, and
also of using optparse to access command line options.
The flame graph tools include a set of scripts that parse output from
various tools (including "perf script"), remove the offsets in the
function and collapse each stack to a single line. The website also
says "perf report could have a report style [...] that output folded
stacks directly, obviating the need for stackcollapse-perf.pl", so here
it is.
This script is a Python rewrite of stackcollapse-perf.pl, using the perf
scripting interface to access the perf data directly from Python.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: Brendan Gregg <bgregg@netflix.com>
Link: http://lkml.kernel.org/r/1460467573-22989-1-git-send-email-pbonzini@redhat.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This patch creates a new script (compaction-times) to report time
spent in mm compaction. It is possible to report times in nanoseconds
(default) or microseconds (-u).
The option -p will break down results by process id, -pv will further
decompose by each compaction entry/exit.
For each compaction entry/exit what is reported is controlled by the
options:
-t report only timing
-m report migration stats
-ms report migration scanner stats
-fs report free scanner stats
The default is to report all.
Entries may be further filtered by pid, pid-range or comm (regex).
The script is useful when analysing workloads that compact memory. The
most common example will be THP allocations on systems with a lot of
uptime that has fragmented memory.
This is an example of using the script to analyse a thpscale from
mmtests which deliberately fragments memory and allocates THP in 4
separate threads
# Recording step, one of the following;
$ perf record -e 'compaction:mm_compaction_*' ./workload
# or:
$ perf script record compaction-times
# Reporting: basic
total: 2444505743ns migration: moved=357738 failed=39275
free_scanner: scanned=2705578 isolated=387875
migration_scanner: scanned=414426 isolated=397013
# Reporting: Per task stall times
$ perf script report compaction-times -- -t -p
total: 2444505743ns
6384[thpscale]: 740800017ns
6385[thpscale]: 274119512ns
6386[thpscale]: 832961337ns
6383[thpscale]: 596624877ns
# Reporting: Per-compaction attempts for task 6385
$ perf script report compaction-times -- -m -pv 6385
total: 274119512ns migration: moved=14893 failed=24285
6385[thpscale]: 274119512ns migration: moved=14893 failed=24285
6385[thpscale].1: 3033277ns migration: moved=511 failed=1
6385[thpscale].2: 9592094ns migration: moved=1524 failed=12
6385[thpscale].3: 2495587ns migration: moved=512 failed=0
6385[thpscale].4: 2561766ns migration: moved=512 failed=0
6385[thpscale].5: 2523521ns migration: moved=512 failed=0
..... output continues ...
Changes since v1:
- report stats for isolate_migratepages and isolate_freepages
(Vlastimil Babka)
- refactor code to achieve above
- add help text
- output to stdout/stderr explicitly
Signed-off-by: Tony Jones <tonyj@suse.com>
Cc: Mel Gorman <mgorman@suse.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Link: http://lkml.kernel.org/r/1439840932-8933-1-git-send-email-tonyj@suse.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add the ability to export detailed information about paired calls and
returns to Python db export and the export-to-postgresql.py script.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@gmail.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add a Python script to export to a postgresql database.
The script requires the Python pyside module and the Qt PostgreSQL
driver. The packages needed are probably named "python-pyside" and
"libqt4-sql-psql"
The caller of the script must be able to create postgresql databases.
The script takes the database name as a parameter. The database and
database tables are created. Data is written to flat files which are
then imported using SQL COPY FROM.
Example:
$ perf record ls
...
$ perf script report export-to-postgresql lsdb
2014-02-14 10:55:38.631431 Creating database...
2014-02-14 10:55:39.291958 Writing to intermediate files...
2014-02-14 10:55:39.350280 Copying to database...
2014-02-14 10:55:39.358536 Removing intermediate files...
2014-02-14 10:55:39.358665 Adding primary keys
2014-02-14 10:55:39.658697 Adding foreign keys
2014-02-14 10:55:39.667412 Done
$ psql lsdb
lsdb-# \d
List of relations
Schema | Name | Type | Owner
--------+-----------------+-------+-------
public | comm_threads | table | acme
public | comms | table | acme
public | dsos | table | acme
public | machines | table | acme
public | samples | table | acme
public | samples_view | view | acme
public | selected_events | table | acme
public | symbols | table | acme
public | threads | table | acme
(9 rows)
lsdb-# \d samples
Table "public.samples"
Column | Type | Modifiers
---------------+---------+-----------
id | bigint | not null
evsel_id | bigint |
machine_id | bigint |
thread_id | bigint |
comm_id | bigint |
dso_id | bigint |
symbol_id | bigint |
sym_offset | bigint |
ip | bigint |
time | bigint |
cpu | integer |
to_dso_id | bigint |
to_symbol_id | bigint |
to_sym_offset | bigint |
to_ip | bigint |
period | bigint |
weight | bigint |
transaction | bigint |
data_src | bigint |
Indexes:
"samples_pkey" PRIMARY KEY, btree (id)
Foreign-key constraints:
"commfk" FOREIGN KEY (comm_id) REFERENCES comms(id)
"dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id)
"evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id)
"machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id)
"symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id)
"threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id)
"todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id)
"tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id)
lsdb-# \d samples_view
View "public.samples_view"
Column | Type | Modifiers
-------------------+-------------------------+-----------
id | bigint |
time | bigint |
cpu | integer |
pid | integer |
tid | integer |
command | character varying(16) |
event | character varying(80) |
ip_hex | text |
symbol | character varying(2048) |
sym_offset | bigint |
dso_short_name | character varying(256) |
to_ip_hex | text |
to_symbol | character varying(2048) |
to_sym_offset | bigint |
to_dso_short_name | character varying(256) |
lsdb=# select * from samples_view;
id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name
--+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+----------
1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown
2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown
3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown
My notes (which may be out-of-date) on setting up postgresql so you can
create databases:
fedora:
$ sudo yum install postgresql postgresql-server python-pyside qt-postgresql
$ sudo su - postgres -c initdb
$ sudo service postgresql start
$ sudo su - postgres
$ createuser -s <your username>
I used the the unix user name in createuser.
If it fails, try createuser without -s and answer the following question
to allow your user to create tables:
Shall the new role be a superuser? (y/n) y
ubuntu:
$ sudo apt-get install postgresql
$ sudo su - postgres
$ createuser <your username>
Shall the new role be a superuser? (y/n) y
You may want to disable automatic startup. One way is to edit
/etc/postgresql/9.3/main/start.conf. Another is to disable the init
script e.g. sudo update-rc.d postgresql disable
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@gmail.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Older kernels (e.g., RHEL6) do system call tracing via the
syscalls:sys_{enter,exit} tracepoints rather than using raw_syscalls:*.
Update perf python and perl scripts to fallback to syscalls:* when
raw_syscalls:* isn't available.
Signed-off-by: Daniel Bristot de Oliveira <bristot@redhat.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Luis Claudio R. Goncalves <lgoncalv@redhat.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/5a6c64081a3375bc3bc66351b14559678ef4d71e.1402507908.git.bristot@redhat.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
So that event_analyzing_sample.py can be shown by "perf script -l"
Signed-off-by: Feng Tang <feng.tang@intel.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1347007349-3102-4-git-send-email-feng.tang@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
A while back I created the dropmonitor protocol, which allowed users to get
reports of dropped frames communicated to them via a netlink socket.
While useful, several people have now asked that I integrate the ability
to do drop monitoring with perf, so they don't have to run additional
tools.
This patch adds a drop monitor script to the perf suite, and provides
the same output that the netlink socket does.
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1309801217-22450-1-git-send-email-nhorman@tuxdriver.com
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The scripts have calls to 'perf trace' that need to be converted to 'perf script', do it.
This problem was introduced in 133dc4c.
Reported-by: Torok Edwin <edwintorok@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Tom Zanussi <tzanussi@gmail.com>
Cc: Torok Edwin <edwintorok@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Including -a unconditionally when recording doesn't allow for the
option of running scripts without it. Future patches will add add it
back if needed at run-time.
Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Acked-by: Arnaldo Carvalho de Melo <acme@ghostprotocols.net>
Set $PERF_EXEC_PATH before starting the record and report scripts, and
make them use it where necessary.
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
LKML-Reference: <1286723403.2955.205.camel@localhost>
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add a perf script which shows packets processing and processed
time. It helps us to investigate networking or network devices.
If you want to use it, install perf and record perf.data like
following.
If you set script, perf gathers records until it ends.
If not, you must Ctrl-C to stop recording.
And if you want a report from record,
If you use some options, you can limit the output.
Option is below.
tx: show only tx packets processing
rx: show only rx packets processing
dev=: show processing on this device
debug: work with debug mode. It shows buffer status.
For example, if you want to show received packets processing
associated with eth4,
106133.171439sec cpu=0
irq_entry(+0.000msec irq=24:eth4)
|
softirq_entry(+0.006msec)
|
|---netif_receive_skb(+0.010msec skb=f2d15900 len=100)
| |
| skb_copy_datagram_iovec(+0.039msec 10291::10291)
|
napi_poll_exit(+0.022msec eth4)
This perf script helps us to analyze the processing time of a
transmit/receive sequence.
Signed-off-by: Koki Sanagi <sanagi.koki@jp.fujitsu.com>
Acked-by: David S. Miller <davem@davemloft.net>
Cc: Neil Horman <nhorman@tuxdriver.com>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Kaneshige Kenji <kaneshige.kenji@jp.fujitsu.com>
Cc: Izumo Taku <izumi.taku@jp.fujitsu.com>
Cc: Kosaki Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Lai Jiangshan <laijs@cn.fujitsu.com>
Cc: Scott Mcmillan <scott.a.mcmillan@intel.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Tom Zanussi <tzanussi@gmail.com>
LKML-Reference: <4C72439D.3040001@jp.fujitsu.com>
Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
This brings a GUI tool that displays an overview of the load
of tasks proportion in each CPUs.
The CPUs forward progress is cut in timeslices. A new timeslice
is created for every runqueue event: a task gets pushed out or
pulled in the runqueue.
For each timeslice, every CPUs rectangle is colored with a red
power that describes the local load against the total load.
This more red is the rectangle, the higher is the given CPU load.
This load is the number of tasks running on the CPU, without
any distinction against the scheduler policy of the tasks, for
now.
Also for each timeslice, the event origin is depicted on the
CPUs that triggered it using a thin colored line on top of the
rectangle timeslice.
These events are:
* sleep: a task went to sleep and has then been pulled out the
runqueue. The origin color in the thin line is dark blue.
* wake up: a task woke up and has then been pushed in the
runqueue. The origin color is yellow.
* wake up new: a new task woke up and has then been pushed in the
runqueue. The origin color is green.
* migrate in: a task migrated in the runqueue due to a load
balancing operation. The origin color is violet.
* migrate out: reverse of the previous one. Migrate in events
usually have paired migrate out events in another runqueue.
The origin color is light blue.
Clicking on a timeslice provides the runqueue event details
and the runqueue state.
The CPU rectangles can be navigated using the usual arrow
controls. Horizontal zooming in/out is possible with the
"+" and "-" buttons.
Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Li Zefan <lizf@cn.fujitsu.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Tom Zanussi <tzanussi@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Venkatesh Pallipadi <venki@google.com>
Cc: Pierre Tardy <tardyp@gmail.com>
Cc: Nikhil Rao <ncrao@google.com>
Cc: Li Zefan <lizf@cn.fujitsu.com>
A small fix for the syscall counts script:
- silence the match output in the shell script
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
LKML-Reference: <1273466820-9330-10-git-send-email-tzanussi@gmail.com>
Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
A small fix for the syscall counts by pid script:
- silence the match output in the shell script
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
LKML-Reference: <1273466820-9330-9-git-send-email-tzanussi@gmail.com>
Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
A small fixe for the failed syscalls by pid script:
- silence the match output in the shell script
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
LKML-Reference: <1273466820-9330-8-git-send-email-tzanussi@gmail.com>
Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
-f, -c 1, -R are now useless for trace events recording, moreover
-M is useless and event hurts.
Remove them from the documentation examples and from record scripts.
Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Tom Zanussi <tzanussi@gmail.com>
It should be possible to run any perf trace script in 'live
mode'. This requires being able to pass in e.g. '-i -' or other
args, which the current shell scripts aren't equipped to handle.
In a few cases, there are required or optional args that also
need special handling. This patch makes changes the current set
of shell scripts as necessary.
Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: fweisbec@gmail.com
Cc: rostedt@goodmis.org
Cc: k-keiichi@bx.jp.nec.com
Cc: acme@ghostprotocols.net
LKML-Reference: <1270184365-8281-11-git-send-email-tzanussi@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
A couple of scripts, one in Python and the other in Perl, that
demonstrate 'live mode' tracing. For each, the output of the
perf event stream is fed continuously to the script, which
continuously aggregates the data and reports the current results
every 3 seconds, or at the optionally specified interval. After
the current results are displayed, the aggregations are cleared
and the cycle begins anew.
To run the scripts, simply pipe the output of the 'perf trace
record' step as input to the corresponding 'perf trace report'
step, using '-' as the filename to -o and -i:
$ perf trace record sctop -o - | perf trace report sctop -i -
Also adds clear_term() utility functions to the Util.pm and
Util.py utility modules, for use by any script to clear the
screen.
Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: fweisbec@gmail.com
Cc: rostedt@goodmis.org
Cc: k-keiichi@bx.jp.nec.com
Cc: acme@ghostprotocols.net
LKML-Reference: <1270184365-8281-10-git-send-email-tzanussi@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Adds a set of scripts that aggregate system call totals and system
call errors. Most are Python scripts that also test basic
functionality of the new Python engine, but there's also one Perl
script added for comparison and for reference in some new
Documentation contained in a later patch.
Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Keiichi KII <k-keiichi@bx.jp.nec.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
LKML-Reference: <1264580883-15324-8-git-send-email-tzanussi@gmail.com>
Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
Ravi Bangoria