Commit Graph

22 Commits

Author SHA1 Message Date
Alexey Budankov 3b1c5d9659 tools build: Implement libzstd feature check, LIBZSTD_DIR and NO_LIBZSTD defines
Implement libzstd feature check, NO_LIBZSTD and LIBZSTD_DIR defines to
override Zstd library sources or disable the feature from the command
line:

  $ make -C tools/perf LIBZSTD_DIR=/path/to/zstd/sources/ clean all
  $ make -C tools/perf NO_LIBZSTD=1 clean all

Auto detection feature status is reported just before compilation
starts.  If your system has some version of the zstd library
preinstalled then the build system finds and uses it during the build.

If you still prefer to compile with some other version of zstd library
you have capability to refer the compilation to that version using
LIBZSTD_DIR define.

Signed-off-by: Alexey Budankov <alexey.budankov@linux.intel.com>
Reviewed-by: Jiri Olsa <jolsa@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/9b4cd8b0-10a3-1f1e-8d6b-5922a7ca216b@linux.intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-01 15:18:10 -03:00
Song Liu 8a1b171821 perf build: Check what binutils's 'disassembler()' signature to use
Commit 003ca0fd2286 ("Refactor disassembler selection") in the binutils
repo, which changed the disassembler() function signature, so we must
use the feature test introduced in fb982666e3 ("tools/bpftool: fix
bpftool build with bintutils >= 2.9") to deal with that.

Committer testing:

After adding the missing function call to test-all.c, and:

  FEATURE_CHECK_LDFLAGS-disassembler-four-args = -bfd -lopcodes

And the fallbacks for cases where we need -liberty and sometimes -lz to
tools/perf/Makefile.config, we get:

  $ make -C tools/perf O=/tmp/build/perf install-bin
  make: Entering directory '/home/acme/git/perf/tools/perf'
    BUILD:   Doing 'make -j8' parallel build

  Auto-detecting system features:
  ...                         dwarf: [ on  ]
  ...            dwarf_getlocations: [ on  ]
  ...                         glibc: [ on  ]
  ...                          gtk2: [ on  ]
  ...                      libaudit: [ on  ]
  ...                        libbfd: [ on  ]
  ...                        libelf: [ on  ]
  ...                       libnuma: [ on  ]
  ...        numa_num_possible_cpus: [ on  ]
  ...                       libperl: [ on  ]
  ...                     libpython: [ on  ]
  ...                      libslang: [ on  ]
  ...                     libcrypto: [ on  ]
  ...                     libunwind: [ on  ]
  ...            libdw-dwarf-unwind: [ on  ]
  ...                          zlib: [ on  ]
  ...                          lzma: [ on  ]
  ...                     get_cpuid: [ on  ]
  ...                           bpf: [ on  ]
  ...                        libaio: [ on  ]
  ...        disassembler-four-args: [ on  ]
    CC       /tmp/build/perf/jvmti/libjvmti.o
    CC       /tmp/build/perf/builtin-bench.o
  <SNIP>
  $
  $

The feature detection test-all.bin gets successfully built and linked:

  $ ls -la /tmp/build/perf/feature/test-all.bin
  -rwxrwxr-x. 1 acme acme 2680352 Mar 19 11:07 /tmp/build/perf/feature/test-all.bin
  $ nm /tmp/build/perf/feature/test-all.bin  | grep -w disassembler
  0000000000061f90 T disassembler
  $

Time to move on to the patches that make use of this disassembler()
routine in binutils's libopcodes.

Signed-off-by: Song Liu <songliubraving@fb.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Jakub Kicinski <jakub.kicinski@netronome.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Roman Gushchin <guro@fb.com>
Cc: Stanislav Fomichev <sdf@google.com>
Link: http://lkml.kernel.org/r/20190312053051.2690567-13-songliubraving@fb.com
[ split from a larger patch, added missing FEATURE_CHECK_LDFLAGS-disassembler-four-args ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-20 16:42:10 -03:00
Arnaldo Carvalho de Melo a96c03e8cd tools build: Add test-reallocarray.c to test-all.c to fix the build
When a test is in the FEATURE_TESTS_BASIC list in tools/build/Makefile.feature
must be added to tools/build/feature/test-all.c, because the successfull
compilation and linking of that test-all.bin file means that all the
features listed in FEATURE_TESTS_BASIC are present in the system, so we
don't have to go on feature by feature test building them.

Since reallocarray() is expected to be present in modern systems, it has
a place in FEATURE_TESTS_BASIC, so that we speed up the build process
building just that file.

For older systems, such as ubuntu:16.04 (build failure reported by Jin
Yao) debian:8, and for the current flagship RHEL distro, RHEL7, the
build will fail as test-all.bin (without test-reallocarray.c included)
passes but reallocarray() isn't present, making the build fail with:

    CC       /tmp/build/perf/libbpf.o
    MKDIR    /tmp/build/perf/fs/
    CC       /tmp/build/perf/fs/tracing_path.o
    LD       /tmp/build/perf/fd/libapi-in.o
    CC       /tmp/build/perf/bpf.o
  libbpf.c: In function 'bpf_object__add_program':
  libbpf.c:367:10: error: implicit declaration of function 'reallocarray' [-Werror=implicit-function-declaration]
    progs = reallocarray(progs, nr_progs + 1, sizeof(progs[0]));
            ^
  libbpf.c:367:2: error: nested extern declaration of 'reallocarray' [-Werror=nested-externs]
    progs = reallocarray(progs, nr_progs + 1, sizeof(progs[0]));
    ^
  libbpf.c:367:8: error: assignment makes pointer from integer without a cast [-Werror=int-conversion]
    progs = reallocarray(progs, nr_progs + 1, sizeof(progs[0]));
          ^
  libbpf.c: In function 'bpf_object__elf_collect':
  libbpf.c:887:10: error: assignment makes pointer from integer without a cast [-Werror=int-conversion]
      reloc = reallocarray(reloc, nr_reloc,
            ^
  libbpf.c: In function 'bpf_program__reloc_text':
  libbpf.c:1394:12: error: assignment makes pointer from integer without a cast [-Werror=int-conversion]
     new_insn = reallocarray(prog->insns, new_cnt, sizeof(*insn));
              ^
    CC       /tmp/build/perf/nlattr.o

Even with:

  $ grep reallocarray /tmp/build/perf/FEATURE-DUMP
  feature-reallocarray=1
  $

Which ubuntu:16.04.5 LTS doesn't have:

  perfbuilder@38a153a1bba8:/$ head -2 /etc/os-release
  NAME="Ubuntu"
  VERSION="16.04.5 LTS (Xenial Xerus)"
  perfbuilder@38a153a1bba8:/$ find /usr/include/ -name "*.h" | xargs grep -w reallocarray
  perfbuilder@38a153a1bba8:/$

Fix it by including it to test-all.c, which ends up forcing the
individual tests to be triggered and for the build process to notice
that indeed reallocarray() is not there:

  perfbuilder@38a153a1bba8:/$ cat /tmp/build/perf/feature/test-all.make.output
  In file included from test-all.c:178:0:
  test-reallocarray.c: In function 'main_test_reallocarray':
  test-reallocarray.c:7:11: error: implicit declaration of function 'reallocarray' [-Werror=implicit-function-declaration]
    return !!reallocarray(NULL, 1, 1);
             ^
  cc1: all warnings being treated as errors
  perfbuilder@38a153a1bba8:/$

That is the only test that is failing on Ubuntu 16.03.5 LTS, so all
tests are forced:

  perfbuilder@38a153a1bba8:/tmp/build/perf/feature$ ls -lSr *.make.output
  <SNIP successful tests>
  -rw-r--r--. 1 perfbuilder perfbuilder   0 Feb 14 15:00 test-dwarf.make.output
  -rw-r--r--. 1 perfbuilder perfbuilder   0 Feb 14 14:16 test-cplus-demangle.make.output
  -rw-r--r--. 1 perfbuilder perfbuilder   0 Feb 14 15:00 test-bpf.make.output
  -rw-r--r--. 1 perfbuilder perfbuilder   0 Feb 14 15:00 test-backtrace.make.output
  -rw-r--r--. 1 perfbuilder perfbuilder 104 Feb 14 15:00 test-bionic.make.output
  -rw-r--r--. 1 perfbuilder perfbuilder 107 Feb 14 15:00 test-libunwind-x86.make.output
  -rw-r--r--. 1 perfbuilder perfbuilder 115 Feb 14 15:00 test-libunwind-aarch64.make.output
  -rw-r--r--. 1 perfbuilder perfbuilder 122 Feb 14 15:00 test-libbabeltrace.make.output
  -rw-r--r--. 1 perfbuilder perfbuilder 254 Feb 14 15:00 test-reallocarray.make.output
  -rw-r--r--. 1 perfbuilder perfbuilder 312 Feb 14 15:00 test-all.make.output
  perfbuilder@38a153a1bba8:/tmp/build/perf/feature$

And that reallocarray() one shows:

  perfbuilder@38a153a1bba8:/tmp/build/perf/feature$ cat test-reallocarray.make.output
  test-reallocarray.c: In function 'main':
  test-reallocarray.c:7:11: error: implicit declaration of function 'reallocarray' [-Werror=implicit-function-declaration]
    return !!reallocarray(NULL, 1, 1);
             ^
  cc1: all warnings being treated as errors
  perfbuilder@38a153a1bba8:/tmp/build/perf/feature$

Which now generates the expected result:

  perfbuilder@38a153a1bba8:~$ grep reallocarray /tmp/build/perf/FEATURE-DUMP
  feature-reallocarray=0
  perfbuilder@38a153a1bba8:~$

The fallback mechanism kicks in and libbpf and perf are again buildable
in systems without reallocarray():

  $ cat tools/include/tools/libc_compat.h
  // SPDX-License-Identifier: (LGPL-2.0+ OR BSD-2-Clause)
  /* Copyright (C) 2018 Netronome Systems, Inc. */

  #ifndef __TOOLS_LIBC_COMPAT_H
  #define __TOOLS_LIBC_COMPAT_H

  #include <stdlib.h>
  #include <linux/overflow.h>

  #ifdef COMPAT_NEED_REALLOCARRAY
  static inline void *reallocarray(void *ptr, size_t nmemb, size_t size)
  {
	  size_t bytes;

	  if (unlikely(check_mul_overflow(nmemb, size, &bytes)))
		  return NULL;
	  return realloc(ptr, bytes);
  }
  #endif
  #endif
  $

Reported-by: Jin Yao <yao.jin@linux.intel.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Jakub Kicinski <jakub.kicinski@netronome.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Song Liu <songliubraving@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Fixes: 531b014e7a ("tools: bpf: make use of reallocarray")
Link: https://lkml.kernel.org/n/tip-aonqku8axii8rxki5g11w40b@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-14 15:18:05 -03:00
Arnaldo Carvalho de Melo 1c3b28fd7a perf coresight: Do not test for libopencsd by default
Since it is not yet that generally available, avoid testing for the
presence of libcoresight in the fast path test-all.bin feature test.

  # dnf search opencsd
  No matches found.
  # dnf search OpenCSD
  No matches found.
  # cat /etc/fedora-release
  Fedora release 29 (Twenty Nine)
  #

I.e. right now, in my system test-all.bin is failing all the time since
Fedora29 doesn't have libopencsd available:

  $ cat /tmp/build/perf/feature/test-all.make.output
  In file included from test-all.c:174:
  test-libopencsd.c:2:10: fatal error: opencsd/c_api/opencsd_c_api.h: No such file or directory
   #include <opencsd/c_api/opencsd_c_api.h>
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  compilation terminated.

See:

  6ab2b762be ("perf build: Disable libbabeltrace check by default")

For the rationale, as soon as libopencsd becomes more generally packaged
and available, we do the same thing we did with babeltrace, enabling it
by default, as done in:

  24787afbcd ("perf tools: Enable LIBBABELTRACE by default")

For now, to explicitely ask for opencsd, make sure you have it installed
and use:

   make -C tools/perf CORESIGHT=1

The feature test output will be there as an empty file:

  $ ls -la /tmp/build/perf/feature/test-libopencsd.make.output

Because the binary used for the feature check was successfully built:

  $ ls -la /tmp/build/perf/feature/test-libopencsd.bin
  -rwxrwxr-x. 1 acme acme 18336 Feb 12 14:49 /tmp/build/perf/feature/test-libopencsd.bin
  $ ldd /tmp/build/perf/feature/test-libopencsd.bin
	linux-vdso.so.1 (0x00007fffe18cc000)
	libopencsd_c_api.so.0 => /lib64/libopencsd_c_api.so.0 (0x00007fb8e67f6000)
	libopencsd.so.0 => /lib64/libopencsd.so.0 (0x00007fb8e676f000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fb8e65a9000)
	libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fb8e6411000)
	libm.so.6 => /lib64/libm.so.6 (0x00007fb8e628d000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fb8e6272000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fb8e6828000)
  $

And the resulting perf binary will be linked with it:

  -rw-rw-r--. 1 acme acme 0 Feb 12 14:49 /tmp/build/perf/feature/test-libopencsd.make.output
  $ ldd ~/bin/perf | grep opencsd
	libopencsd_c_api.so.0 => /lib64/libopencsd_c_api.so.0 (0x00007fd43097f000)
	libopencsd.so.0 => /lib64/libopencsd.so.0 (0x00007fd4308f8000)
  $

To make sure this gets built before pushing things upstream I have a
ubuntu:19.04-x-arm64 container that has:

  [root@quaco x-arm64]# grep CORESIGHT Dockerfile
  ENV EXTRA_MAKE_ARGS=CORESIGHT=1
  [root@quaco x-arm64]#

So that I always build with libopencsd before pushing things upstream.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kim Phillips <kim.phillips@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
Cc: Mike Leach <mike.leach@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Suzuki Poulouse <suzuki.poulose@arm.com>
Link: https://lkml.kernel.org/n/tip-20vyy39jw9jgrijesi30fgox@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-14 15:18:05 -03:00
Alexey Budankov 2a07d81474 tools build feature: Check if libaio is available
This will be used by 'perf record' to speed up reading the perf ring
buffer.

Committer testing:

  $ make -C tools/perf O=/tmp/build/perf
  make: Entering directory '/home/acme/git/perf/tools/perf'
    BUILD:   Doing 'make -j8' parallel build

  Auto-detecting system features:
  ...                         dwarf: [ on  ]
  ...            dwarf_getlocations: [ on  ]
  ...                         glibc: [ on  ]
  ...                          gtk2: [ OFF ]
  ...                      libaudit: [ OFF ]
  ...                        libbfd: [ OFF ]
  ...                        libelf: [ on  ]
  ...                       libnuma: [ OFF ]
  ...        numa_num_possible_cpus: [ OFF ]
  ...                       libperl: [ OFF ]
  ...                     libpython: [ OFF ]
  ...                      libslang: [ on  ]
  ...                     libcrypto: [ on  ]
  ...                     libunwind: [ on  ]
  ...            libdw-dwarf-unwind: [ on  ]
  ...                          zlib: [ on  ]
  ...                          lzma: [ on  ]
  ...                     get_cpuid: [ on  ]
  ...                           bpf: [ on  ]
  ...                        libaio: [ on  ]

  $ ls -la /tmp/build/perf/feature/test-libaio.*
  -rwxrwxr-x. 1 acme acme 18296 Nov 26 08:49 /tmp/build/perf/feature/test-libaio.bin
  -rw-rw-r--. 1 acme acme  1165 Nov 26 08:49 /tmp/build/perf/feature/test-libaio.d
  -rw-rw-r--. 1 acme acme     0 Nov 26 08:49 /tmp/build/perf/feature/test-libaio.make.output
  $
  $ grep -i aio /tmp/build/perf/FEATURE-DUMP
  feature-libaio=1
  $

Signed-off-by: Alexey Budankov <alexey.budankov@linux.intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Reviewed-by: Jiri Olsa <jolsa@redhat.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/5fcda10c-6c63-68df-383a-c6d9e5d1f918@linux.intel.com
[ split from a larger patch ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-12-17 14:54:54 -03:00
Arnaldo Carvalho de Melo 11c6cbe706 tools build feature: Check if eventfd() is available
A new 'perf bench epoll' will use this, and to disable it for older
systems, add a feature test for this API.

This is just a simple program that if successfully compiled, means that
the feature is present, at least at the library level, in a build that
sets the output directory to /tmp/build/perf (using O=/tmp/build/perf),
we end up with:

  $ ls -la /tmp/build/perf/feature/test-eventfd*
  -rwxrwxr-x. 1 acme acme 8176 Nov 21 15:58 /tmp/build/perf/feature/test-eventfd.bin
  -rw-rw-r--. 1 acme acme  588 Nov 21 15:58 /tmp/build/perf/feature/test-eventfd.d
  -rw-rw-r--. 1 acme acme    0 Nov 21 15:58 /tmp/build/perf/feature/test-eventfd.make.output
  $ ldd /tmp/build/perf/feature/test-eventfd.bin
	  linux-vdso.so.1 (0x00007fff3bf3f000)
	  libc.so.6 => /lib64/libc.so.6 (0x00007fa984061000)
	  /lib64/ld-linux-x86-64.so.2 (0x00007fa984417000)
  $ grep eventfd -A 2 -B 2 /tmp/build/perf/FEATURE-DUMP
  feature-dwarf=1
  feature-dwarf_getlocations=1
  feature-eventfd=1
  feature-fortify-source=1
  feature-sync-compare-and-swap=1
  $

The main thing here is that in the end we'll have -DHAVE_EVENTFD in
CFLAGS, and then the 'perf bench' entry needing that API can be
selectively pruned.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Davidlohr Bueso <dbueso@suse.de>
Cc: Jason Baron <jbaron@akamai.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: https://lkml.kernel.org/n/tip-wkeldwob7dpx6jvtuzl8164k@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-11-21 22:25:44 -03:00
Arnaldo Carvalho de Melo 8feb8efef9 tools build feature: Check if get_current_dir_name() is available
As the namespace support code will use this, which is not available in
some non _GNU_SOURCE libraries such as Android's bionic used in my
container build tests (r12b and r15c at the moment).

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: https://lkml.kernel.org/n/tip-x56ypm940pwclwu45d7jfj47@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-11-19 12:12:17 -08:00
Mathieu Poirier aa6292f484 perf tools: Integrating the CoreSight decoding library
The Open CoreSight Decoding Library (openCSD) is a free and open library
to decode traces collected by the CoreSight hardware infrastructure.

This patch adds the required mechanic to recognise the presence of the
openCSD library on a system and set up miscellaneous flags to be used in
the compilation of the trace decoding feature.

Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Kim Phillips <kim.phillips@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
Cc: Mike Leach <mike.leach@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Suzuki Poulouse <suzuki.poulose@arm.com>
Link: http://lkml.kernel.org/r/1516211539-5166-2-git-send-email-mathieu.poirier@linaro.org
Link: http://lkml.kernel.org/r/1516635644-24819-1-git-send-email-mathieu.poirier@linaro.org
[ Merged missing test-libopencsd.c file, provided later by Mathieu ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-01-25 06:37:23 -03:00
Arnaldo Carvalho de Melo 25ab5abf5b tools build feature: Check if pthread_barrier_t is available
As 'perf bench futex wake-parallel" will use this, which is not
available in older systems such as versions of the android NDK used in
my container build tests (r12b and r15c at the moment).

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: James Yang <james.yang@arm.com
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kim Phillips <kim.phillips@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: https://lkml.kernel.org/n/tip-1i7iv54in4wj08lwo55b0pzv@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-12-05 10:21:59 -03:00
Greg Kroah-Hartman b24413180f License cleanup: add SPDX GPL-2.0 license identifier to files with no license
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.

By default all files without license information are under the default
license of the kernel, which is GPL version 2.

Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier.  The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.

This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.

How this work was done:

Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
 - file had no licensing information it it.
 - file was a */uapi/* one with no licensing information in it,
 - file was a */uapi/* one with existing licensing information,

Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.

The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne.  Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.

The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed.  Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.

Criteria used to select files for SPDX license identifier tagging was:
 - Files considered eligible had to be source code files.
 - Make and config files were included as candidates if they contained >5
   lines of source
 - File already had some variant of a license header in it (even if <5
   lines).

All documentation files were explicitly excluded.

The following heuristics were used to determine which SPDX license
identifiers to apply.

 - when both scanners couldn't find any license traces, file was
   considered to have no license information in it, and the top level
   COPYING file license applied.

   For non */uapi/* files that summary was:

   SPDX license identifier                            # files
   ---------------------------------------------------|-------
   GPL-2.0                                              11139

   and resulted in the first patch in this series.

   If that file was a */uapi/* path one, it was "GPL-2.0 WITH
   Linux-syscall-note" otherwise it was "GPL-2.0".  Results of that was:

   SPDX license identifier                            # files
   ---------------------------------------------------|-------
   GPL-2.0 WITH Linux-syscall-note                        930

   and resulted in the second patch in this series.

 - if a file had some form of licensing information in it, and was one
   of the */uapi/* ones, it was denoted with the Linux-syscall-note if
   any GPL family license was found in the file or had no licensing in
   it (per prior point).  Results summary:

   SPDX license identifier                            # files
   ---------------------------------------------------|------
   GPL-2.0 WITH Linux-syscall-note                       270
   GPL-2.0+ WITH Linux-syscall-note                      169
   ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause)    21
   ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)    17
   LGPL-2.1+ WITH Linux-syscall-note                      15
   GPL-1.0+ WITH Linux-syscall-note                       14
   ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause)    5
   LGPL-2.0+ WITH Linux-syscall-note                       4
   LGPL-2.1 WITH Linux-syscall-note                        3
   ((GPL-2.0 WITH Linux-syscall-note) OR MIT)              3
   ((GPL-2.0 WITH Linux-syscall-note) AND MIT)             1

   and that resulted in the third patch in this series.

 - when the two scanners agreed on the detected license(s), that became
   the concluded license(s).

 - when there was disagreement between the two scanners (one detected a
   license but the other didn't, or they both detected different
   licenses) a manual inspection of the file occurred.

 - In most cases a manual inspection of the information in the file
   resulted in a clear resolution of the license that should apply (and
   which scanner probably needed to revisit its heuristics).

 - When it was not immediately clear, the license identifier was
   confirmed with lawyers working with the Linux Foundation.

 - If there was any question as to the appropriate license identifier,
   the file was flagged for further research and to be revisited later
   in time.

In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.

Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights.  The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.

Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.

In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.

Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
 - a full scancode scan run, collecting the matched texts, detected
   license ids and scores
 - reviewing anything where there was a license detected (about 500+
   files) to ensure that the applied SPDX license was correct
 - reviewing anything where there was no detection but the patch license
   was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
   SPDX license was correct

This produced a worksheet with 20 files needing minor correction.  This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.

These .csv files were then reviewed by Greg.  Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected.  This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.)  Finally Greg ran the script using the .csv files to
generate the patches.

Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-02 11:10:55 +01:00
Arnaldo Carvalho de Melo 86bcdb5a43 tools build: Add test for setns()
And provide an alternative implementation to keep perf building on older
distros as we're about to add initial support for namespaces.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Krister Johansen <kjlx@templeofstupid.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-bqdwijunhjlvps1ardykhw1i@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-07-18 23:14:08 -03:00
Arnaldo Carvalho de Melo 120010cb1e tools build: Add test for sched_getcpu()
Instead of trying to go on adding more ifdef conditions, do a feature
test and define HAVE_SCHED_GETCPU_SUPPORT instead, then use it to
provide the prototype. No need to change the stub, as it is already a
__weak symbol.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-yge89er9g90sc0v6k0a0r5tr@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-03-03 19:07:19 -03:00
Masami Hiramatsu e26e63be64 perf build: Add sdt feature detection
This checks whether sys/sdt.h is available or not, which is required for
DTRACE_PROBE().

We can disable this feature by passing NO_SDT=1 when building.

This flag will be used for SDT test case and further SDT events in
perftools.

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/146831795615.17065.17513820540591053933.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-07-13 23:09:09 -03:00
Arnaldo Carvalho de Melo 1c1a3a4729 perf tools: Add feature detection for gelf_getnote()
That is not present on some libelf implementations, such as the one used
in Alpine Linux: libelf-0.8.13.

This ends up disabling the SDT code, that relies on this function.

One alternative would be to provide an weak fallback implementation or
the open coded variant used by the buildid sysfs notes reading code.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-82lh22ybedy9b9lych8xj12g@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-07-12 15:20:41 -03:00
Arnaldo Carvalho de Melo 2492c465ad perf build: Add feature detection for libelf's elf_getshdrstrndx()
That appeared after 0.140, and will be used in the SDT code, so, to
avoid bisection break on older systems, add a feature detection and
provide a stub with a pr_debug() to keep it building.

Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/n/tip-80y0eldgweorqnwha9rvfxjr@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-07-04 19:38:59 -03:00
Arnaldo Carvalho de Melo 4924734570 perf probe: Check if dwarf_getlocations() is available
If not, tell the user that:

  config/Makefile:273: Old libdw.h, finding variables at given 'perf probe' point will not work, install elfutils-devel/libdw-dev >= 0.157

And return -ENOTSUPP in die_get_var_range(), failing features that
need it, like the one pointed out above.

This fixes the build on older systems, such as Ubuntu 12.04.5.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Vinson Lee <vlee@freedesktop.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-9l7luqkq4gfnx7vrklkq4obs@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-05-12 11:26:59 -03:00
Stephane Eranian 8ee4646038 perf build: Add libcrypto feature detection
Will be used to generate build-ids in the jitdump code.

Signed-off-by: Stephane Eranian <eranian@google.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Carl Love <cel@us.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: John McCutchan <johnmccutchan@google.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Pawel Moll <pawel.moll@arm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sonny Rao <sonnyrao@chromium.org>
Cc: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
Link: http://lkml.kernel.org/r/1448874143-7269-3-git-send-email-eranian@google.com
[ tools/perf/Makefile.perf comment about NO_LIBCRYPTO and added it to tests/make ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-02-05 09:46:45 -03:00
Wang Nan 0c4d40d580 tools build: Add BPF feature check to test-all
The test-all.c file doesn't check BPF related features. For an
environment with all other features enabled, BPF would be considered
enabled without doing real feature check.

This patch adds test-bpf.c into test-all.c.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1452520124-2073-11-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-01-11 19:22:21 -03:00
Arnaldo Carvalho de Melo b0063dbfb0 tools build: Add test for presence of __get_cpuid() gcc builtin
The auxtrace code needed by Intel PT uses the __get_cpuid() gcc builtin,
that is not present in old systems, breaking the build.

Add a test to check for that builtin and disable AUXTRACE in those
systems.

  [acme@rhel5 linux]$  make NO_LIBPERL=1 -C tools/perf O=/tmp/build/perf install-bin
  make: Entering directory `/home/acme/git/linux/tools/perf'
    BUILD:   Doing 'make -j2' parallel build

  Auto-detecting system features:
  <SNIP>
  ...                          lzma: [ on  ]
  ...                     get_cpuid: [ OFF ]
  <SNIP>
  config/Makefile:630: Your gcc lacks the __get_cpuid() builtin, disables support for auxtrace/Intel PT, please install a newer gcc
    MKDIR    /tmp/build/perf/util/
  <SNIP>

This fixes the build on old systems such as RHEL/CentOS 5.11.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Borislav Petkov <bp@suse.de>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: "Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Stephane Eranian <eranian@google.com>
Cc: Victor Kamensky <victor.kamensky@linaro.org>
Cc: Vinson Lee <vlee@twopensource.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-d4puslul0jltoodzpx9r4sje@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-17 13:14:35 -03:00
Arnaldo Carvalho de Melo f8ac8606fd tools build: Add test for presence of numa_num_possible_cpus() in libnuma
The existing numa test checks only if numa.h and numa_available() are
present, but that can be satisfied with an old libnuma that is not
enough for the 'perf bench numa' entry, so add a test to check for that:

  [acme@rhel5 linux]$  make NO_AUXTRACE=1 NO_LIBPERL=1 -C tools/perf O=/tmp/build/perf install-bin
  make: Entering directory `/home/acme/git/linux/tools/perf'
    BUILD:   Doing 'make -j2' parallel build

  Auto-detecting system features:
  ...                        libelf: [ on  ]
  ...                       libnuma: [ on  ]
  ...        numa_num_possible_cpus: [ OFF ]
  ...                       libperl: [ on  ]

  <SNIP>
  config/Makefile:577: Old numa library found, disables 'perf bench numa mem' benchmark, please install numactl-devel/libnuma-devel/libnuma-dev >= 2.0.8
    INSTALL  binaries
  <SNIP>

This fixes the build on old systems such as RHEL/CentOS 5.11.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Borislav Petkov <bp@suse.de>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: "Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Stephane Eranian <eranian@google.com>
Cc: Victor Kamensky <victor.kamensky@linaro.org>
Cc: Vinson Lee <vlee@twopensource.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-zqriqkezppi2de2iyjin1tnc@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-17 13:14:35 -03:00
Jiri Olsa 6c6f0f6164 tools build: Add feature check for lzma library
Will be used to decompress 'xz' objects. The check detects
the liblzma.so devel library normally delivered by xz package.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
2015-03-21 14:53:39 -03:00
Jiri Olsa e6c76d6203 perf build: Move feature checks code under tools/build
Moving feature checks code under tools/build directory.

Changing also $feature_dir to point to new feature directory location
and perf Makefiles to include Makefile.feature from new location.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <david.ahern@oracle.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/n/tip-3lamtb30dhf4wo99y1n8kxg0@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-03-21 14:53:35 -03:00