linux/scripts
Linus Torvalds b791d1bdf9 The Kernel Concurrency Sanitizer (KCSAN)
KCSAN is a dynamic race detector, which relies on compile-time
 instrumentation, and uses a watchpoint-based sampling approach to detect
 races.
 
 The feature was under development for quite some time and has already found
 legitimate bugs.
 
 Unfortunately it comes with a limitation, which was only understood late in
 the development cycle:
 
   It requires an up to date CLANG-11 compiler
 
 CLANG-11 is not yet released (scheduled for June), but it's the only
 compiler today which handles the kernel requirements and especially the
 annotations of functions to exclude them from KCSAN instrumentation
 correctly.
 
 These annotations really need to work so that low level entry code and
 especially int3 text poke handling can be completely isolated.
 
 A detailed discussion of the requirements and compiler issues can be found
 here:
 
   https://lore.kernel.org/lkml/CANpmjNMTsY_8241bS7=XAfqvZHFLrVEkv_uM4aDUWE_kh3Rvbw@mail.gmail.com/
 
 We came to the conclusion that trying to work around compiler limitations
 and bugs again would end up in a major trainwreck, so requiring a working
 compiler seemed to be the best choice.
 
 For Continous Integration purposes the compiler restriction is manageable
 and that's where most xxSAN reports come from.
 
 For a change this limitation might make GCC people actually look at their
 bugs. Some issues with CSAN in GCC are 7 years old and one has been 'fixed'
 3 years ago with a half baken solution which 'solved' the reported issue
 but not the underlying problem.
 
 The KCSAN developers also ponder to use a GCC plugin to become independent,
 but that's not something which will show up in a few days.
 
 Blocking KCSAN until wide spread compiler support is available is not a
 really good alternative because the continuous growth of lockless
 optimizations in the kernel demands proper tooling support.
 -----BEGIN PGP SIGNATURE-----
 
 iQJHBAABCgAxFiEEQp8+kY+LLUocC4bMphj1TA10mKEFAl7im98THHRnbHhAbGlu
 dXRyb25peC5kZQAKCRCmGPVMDXSYoQ3xD/9+q87OmwnyoRTs6O3GDDbWZYoJGolh
 rctDOAYW8RSS73Fiw23z8hKlLl9tJCya6/X8Q9qoonB1YeIEPPRVj5HJWAMUNEIs
 YgjlZJFmh+mnbP/KQFctm3AWpoX8kqt3ncqj6zG72oQ9qKui691BY/2NmGVSLxUV
 DqtUYSKmi51XEQtZuXRuHEf3zBxoyeD43DaSCdJAXd6f5O2X7tmrWDuazHVeKzHV
 lhijvkyBvGMWvPg0IBrXkkLmeOvS0++MTGm3o+L72XF6nWpzTkcV7N0E9GEDFg45
 zwcidRVKD5d/1DoU5Tos96rCJpBEGh/wimlu0z14mcZpNiJgRQH5rzVEO9Y14UcP
 KL9FgRrb5dFw7yfX2zRQ070OFJ4AEDBMK0o5Lbu/QO5KLkvFkqnuWlQfmmtZJWCW
 DTRw/FgUgU7lvyPjRrao6HBvwy+yTb0u9K5seCOTRkuepR9nPJs0710pFiBsNCfV
 RY3cyggNBipAzgBOgLxixnq9+rHt70ton6S8Gijxpvt0dGGfO8k0wuEhFtA4zKrQ
 6HGK+pidxnoVdEgyQZhS+qzMMkyiUL0FXdaGJ2IX+/DC+Ij1UrUPjZBn7v25M0hQ
 ESkvxWKCn7snH4/NJsNxqCV1zyEc3zAW/WvLJUc9I7H8zPwtVvKWPrKEMzrJJ5bA
 aneySilbRxBFUg==
 =iplm
 -----END PGP SIGNATURE-----

Merge tag 'locking-kcsan-2020-06-11' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull the Kernel Concurrency Sanitizer from Thomas Gleixner:
 "The Kernel Concurrency Sanitizer (KCSAN) is a dynamic race detector,
  which relies on compile-time instrumentation, and uses a
  watchpoint-based sampling approach to detect races.

  The feature was under development for quite some time and has already
  found legitimate bugs.

  Unfortunately it comes with a limitation, which was only understood
  late in the development cycle:

     It requires an up to date CLANG-11 compiler

  CLANG-11 is not yet released (scheduled for June), but it's the only
  compiler today which handles the kernel requirements and especially
  the annotations of functions to exclude them from KCSAN
  instrumentation correctly.

  These annotations really need to work so that low level entry code and
  especially int3 text poke handling can be completely isolated.

  A detailed discussion of the requirements and compiler issues can be
  found here:

    https://lore.kernel.org/lkml/CANpmjNMTsY_8241bS7=XAfqvZHFLrVEkv_uM4aDUWE_kh3Rvbw@mail.gmail.com/

  We came to the conclusion that trying to work around compiler
  limitations and bugs again would end up in a major trainwreck, so
  requiring a working compiler seemed to be the best choice.

  For Continous Integration purposes the compiler restriction is
  manageable and that's where most xxSAN reports come from.

  For a change this limitation might make GCC people actually look at
  their bugs. Some issues with CSAN in GCC are 7 years old and one has
  been 'fixed' 3 years ago with a half baken solution which 'solved' the
  reported issue but not the underlying problem.

  The KCSAN developers also ponder to use a GCC plugin to become
  independent, but that's not something which will show up in a few
  days.

  Blocking KCSAN until wide spread compiler support is available is not
  a really good alternative because the continuous growth of lockless
  optimizations in the kernel demands proper tooling support"

* tag 'locking-kcsan-2020-06-11' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (76 commits)
  compiler_types.h, kasan: Use __SANITIZE_ADDRESS__ instead of CONFIG_KASAN to decide inlining
  compiler.h: Move function attributes to compiler_types.h
  compiler.h: Avoid nested statement expression in data_race()
  compiler.h: Remove data_race() and unnecessary checks from {READ,WRITE}_ONCE()
  kcsan: Update Documentation to change supported compilers
  kcsan: Remove 'noinline' from __no_kcsan_or_inline
  kcsan: Pass option tsan-instrument-read-before-write to Clang
  kcsan: Support distinguishing volatile accesses
  kcsan: Restrict supported compilers
  kcsan: Avoid inserting __tsan_func_entry/exit if possible
  ubsan, kcsan: Don't combine sanitizer with kcov on clang
  objtool, kcsan: Add kcsan_disable_current() and kcsan_enable_current_nowarn()
  kcsan: Add __kcsan_{enable,disable}_current() variants
  checkpatch: Warn about data_race() without comment
  kcsan: Use GFP_ATOMIC under spin lock
  Improve KCSAN documentation a bit
  kcsan: Make reporting aware of KCSAN tests
  kcsan: Fix function matching in report
  kcsan: Change data_race() to no longer require marking racing accesses
  kcsan: Move kcsan_{disable,enable}_current() to kcsan-checks.h
  ...
2020-06-11 18:55:43 -07:00
..
atomic Rebase locking/kcsan to locking/urgent 2020-06-11 20:02:46 +02:00
basic modpost,fixdep: Replace zero-length array with flexible-array 2020-05-26 00:03:16 +09:00
coccinelle net: remove newlines in NL_SET_ERR_MSG_MOD 2020-05-07 17:56:14 -07:00
dtc kbuild: check libyaml installation for 'make dt_binding_check' 2020-04-17 10:45:23 -05:00
dummy-tools kbuild: add dummy toolchains to enable all cc-option etc. in Kconfig 2020-04-09 00:13:45 +09:00
gcc-plugins Merge branch 'rwonce/rework' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux 2020-06-10 14:46:54 -07:00
gdb scripts/gdb: repair rb_first() and rb_last() 2020-05-07 19:27:20 -07:00
genksyms .gitignore: add SPDX License Identifier 2020-03-25 11:50:48 +01:00
kconfig kconfig: announce removal of 'kvmconfig' and 'xenconfig' shorthands 2020-05-26 00:02:32 +09:00
ksymoops
mod Kbuild updates for v5.8 2020-06-06 12:00:25 -07:00
package kbuild: add variables for compression tools 2020-06-06 23:42:01 +09:00
selinux SPDX patches for 5.7-rc1. 2020-04-03 13:12:26 -07:00
tracing treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 166 2019-05-30 11:26:39 -07:00
.gitignore .gitignore: add SPDX License Identifier 2020-03-25 11:50:48 +01:00
Kbuild.include kbuild: use -MMD instead of -MD to exclude system headers from dependency 2020-05-12 13:28:33 +09:00
Kconfig.include gcc-plugins: drop support for GCC <= 4.7 2020-04-09 00:13:45 +09:00
Lindent License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
Makefile kbuild: remove -I$(srctree)/tools/include from scripts/Makefile 2020-04-09 00:13:45 +09:00
Makefile.asm-generic kbuild: force all architectures except um to include mandatory-y 2019-03-17 12:56:32 +09:00
Makefile.build kbuild: update modules.order only when contained modules are updated 2020-06-03 13:22:17 +09:00
Makefile.clean kbuild: add infrastructure to build userspace programs 2020-05-17 18:52:01 +09:00
Makefile.dtbinst kbuild: refactor Makefile.dtbinst more 2020-03-25 10:19:43 +09:00
Makefile.extrawarn kbuild: Enable -Wtautological-compare 2020-04-09 00:13:45 +09:00
Makefile.gcc-plugins gcc-plugins: structleak: Generalize to all variable types 2019-03-04 09:29:41 -08:00
Makefile.headersinst kbuild: move headers_check rule to usr/include/Makefile 2019-11-15 00:23:10 +09:00
Makefile.host kbuild: use -MMD instead of -MD to exclude system headers from dependency 2020-05-12 13:28:33 +09:00
Makefile.kasan kasan/arm64: fix CONFIG_KASAN_SW_TAGS && KASAN_INLINE 2019-08-15 13:24:04 +01:00
Makefile.kcov treewide: Add SPDX license identifier - Makefile/Kconfig 2019-05-21 10:50:46 +02:00
Makefile.kcsan kcsan: Pass option tsan-instrument-read-before-write to Clang 2020-06-11 20:04:01 +02:00
Makefile.lib Rebase locking/kcsan to locking/urgent 2020-06-11 20:02:46 +02:00
Makefile.modfinal kbuild: move modkern_{c,a}flags to Makefile.lib from Makefile.build 2019-08-22 01:14:11 +09:00
Makefile.modinst kbuild: modinst: read modules.order instead of $(MODVERDIR)/*.mod 2019-07-17 22:39:27 +09:00
Makefile.modpost modpost: move -d option in scripts/Makefile.modpost 2020-06-06 23:38:13 +09:00
Makefile.modsign kbuild: modsign: read modules.order instead of $(MODVERDIR)/*.mod 2019-07-17 22:39:27 +09:00
Makefile.package kbuild: add variables for compression tools 2020-06-06 23:42:01 +09:00
Makefile.ubsan ubsan: split "bounds" checker from other options 2020-04-07 10:43:44 -07:00
Makefile.userprogs kbuild: add infrastructure to build userspace programs 2020-05-17 18:52:01 +09:00
adjust_autoksyms.sh kbuild: split adjust_autoksyms.sh in two parts 2020-03-03 20:49:21 +09:00
asn1_compiler.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 36 2019-05-24 17:27:11 +02:00
bin2c.c kbuild: move bin2c back to scripts/ from scripts/basic/ 2018-07-18 01:18:05 +09:00
bloat-o-meter bloat-o-meter: ignore __addressable_ symbols 2018-12-28 12:11:44 -08:00
bootgraph.pl treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 391 2019-06-05 17:37:11 +02:00
bpf_helpers_doc.py bpf: Minor fixes to BPF helpers documentation 2020-05-11 21:20:53 +02:00
cc-can-link.sh bpfilter: check compiler capability in Kconfig 2018-06-28 13:36:39 +09:00
check-sysctl-docs docs: add a script to check sysctl docs 2020-02-25 03:35:16 -07:00
check_extable.sh License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
checkincludes.pl License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
checkkconfigsymbols.py treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 166 2019-05-30 11:26:39 -07:00
checkpatch.pl The Kernel Concurrency Sanitizer (KCSAN) 2020-06-11 18:55:43 -07:00
checkstack.pl scripts/checkstack.pl: fix arm sp regex 2020-05-26 00:03:16 +09:00
checksyscalls.sh checksyscalls: fix up mq_timedreceive and stat exceptions 2019-02-19 21:27:53 +01:00
checkversion.pl License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
clang-version.sh kbuild: update comment block of scripts/clang-version.sh 2019-03-04 22:34:54 +09:00
cleanfile License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
cleanpatch License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
coccicheck coccicheck: return proper error code on fail 2018-08-14 08:58:56 +09:00
config scripts/config: allow colons in option strings for sed 2020-04-23 01:10:16 +09:00
const_structs.checkpatch
decode_stacktrace.sh scripts/decode_stacktrace: Accept dash/underscore in modules 2019-07-12 11:05:41 -07:00
decodecode scripts/decodecode: fix trapping instruction formatting 2020-05-07 19:27:20 -07:00
depmod.sh kbuild: modules_install: warn when missing System.map file 2018-09-09 09:14:07 +09:00
diffconfig License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
documentation-file-ref-check scripts: documentation-file-ref-check: Add line break before exit 2020-04-15 15:13:13 -06:00
export_report.pl modpost: move the namespace field in Module.symvers last 2020-03-17 08:59:03 +09:00
extract-cert.c
extract-ikconfig
extract-module-sig.pl License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
extract-sys-certs.pl License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
extract-vmlinux treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 378 2019-06-05 17:37:10 +02:00
extract_xc3028.pl treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 339 2019-06-05 17:37:07 +02:00
faddr2line scripts/faddr2line: fix location of start_kernel in comment 2018-11-18 10:15:09 -08:00
file-size.sh kbuild: Use ls(1) instead of stat(1) to obtain file size 2018-03-26 02:01:24 +09:00
find-unused-docs.sh scripts/find-unused-docs: Fix massive false positives 2020-01-27 14:25:06 -07:00
gcc-goto.sh jump_label: move 'asm goto' support test to Kconfig 2019-01-06 09:46:51 +09:00
gcc-ld License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
gcc-plugin.sh gcc-plugins: drop support for GCC <= 4.7 2020-04-09 00:13:45 +09:00
gcc-version.sh kbuild: clean up scripts/gcc-version.sh 2019-03-04 22:35:04 +09:00
gcc-x86_32-has-stack-protector.sh stack-protector: test compiler capability in Kconfig and drop AUTO mode 2018-06-08 18:56:00 +09:00
gcc-x86_64-has-stack-protector.sh stack-protector: Fix test with 32-bit userland and CONFIG_64BIT=y 2018-06-25 23:21:13 +09:00
gen_autoksyms.sh kbuild: generate autoksyms.h early 2020-03-03 20:49:21 +09:00
gen_compile_commands.py gen_compile_commands: lower the entry count threshold 2019-07-27 12:18:19 +09:00
gen_ksymdeps.sh kbuild: simplify dependency generation for CONFIG_TRIM_UNUSED_KSYMS 2018-12-01 23:13:14 +09:00
get_abi.pl doc: ABI scripts: add a SPDX header file 2019-06-21 16:58:37 +02:00
get_dvb_firmware treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 61 2019-05-24 17:36:45 +02:00
get_maintainer.pl get_maintainer: fix unexpected behavior for path/to//file (double slashes) 2020-06-04 19:06:24 -07:00
gfp-translate treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 166 2019-05-30 11:26:39 -07:00
headerdep.pl License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
headers_check.pl License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
headers_install.sh Fix for arch/sh build regression with newer binutils, removal of SH5, 2020-06-06 15:22:01 -07:00
insert-sys-cert.c
jobserver-exec docs, parallelism: Rearrange how jobserver reservations are made 2019-11-22 10:35:18 -07:00
kallsyms.c gcc-10 warnings: fix low-hanging fruit 2020-05-04 09:16:37 -07:00
kernel-doc Replace HTTP links with HTTPS ones: documentation 2020-06-08 09:30:19 -06:00
ld-version.sh License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
leaking_addresses.pl treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 166 2019-05-30 11:26:39 -07:00
link-vmlinux.sh Kbuild updates for v5.8 2020-06-06 12:00:25 -07:00
makelst License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
markup_oops.pl treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 373 2019-06-05 17:37:10 +02:00
mkcompile_h kbuild: use CONFIG_CC_VERSION_TEXT to construct LINUX_COMPILER macro 2020-05-12 13:28:33 +09:00
mkmakefile kbuild: get rid of $(realpath ...) from scripts/mkmakefile 2019-08-29 23:54:29 +09:00
mksysmap mksysmap: Fix the mismatch of '.L' symbols in System.map 2020-06-06 23:39:20 +09:00
mkuboot.sh License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
module-common.lds
modules-check.sh kbuild: make module name conflict fatal error 2020-05-26 00:03:16 +09:00
namespace.pl namespace: fix namespace.pl script to support relative paths 2019-10-05 15:29:49 +09:00
nsdeps scripts/nsdeps: support nsdeps for external module builds 2019-11-11 20:10:01 +09:00
objdiff treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 199 2019-05-30 11:29:23 -07:00
parse-maintainers.pl parse-maintainers: Do not sort section content by default 2020-03-26 15:08:27 -07:00
patch-kernel License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
profile2linkerlist.pl License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
prune-kernel License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
recordmcount.c ARM: 8950/1: ftrace/recordmcount: filter relocation types 2020-01-19 16:08:25 +00:00
recordmcount.h recordmcount: Fix nop_mcount() function 2019-10-12 20:49:33 -04:00
recordmcount.pl treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 166 2019-05-30 11:26:39 -07:00
setlocalversion scripts: setlocalversion: replace backquote to dollar parenthesis 2019-11-11 20:10:01 +09:00
show_delta treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 438 2019-06-05 17:37:17 +02:00
sign-file.c
sorttable.c scripts/sorttable: Implement build-time ORC unwind table sorting 2019-12-13 10:47:58 +01:00
sorttable.h scripts/sorttable: Implement build-time ORC unwind table sorting 2019-12-13 10:47:58 +01:00
spdxcheck-test.sh scripts: add spdxcheck.py self test 2018-12-28 12:11:44 -08:00
spdxcheck.py spdxcheck.py: fix directory structures 2019-06-01 15:51:31 -07:00
spelling.txt scripts/spelling: add a few more typos 2020-06-10 19:14:17 -07:00
sphinx-pre-install scripts: sphinx-pre-install: change the output order 2020-04-28 12:52:52 -06:00
split-man.pl MAINTAINERS & files: Canonize the e-mails I use at files 2018-05-04 06:21:06 -04:00
stackdelta License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
stackusage License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
subarch.include selftests: add headers_install to lib.mk 2018-09-05 08:12:09 -06:00
tags.sh kbuild: add a flag to force absolute path for srctree 2019-07-11 00:05:09 +09:00
tools-support-relr.sh scripts/tools-support-relr.sh: un-quote variables 2019-11-13 10:52:05 +00:00
unifdef.c unifdef: use memcpy instead of strncpy 2018-11-30 14:45:01 -08:00
ver_linux ver_linux: Query ld cache for versions of libc/libcpp run-time 2020-02-10 13:35:15 -08:00
xen-hypercalls.sh License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
xz_wrap.sh kbuild: add variables for compression tools 2020-06-06 23:42:01 +09:00