Waiman Long 34d54f3d69 locking/pvqspinlock: Relax cmpxchg's to improve performance on some architectures ... All the locking related cmpxchg's in the following functions are replaced with the _acquire variants: - pv_queued_spin_steal_lock() - trylock_clear_pending() This change should help performance on architectures that use LL/SC. The cmpxchg in pv_kick_node() is replaced with a relaxed version with explicit memory barrier to make sure that it is fully ordered in the writing of next->lock and the reading of pn->state whether the cmpxchg is a success or failure without affecting performance in non-LL/SC architectures. On a 2-socket 12-core 96-thread Power8 system with pvqspinlock explicitly enabled, the performance of a locking microbenchmark with and without this patch on a 4.13-rc4 kernel with Xinhui's PPC qspinlock patch were as follows: # of thread w/o patch with patch % Change ----------- --------- ---------- -------- 8 5054.8 Mop/s 5209.4 Mop/s +3.1% 16 3985.0 Mop/s 4015.0 Mop/s +0.8% 32 2378.2 Mop/s 2396.0 Mop/s +0.7% Suggested-by: Peter Zijlstra <peterz@infradead.org> Signed-off-by: Waiman Long <longman@redhat.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Cc: Andrea Parri <parri.andrea@gmail.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Pan Xinhui <xinhui@linux.vnet.ibm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Will Deacon <will.deacon@arm.com> Link: http://lkml.kernel.org/r/1502741222-24360-1-git-send-email-longman@redhat.com Signed-off-by: Ingo Molnar <mingo@kernel.org>		2017-08-29 15:14:38 +02:00
..
bpf	bpf: fix bpf_prog_get_info_by_fd to dump correct xlated_prog_len	2017-07-29 23:29:41 -07:00
cgroup	cpuset: Make nr_cpusets private	2017-08-10 12:28:57 +02:00
configs	config: android-base: disable CONFIG_NFSD and CONFIG_NFS_FS	2017-06-09 11:47:38 +02:00
debug	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/debug.h>	2017-03-02 08:42:34 +01:00
events	perf/core: Fix time on IOC_ENABLE	2017-08-10 12:01:09 +02:00
gcov	gcov: support GCC 7.1	2017-05-12 15:57:15 -07:00
irq	genirq/ipi: Fixup checks against nr_cpu_ids	2017-08-20 10:49:05 +02:00
livepatch	livepatch: Fix stacking of patches with respect to RCU	2017-06-20 10:42:19 +02:00
locking	locking/pvqspinlock: Relax cmpxchg's to improve performance on some architectures	2017-08-29 15:14:38 +02:00
power	mm: fix global NR_SLAB_.*CLAIMABLE counter reads	2017-08-10 15:54:06 -07:00
printk	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/pmladek/printk	2017-07-05 11:11:26 -07:00
rcu	rcu: Remove RCU CPU stall warnings from Tiny RCU	2017-06-08 18:52:45 -07:00
sched	smp: Avoid using two cache lines for struct call_single_data	2017-08-29 15:14:38 +02:00
time	timers: Fix overflow in get_next_timer_interrupt	2017-08-01 14:20:53 +02:00
trace	Various bug fixes:	2017-08-24 14:08:22 -07:00
.gitignore	…
Kconfig.freezer	…
Kconfig.hz	…
Kconfig.locks	…
Kconfig.preempt	…
Makefile	kernel/watchdog: split up config options	2017-07-12 16:26:02 -07:00
acct.c	sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>	2017-03-02 08:42:39 +01:00
async.c	async: Adjust system_state checks	2017-05-23 10:01:37 +02:00
audit.c	Merge branch 'stable-4.13' of git://git.infradead.org/users/pcmoore/audit	2017-07-20 10:22:26 -07:00
audit.h	audit: style fix	2017-06-12 18:07:43 -04:00
audit_fsnotify.c	Merge branch 'fsnotify' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs	2017-05-03 11:05:15 -07:00
audit_tree.c	Merge branch 'fsnotify' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs	2017-05-03 11:05:15 -07:00
audit_watch.c	audit/stable-4.13 PR 20170816	2017-08-16 16:48:34 -07:00
auditfilter.c	audit: kernel generated netlink traffic should have a portid of 0	2017-05-02 10:16:05 -04:00
auditsc.c	Merge branch 'stable-4.13' of git://git.infradead.org/users/pcmoore/audit	2017-07-05 11:24:05 -07:00
backtracetest.c	…
bounds.c	…
capability.c	…
compat.c	Merge branch 'misc.compat' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs	2017-07-06 20:57:13 -07:00
configs.c	…
context_tracking.c	…
cpu.c	smp/hotplug: Replace BUG_ON and react useful	2017-07-11 22:25:44 +02:00
cpu_pm.c	…
crash_core.c	kdump: protect vmcoreinfo data under the crash memory	2017-07-12 16:26:00 -07:00
crash_dump.c	…
cred.c	doc: ReSTify credentials.txt	2017-05-18 10:30:19 -06:00
delayacct.c	sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>	2017-03-02 08:42:39 +01:00
dma.c	…
elfcore.c	…
exec_domain.c	…
exit.c	locking/lockdep: Implement the 'crossrelease' feature	2017-08-10 12:29:07 +02:00
extable.c	lib/extable.c: use bsearch() library function in search_extable()	2017-07-10 16:32:35 -07:00
fork.c	Merge branch 'linus' into locking/core, to resolve conflicts	2017-08-11 13:51:59 +02:00
freezer.c	…
futex.c	futex: Remove duplicated code and fix undefined behaviour	2017-08-25 22:49:59 +02:00
futex_compat.c	…
groups.c	kernel/groups.c: use sort library function	2017-07-10 16:32:34 -07:00
hung_task.c	kernel/hung_task.c: defer showing held locks	2017-05-08 17:15:10 -07:00
irq_work.c	…
jump_label.c	jump_label: Provide hotplug context variants	2017-08-10 12:28:59 +02:00
kallsyms.c	kernel/kallsyms.c: replace all_var with IS_ENABLED(CONFIG_KALLSYMS_ALL)	2017-07-10 16:32:34 -07:00
kcmp.c	kcmp: add KCMP_EPOLL_TFD mode to compare epoll target files	2017-07-12 16:26:01 -07:00
kcov.c	kcov: simplify interrupt check	2017-05-08 17:15:12 -07:00
kexec.c	kdump: protect vmcoreinfo data under the crash memory	2017-07-12 16:26:00 -07:00
kexec_core.c	kdump: protect vmcoreinfo data under the crash memory	2017-07-12 16:26:00 -07:00
kexec_file.c	kexec_file: adjust declaration of kexec_purgatory	2017-07-12 16:26:02 -07:00
kexec_internal.h	kexec_file: adjust declaration of kexec_purgatory	2017-07-12 16:26:02 -07:00
kmod.c	kmod: fix wait on recursive loop	2017-08-18 15:32:01 -07:00
kprobes.c	kprobes: Ensure that jprobe probepoints are at function entry	2017-07-08 11:05:35 +02:00
ksysfs.c	kexec: move vmcoreinfo out of the kernel's .bss section	2017-07-12 16:25:59 -07:00
kthread.c	cgroup, kthread: close race window where new kthreads can be migrated to non-root cgroups	2017-03-17 10:18:47 -04:00
latencytop.c	sched/headers: Prepare to move sched_info_on() and force_schedstat_enabled() from <linux/sched.h> to <linux/sched/stat.h>	2017-03-02 08:42:39 +01:00
membarrier.c	…
memremap.c	mm, memory_hotplug: replace for_device by want_memblock in arch_add_memory	2017-07-06 16:24:32 -07:00
module-internal.h	…
module.c	Modules updates for v4.13	2017-07-12 17:22:01 -07:00
module_signing.c	…
notifier.c	kernel/notifier.c: simplify expression	2017-02-24 17:46:56 -08:00
nsproxy.c	perf: Add PERF_RECORD_NAMESPACES to include namespaces related info	2017-03-13 15:57:41 -03:00
padata.c	padata: Avoid nested calls to cpus_read_lock() in pcrypt_init_padata()	2017-05-26 10:10:37 +02:00
panic.c	locking/refcounts, x86/asm: Implement fast refcount overflow protection	2017-08-17 10:40:26 +02:00
params.c	boot/param: Move next_arg() function to lib/cmdline.c for later reuse	2017-04-18 10:37:13 +02:00
pid.c	pids: make task_tgid_nr_ns() safe	2017-08-21 12:47:31 -07:00
pid_namespace.c	pid_ns: Sleep in TASK_INTERRUPTIBLE in zap_pid_ns_processes	2017-05-13 17:26:01 -05:00
profile.c	sched/headers: Prepare to move sched_info_on() and force_schedstat_enabled() from <linux/sched.h> to <linux/sched/stat.h>	2017-03-02 08:42:39 +01:00
ptrace.c	ptrace: Properly initialize ptracer_cred on fork	2017-05-23 07:40:44 -05:00
range.c	…
reboot.c	…
relay.c	Merge branch 'work.splice' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs	2017-05-02 11:38:06 -07:00
resource.c	…
seccomp.c	seccomp: Switch from atomic_t to recount_t	2017-06-26 09:24:00 -07:00
signal.c	signal: don't remove SIGNAL_UNKILLABLE for traced tasks.	2017-08-18 15:32:02 -07:00
smp.c	smp: Avoid using two cache lines for struct call_single_data	2017-08-29 15:14:38 +02:00
smpboot.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h>	2017-03-02 08:42:35 +01:00
smpboot.h	…
softirq.c	sched/core: Remove 'task' parameter and rename tsk_restore_flags() to current_restore_flags()	2017-04-11 09:06:32 +02:00
stacktrace.c	stacktrace/x86: add function for detecting reliable stack traces	2017-03-08 09:18:02 +01:00
stop_machine.c	stop_machine: Provide stop_machine_cpuslocked()	2017-05-26 10:10:36 +02:00
sys.c	fix a braino in compat_sys_getrlimit()	2017-07-12 09:15:00 -07:00
sys_ni.c	…
sysctl.c	kernel/watchdog: split up config options	2017-07-12 16:26:02 -07:00
sysctl_binary.c	kernel/sysctl_binary.c: check name array length in deprecated_sysctl_warning()	2017-07-12 16:26:00 -07:00
task_work.c	…
taskstats.c	taskstats: add e/u/stime for TGID command	2017-05-08 17:15:12 -07:00
test_kprobes.c	…
torture.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/clock.h>	2017-03-02 08:42:27 +01:00
tracepoint.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h>	2017-03-02 08:42:35 +01:00
tsacct.c	sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>	2017-03-02 08:42:39 +01:00
ucount.c	ucount: Remove the atomicity from ucount->count	2017-03-06 15:26:37 -06:00
uid16.c	sched/headers: Prepare to remove <linux/cred.h> inclusion from <linux/sched.h>	2017-03-02 08:42:31 +01:00
up.c	smp: Avoid using two cache lines for struct call_single_data	2017-08-29 15:14:38 +02:00
user-return-notifier.c	…
user.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/user.h>	2017-03-02 08:42:29 +01:00
user_namespace.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/signal.h>	2017-03-02 08:42:29 +01:00
utsname.c	sched/headers: Prepare to move the task_lock()/unlock() APIs to <linux/sched/task.h>	2017-03-02 08:42:38 +01:00
utsname_sysctl.c	sched/headers: Remove <linux/rwsem.h> from <linux/sched.h>	2017-03-03 01:45:36 +01:00
watchdog.c	kernel/watchdog: Prevent false positives with turbo modes	2017-08-18 12:35:02 +02:00
watchdog_hld.c	kernel/watchdog: Prevent false positives with turbo modes	2017-08-18 12:35:02 +02:00
workqueue.c	locking/lockdep: Untangle xhlock history save/restore from task independence	2017-08-29 15:14:38 +02:00
workqueue_internal.h	…