linux/arch/sh/include/asm
Linus Torvalds 1cf35d4771 mm: split 'tlb_flush_mmu()' into tlb flushing and memory freeing parts
The mmu-gather operation 'tlb_flush_mmu()' has done two things: the
actual tlb flush operation, and the batched freeing of the pages that
the TLB entries pointed at.

This splits the operation into separate phases, so that the forced
batched flushing done by zap_pte_range() can now do the actual TLB flush
while still holding the page table lock, but delay the batched freeing
of all the pages to after the lock has been dropped.

This in turn allows us to avoid a race condition between
set_page_dirty() (as called by zap_pte_range() when it finds a dirty
shared memory pte) and page_mkclean(): because we now flush all the
dirty page data from the TLB's while holding the pte lock,
page_mkclean() will be held up walking the (recently cleaned) page
tables until after the TLB entries have been flushed from all CPU's.

Reported-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Tested-by: Dave Hansen <dave.hansen@intel.com>
Acked-by: Hugh Dickins <hughd@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Russell King - ARM Linux <linux@arm.linux.org.uk>
Cc: Tony Luck <tony.luck@intel.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-04-25 16:05:40 -07:00
..
Kbuild locking/mcs: Allow architecture specific asm files to be used for contended case 2014-02-09 21:18:52 +01:00
adc.h
addrspace.h sh: nommu: use 32-bit phys mode. 2010-11-04 12:32:24 +09:00
alignment.h sh: Split out the unaligned counters and user bits. 2010-01-12 16:12:25 +09:00
asm-offsets.h kbuild: move asm-offsets.h to include/generated 2009-12-12 13:08:14 +01:00
atomic-grb.h sh: consolidate atomic_cmpxchg()/atomic_add_unless() definitions. 2010-01-08 17:02:17 +09:00
atomic-irq.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
atomic-llsc.h sh: consolidate atomic_cmpxchg()/atomic_add_unless() definitions. 2010-01-08 17:02:17 +09:00
atomic.h sh: Fix up tracepoint build fallout from static key introduction. 2012-04-27 11:12:38 +09:30
barrier.h arch: Clean up asm/barrier.h implementations using asm-generic/barrier.h 2014-01-12 10:37:15 +01:00
bitops-grb.h sh: Convert to generic bitops for IRQ-toggling implementation. 2008-12-22 18:42:53 +09:00
bitops-llsc.h sh: Relax inline assembly constraints 2009-01-29 11:57:09 +09:00
bitops-op32.h sh: Provide optimized non-atomic bitops for SH-2A. 2008-12-22 18:42:54 +09:00
bitops.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
bl_bit.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
bl_bit_32.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
bl_bit_64.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
bug.h sh: Consolidate die definitions for trap handlers. 2012-06-14 14:18:51 +09:00
bugs.h sh: Idle loop chainsawing for SMP-based light sleep. 2009-10-16 17:20:58 +09:00
cache.h Merge branch 'for-35' of git://repo.or.cz/linux-kbuild 2010-06-01 08:55:52 -07:00
cache_insns.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
cache_insns_32.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
cache_insns_64.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
cacheflush.h sh: Assume new page cache pages have dirty dcache lines. 2010-12-01 15:39:51 +09:00
checksum.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
checksum_32.h
clkdev.h clk: Add common __clk_get(), __clk_put() implementations 2013-12-04 17:19:44 +01:00
clock.h sh: sh header sh_clk_ops rename 2012-03-12 22:19:10 +01:00
cmpxchg-grb.h sh: Fix up xchg/cmpxchg corruption with gUSA RB. 2011-06-08 15:22:39 +09:00
cmpxchg-irq.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
cmpxchg-llsc.h sh: Fix sh4a llsc-based cmpxchg() 2009-06-11 09:31:55 +03:00
cmpxchg.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
device.h sh: Defer to asm-generic/device.h. 2012-01-13 16:27:38 +09:00
dma-mapping.h sh: dma_debug: add debug_dma_mapping_error support 2012-11-28 15:29:00 +01:00
dma-register.h dmaengine: shdma: separate DMA headers. 2010-03-02 11:09:04 +09:00
dma.h sh: Kill off MAX_DMA_ADDRESS leftovers. 2012-05-19 18:50:09 +09:00
dmabrg.h
dwarf.h sh: Optimise FDE/CIE lookup by using red-black trees 2010-02-08 11:29:15 +09:00
elf.h sh: Fix FDPIC binary loader 2013-01-11 21:02:57 +09:00
entry-macros.S sh: Remove implicit sign extension from assembler immediates 2009-08-24 17:09:53 +09:00
fb.h
fixmap.h sh: use generic fixmap.h 2014-01-23 16:36:54 -08:00
flat.h flat: fix data sections alignment 2009-05-29 08:40:02 -07:00
fpu.h sh: move fpu_counter into ARCH specific thread_struct 2013-11-13 12:09:13 +09:00
freq.h
ftrace.h sh: Provide CALLER_ADDRx definitions even when ftrace is disabled. 2009-10-14 11:49:49 +09:00
futex-irq.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
futex.h futex: Sanitize futex ops argument types 2011-03-11 12:23:31 +01:00
gpio.h sh: Remove unused sh_pfc_register_info() function 2013-01-25 09:24:30 +09:00
hardirq.h sh: Count NMIs in irq_cpustat_t. 2009-10-14 16:42:28 +09:00
hd64461.h sh: mach-hp6xx: Fix up the hp6xx build for hd64461 changes. 2009-05-20 11:27:13 +09:00
heartbeat.h sh: Prevent heartbeat from scribbling over non-LED bits. 2009-08-18 21:16:29 +09:00
hugetlb.h mm/hugetlb: add more arch-defined huge_pte functions 2013-04-29 15:54:33 -07:00
hw_breakpoint.h UAPI: (Scripted) Disintegrate arch/sh/include/asm 2012-10-09 09:47:37 +01:00
hw_irq.h atomic: use <linux/atomic.h> 2011-07-26 16:49:47 -07:00
i2c-sh7760.h sh: edosk7760 evt2irq migration. 2012-05-18 14:52:31 +09:00
io.h Kconfig: rename HAS_IOPORT to HAS_IOPORT_MAP 2014-04-07 16:36:11 -07:00
io_generic.h sh: machvec IO death. 2010-11-01 09:49:04 -04:00
io_noioport.h SH: Convert out[bwl] macros to inline functions 2012-07-12 13:12:13 +09:00
io_trapped.h Kconfig: rename HAS_IOPORT to HAS_IOPORT_MAP 2014-04-07 16:36:11 -07:00
irq.h sh: Kill off machvec IRQ hinting. 2012-05-21 17:54:01 +09:00
irqflags.h Fix IRQ flag handling naming 2010-10-07 14:08:55 +01:00
kdebug.h sh64: Convert to unwinder API. 2012-05-24 15:03:46 +09:00
kexec.h sh: convert kexec crash kernel management to LMB. 2010-05-07 14:54:55 +09:00
kgdb.h sh: kgdb: Unset CACHE_FLUSH_IS_SAFE for SMP. 2012-04-11 10:48:24 +09:00
kmap_types.h kmap_types: make most arches use generic header file 2009-06-16 19:47:51 -07:00
kprobes.h sh: Add kprobe-based event tracer. 2010-06-14 15:16:53 +09:00
linkage.h
machvec.h Kconfig: rename HAS_IOPORT to HAS_IOPORT_MAP 2014-04-07 16:36:11 -07:00
mc146818rtc.h
mmu.h sh: Fix up NUMA build for 29-bit. 2010-03-10 16:29:48 +09:00
mmu_context.h treewide: fix "distingush" typo 2013-10-14 15:38:33 +02:00
mmu_context_32.h sh: Mass ctrl_in/outX to __raw_read/writeX conversion. 2010-01-26 12:58:40 +09:00
mmu_context_64.h
mmzone.h Fix node_start/end_pfn() definition for mm/page_cgroup.c 2011-06-27 14:13:09 -07:00
module.h Make most arch asm/module.h files use asm-generic/module.h 2012-09-28 14:31:03 +09:30
mutex-llsc.h arch: Make __mutex_fastpath_lock_retval return whether fastpath succeeded or not 2013-06-26 12:10:55 +02:00
mutex.h sh: Add SH-4A optimized fastpath mutex implementation. 2008-12-22 18:42:55 +09:00
page.h sh: Fix cached/uncaced address calculation in 29bit mode 2011-11-04 22:13:50 +09:00
pci.h PCI: collapse pcibios_resource_to_bus 2012-02-23 20:19:04 -07:00
perf_event.h irq_work: Add generic hardirq context callbacks 2010-10-18 19:58:50 +02:00
pgalloc.h sh: handle pgtable_page_ctor() fail 2013-11-15 09:32:18 +09:00
pgtable-2level.h sh: Rename split-level pgtable headers. 2010-01-13 19:18:39 +09:00
pgtable-3level.h sh: Rename split-level pgtable headers. 2010-01-13 19:18:39 +09:00
pgtable.h consolidate io_remap_pfn_range definitions 2013-06-29 12:46:35 +04:00
pgtable_32.h Fix common misspellings 2011-03-31 11:26:23 -03:00
pgtable_64.h sh64: Kill off unused fixed I/O mapping window. 2012-05-14 15:16:11 +09:00
posix_types.h UAPI: (Scripted) Disintegrate arch/sh/include/asm 2012-10-09 09:47:37 +01:00
processor.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
processor_32.h sh: move fpu_counter into ARCH specific thread_struct 2013-11-13 12:09:13 +09:00
processor_64.h sh: move fpu_counter into ARCH specific thread_struct 2013-11-13 12:09:13 +09:00
ptrace.h UAPI: (Scripted) Disintegrate arch/sh/include/asm 2012-10-09 09:47:37 +01:00
ptrace_32.h UAPI: (Scripted) Disintegrate arch/sh/include/asm 2012-10-09 09:47:37 +01:00
ptrace_64.h UAPI: (Scripted) Disintegrate arch/sh/include/asm 2012-10-09 09:47:37 +01:00
push-switch.h
reboot.h sh: machine_ops based reboot support. 2010-01-20 16:42:52 +09:00
romimage-macros.h sh: add romimage-macros.h 2009-09-11 13:19:16 +09:00
rtc.h sh: rtc-generic support. 2009-04-27 17:34:39 +09:00
rwsem.h rwsem: Move duplicate function prototypes to linux/rwsem.h 2011-01-27 12:30:39 +01:00
seccomp.h sh: seccomp support. 2008-08-02 04:39:32 +09:00
sections.h mtd/uclinux: Use generic __bss_stop instead of _ebss 2012-06-27 09:59:43 +02:00
segment.h
setup.h UAPI: (Scripted) Disintegrate arch/sh/include/asm 2012-10-09 09:47:37 +01:00
sfp-machine.h
sh7760fb.h
sh_bios.h sh: Kill off more unused sh_bios callbacks. 2010-01-12 15:26:11 +09:00
shmparam.h
siu.h sh: remove unused DMA device pointer from SIU platform data 2012-07-20 11:23:44 +05:30
smc37c93x.h
smp-ops.h sh: CPU hotplug support. 2010-04-26 19:08:55 +09:00
smp.h atomic: use <linux/atomic.h> 2011-07-26 16:49:47 -07:00
sparsemem.h
spi.h
spinlock.h locking: Convert raw_rwlock functions to arch_rwlock 2009-12-14 23:55:32 +01:00
spinlock_types.h locking: Convert raw_rwlock to arch_rwlock 2009-12-14 23:55:32 +01:00
sram.h sh: Provide a generic SRAM pool for tiny memories. 2010-10-15 02:09:00 +09:00
stackprotector.h sh: initial stack protector support. 2012-04-19 15:45:57 +09:00
stacktrace.h sh: remove warning and warning_symbol from struct stacktrace_ops 2011-05-23 14:42:15 +09:00
string.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
string_32.h
string_64.h sh: Add SH-5 optimized memcpy()/memset()/strcpy()/strlen(). 2008-12-22 18:44:05 +09:00
suspend.h SH: cpuidle: check error code at init 2013-04-22 00:35:53 +02:00
switch_to.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
switch_to_32.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
switch_to_64.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
syscall.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
syscall_32.h sh: Add ftrace syscall tracing support 2009-07-06 20:16:33 +09:00
syscall_64.h sh: Fix up spurious syscall restarting. 2009-01-29 11:56:03 +09:00
syscalls.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
syscalls_32.h sh: push extra copy of r0-r2 for syscall parameters 2014-04-03 16:20:52 -07:00
syscalls_64.h sh: switch to generic fork/vfork/clone 2012-11-28 22:36:47 -05:00
thread_info.h preempt: Make PREEMPT_ACTIVE generic 2013-11-13 20:21:47 +01:00
timex.h sh: Only provide a PCLK definition for legacy CPG CPUs. 2009-12-29 11:09:30 +09:00
tlb.h mm: split 'tlb_flush_mmu()' into tlb flushing and memory freeing parts 2014-04-25 16:05:40 -07:00
tlb_64.h sh: Fix up broken kerneldoc comments. 2008-08-04 12:51:06 +09:00
tlbflush.h sh: Provide a global TLB flush for U/I-TLB clear. 2010-07-02 15:44:09 +09:00
topology.h sched/numa: Rewrite the CONFIG_NUMA sched domain support 2012-05-09 15:00:55 +02:00
traps.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
traps_32.h sh: don't pass saved userspace state to exception handlers 2014-04-03 16:20:52 -07:00
traps_64.h sh64: Provide EXPEVT helper. 2012-05-14 13:04:08 +09:00
types.h UAPI: (Scripted) Disintegrate arch/sh/include/asm 2012-10-09 09:47:37 +01:00
uaccess.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
uaccess_32.h sh: use the new generic strnlen_user() function 2012-06-13 10:28:37 +09:00
uaccess_64.h sh: use the new generic strnlen_user() function 2012-06-13 10:28:37 +09:00
unaligned-sh4a.h Fix common misspellings 2011-03-31 11:26:23 -03:00
unaligned.h sh: Convert to asm-generic/unaligned.h. 2009-06-14 23:24:43 +09:00
uncached.h sh: nommu: Support building without an uncached mapping. 2010-11-04 12:46:19 +09:00
unistd.h consolidate cond_syscall and SYSCALL_ALIAS declarations 2013-03-03 22:55:19 -05:00
unwinder.h sh: unwinder: Introduce UNWINDER_BUG() and UNWINDER_BUG_ON() 2009-08-21 13:02:44 +01:00
user.h
vga.h
vmlinux.lds.h sh64: fix up memory offset calculation. 2010-02-12 15:41:45 +09:00
watchdog.h sh: Mass ctrl_in/outX to __raw_read/writeX conversion. 2010-01-26 12:58:40 +09:00
word-at-a-time.h sh: use the new generic strnlen_user() function 2012-06-13 10:28:37 +09:00