linux/arch/ia64/include/asm
Linus Torvalds 1cf35d4771 mm: split 'tlb_flush_mmu()' into tlb flushing and memory freeing parts
The mmu-gather operation 'tlb_flush_mmu()' has done two things: the
actual tlb flush operation, and the batched freeing of the pages that
the TLB entries pointed at.

This splits the operation into separate phases, so that the forced
batched flushing done by zap_pte_range() can now do the actual TLB flush
while still holding the page table lock, but delay the batched freeing
of all the pages to after the lock has been dropped.

This in turn allows us to avoid a race condition between
set_page_dirty() (as called by zap_pte_range() when it finds a dirty
shared memory pte) and page_mkclean(): because we now flush all the
dirty page data from the TLB's while holding the pte lock,
page_mkclean() will be held up walking the (recently cleaned) page
tables until after the TLB entries have been flushed from all CPU's.

Reported-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Tested-by: Dave Hansen <dave.hansen@intel.com>
Acked-by: Hugh Dickins <hughd@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Russell King - ARM Linux <linux@arm.linux.org.uk>
Cc: Tony Luck <tony.luck@intel.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-04-25 16:05:40 -07:00
..
native
sn Disintegrate asm/system.h for IA64 2012-03-28 18:30:02 +01:00
uv Disintegrate asm/system.h for IA64 2012-03-28 18:30:02 +01:00
Kbuild locking/mcs: Allow architecture specific asm files to be used for contended case 2014-02-09 21:18:52 +01:00
acpi-ext.h
acpi.h ia64/xen: Remove Xen support for ia64 2013-12-10 16:11:07 -08:00
agp.h agp: kill phys_to_gart() and gart_to_phys() 2009-08-03 09:05:00 +01:00
asm-offsets.h kbuild: move asm-offsets.h to include/generated 2009-12-12 13:08:14 +01:00
asmmacro.h Rename .data..patch.XXX to .data..patch.XXX. 2010-03-03 11:25:59 +01:00
atomic.h [IA64] Redefine ATOMIC_INIT and ATOMIC64_INIT to drop the casts 2012-07-26 10:55:26 -07:00
barrier.h arch: Introduce smp_load_acquire(), smp_store_release() 2014-01-12 10:37:17 +01:00
bitops.h [IA64] Use asm-generic/bitops/builtin-ffs.h 2013-08-26 14:22:58 -07:00
bug.h
bugs.h
cache.h Rename .data.read_mostly to .data..read_mostly. 2010-03-03 11:26:00 +01:00
cacheflush.h block: add helpers to run flush_dcache_page() against a bio and a request's pages 2009-11-26 09:16:19 +01:00
checksum.h
clocksource.h clocksource: Change __ARCH_HAS_CLOCKSOURCE_DATA to a CONFIG option 2011-07-21 13:34:05 -07:00
cpu.h
cputime.h cputime: Generic on-demand virtual cputime accounting 2013-01-27 19:23:27 +01:00
current.h
cyclone.h
delay.h
device.h driver core / ACPI: Move ACPI support to core device and driver types 2012-11-15 00:28:00 +01:00
div64.h
dma-mapping.h ia64: dma_debug: add debug_dma_mapping_error support 2012-11-28 15:28:58 +01:00
dma.h
dmi.h firmware/dmi_scan: generalize for use by other archs 2014-01-23 16:36:57 -08:00
elf.h default SET_PERSONALITY() in linux/elf.h 2013-02-26 02:46:08 -05:00
emergency-restart.h
esi.h
fb.h
fpswa.h
ftrace.h [IA64] sanity in #include files. Move fnptr to types.h 2010-01-06 15:52:35 -08:00
futex.h Wrong asm register contraints in the futex implementation 2013-03-19 16:14:53 -07:00
gcc_intrin.h UAPI: (Scripted) Disintegrate arch/ia64/include/asm 2012-10-09 09:47:00 +01:00
gpio.h gpiolib/arches: Centralise bolierplate asm/gpio.h 2012-05-11 18:00:14 -06:00
hardirq.h [IA64] Move local_softirq_pending() definition 2010-10-19 13:31:11 +02:00
hpsim.h hpsim, initialize chip for assigned irqs 2012-03-08 12:26:30 -08:00
hugetlb.h mm/hugetlb: add more arch-defined huge_pte functions 2013-04-29 15:54:33 -07:00
hw_irq.h ia64: Remove redundant declaration of irq_desc[] 2011-03-29 14:48:05 +02:00
idle.h [IA64] xen_domu_defconfig: fix build issues/warnings 2009-05-05 11:43:13 -07:00
intrinsics.h UAPI: (Scripted) Disintegrate arch/ia64/include/asm 2012-10-09 09:47:00 +01:00
io.h ia64: add early_memremap() alias for early_ioremap() 2013-09-05 13:29:28 +01:00
iommu.h iommu: Remove group_mf 2012-06-25 13:48:30 +02:00
iommu_table.h ia64, iommu: Add a dummy iommu_table.h file in IA64. 2010-10-08 13:11:11 -07:00
iosapic.h IA64: drivers: remove __dev* attributes. 2013-01-03 15:57:13 -08:00
irq.h ia64: move nr-irqs.h to include/generated 2009-12-12 13:08:14 +01:00
irq_regs.h
irq_remapping.h iommu: rename intr_remapping.[ch] to irq_remapping.[ch] 2012-05-07 14:35:00 +02:00
irqflags.h [IA64] Fix include dependency in asm/irqflags.h 2013-06-17 13:39:52 -07:00
kdebug.h
kexec.h Disintegrate asm/system.h for IA64 2012-03-28 18:30:02 +01:00
kmap_types.h kmap_types: make most arches use generic header file 2009-06-16 19:47:51 -07:00
kprobes.h [IA64] sanity in #include files. Move fnptr to types.h 2010-01-06 15:52:35 -08:00
kregs.h
kvm_host.h kvm/x86: Convert iommu_flags to iommu_noncoherent 2013-10-30 19:02:13 +01:00
libata-portmap.h
linkage.h consolidate cond_syscall and SYSCALL_ALIAS declarations 2013-03-03 22:55:19 -05:00
local.h
local64.h arch: Implement local64_t 2010-06-09 11:12:36 +02:00
machvec.h ia64/xen: Remove Xen support for ia64 2013-12-10 16:11:07 -08:00
machvec_dig.h [IA64] Rename platform_name to ia64_platform_name 2012-07-25 11:47:16 -07:00
machvec_dig_vtd.h [IA64] Rename platform_name to ia64_platform_name 2012-07-25 11:47:16 -07:00
machvec_hpsim.h [IA64] Rename platform_name to ia64_platform_name 2012-07-25 11:47:16 -07:00
machvec_hpzx1.h [IA64] Rename platform_name to ia64_platform_name 2012-07-25 11:47:16 -07:00
machvec_hpzx1_swiotlb.h [IA64] Rename platform_name to ia64_platform_name 2012-07-25 11:47:16 -07:00
machvec_init.h
machvec_sn2.h [IA64] Rename platform_name to ia64_platform_name 2012-07-25 11:47:16 -07:00
machvec_uv.h [IA64] Rename platform_name to ia64_platform_name 2012-07-25 11:47:16 -07:00
mc146818rtc.h
mca.h Fix initialization of CMCI/CMCP interrupts 2013-04-02 09:37:06 -07:00
mca_asm.h Disintegrate asm/system.h for IA64 2012-03-28 18:30:02 +01:00
meminit.h ia64/xen: Remove Xen support for ia64 2013-12-10 16:11:07 -08:00
mman.h UAPI: (Scripted) Disintegrate arch/ia64/include/asm 2012-10-09 09:47:00 +01:00
mmu.h
mmu_context.h
mmzone.h [IA64] removing redundant ifdef 2010-05-18 14:45:52 -07:00
module.h Make most arch asm/module.h files use asm-generic/module.h 2012-09-28 14:31:03 +09:30
msidef.h
mutex.h arch: Make __mutex_fastpath_lock_retval return whether fastpath succeeded or not 2013-06-26 12:10:55 +02:00
nodedata.h
numa.h Fix build error for numa_clear_node() under IA64 2013-04-02 09:39:48 -07:00
page.h Disintegrate asm/system.h for IA64 2012-03-28 18:30:02 +01:00
pal.h Fix common misspellings 2011-03-31 11:26:23 -03:00
param.h UAPI: (Scripted) Disintegrate arch/ia64/include/asm 2012-10-09 09:47:00 +01:00
paravirt.h ia64/xen: Remove Xen support for ia64 2013-12-10 16:11:07 -08:00
paravirt_patch.h
paravirt_privop.h [IA64] fix allmodconfig compilation breakage. 2009-04-20 09:46:29 -07:00
parport.h IA64: drivers: remove __dev* attributes. 2013-01-03 15:57:13 -08:00
patch.h
pci.h ia64: Remove acpi_get_pxm() usage 2014-02-03 10:39:22 -07:00
percpu.h Merge branch 'for-35' of git://repo.or.cz/linux-kbuild 2010-06-01 08:55:52 -07:00
perfmon.h UAPI: (Scripted) Disintegrate arch/ia64/include/asm 2012-10-09 09:47:00 +01:00
pgalloc.h ia64: handle pgtable_page_ctor() fail 2013-11-15 09:32:17 +09:00
pgtable.h consolidate io_remap_pfn_range definitions 2013-06-29 12:46:35 +04:00
processor.h printk/cache: mark printk_once test variable __read_mostly 2014-01-23 16:36:56 -08:00
ptrace.h new helper: current_user_stack_pointer() 2012-12-19 18:07:39 -05:00
pvclock-abi.h ia64/xen: Remove Xen support for ia64 2013-12-10 16:11:07 -08:00
rwsem.h rwsem: Move duplicate function prototypes to linux/rwsem.h 2011-01-27 12:30:39 +01:00
sal.h Disintegrate asm/system.h for IA64 2012-03-28 18:30:02 +01:00
scatterlist.h remove needless ISA_DMA_THRESHOLD 2010-08-07 18:15:50 +02:00
sections.h
segment.h
serial.h
shmparam.h
siginfo.h UAPI: (Scripted) Disintegrate arch/ia64/include/asm 2012-10-09 09:47:00 +01:00
signal.h consolidate kernel-side struct sigaction declarations 2013-02-03 15:09:22 -05:00
smp.h IA64: drivers: remove __dev* attributes. 2013-01-03 15:57:13 -08:00
sparsemem.h
spinlock.h lockref: Implement lockref for Itanium 2013-09-04 17:18:56 -07:00
spinlock_types.h locking: Convert raw_rwlock to arch_rwlock 2009-12-14 23:55:32 +01:00
string.h
swiotlb.h swiotlb: Remove duplicate swiotlb_force extern declarations 2009-11-15 09:03:10 +01:00
switch_to.h cputime: Consolidate vtime handling on context switch 2012-08-20 13:05:28 +02:00
syscall.h [IA64] Remove COMPAT_IA32 support 2010-02-08 10:42:17 -08:00
termios.h UAPI: (Scripted) Disintegrate arch/ia64/include/asm 2012-10-09 09:47:00 +01:00
thread_info.h preempt: Make PREEMPT_ACTIVE generic 2013-11-13 20:21:47 +01:00
timex.h
tlb.h mm: split 'tlb_flush_mmu()' into tlb flushing and memory freeing parts 2014-04-25 16:05:40 -07:00
tlbflush.h
topology.h sched: Remove unused mc_capable() and smt_capable() 2014-03-11 12:05:45 +01:00
types.h UAPI: (Scripted) Disintegrate arch/ia64/include/asm 2012-10-09 09:47:00 +01:00
uaccess.h
unaligned.h
uncached.h
unistd.h [IA64] Wire up new sched_setattr and sched_getattr syscalls 2014-01-28 09:52:53 -08:00
unwind.h
user.h
ustack.h UAPI: (Scripted) Disintegrate arch/ia64/include/asm 2012-10-09 09:47:00 +01:00
vga.h
xor.h