linux/arch/x86/mm
Peter Zijlstra b945d6b255 rbtree: Undo augmented trees performance damage and regression
Reimplement augmented RB-trees without sprinkling extra branches
all over the RB-tree code (which lives in the scheduler hot path).

This approach is 'borrowed' from Fabio's BFQ implementation and
relies on traversing the rebalance path after the RB-tree-op to
correct the heap property for insertion/removal and make up for
the damage done by the tree rotations.

For insertion the rebalance path is trivially that from the new
node upwards to the root, for removal it is that from the deepest
node in the path from the to be removed node that will still
be around after the removal.

[ This patch also fixes a video driver regression reported by
  Ali Gholami Rudi - the memtype->subtree_max_end was updated
  incorrectly. ]

Acked-by: Suresh Siddha <suresh.b.siddha@intel.com>
Acked-by: Venkatesh Pallipadi <venki@google.com>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Tested-by: Ali Gholami Rudi <ali@rudi.ir>
Cc: Fabio Checconi <fabio@gandalf.sssup.it>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
LKML-Reference: <1275414172.27810.27961.camel@twins>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-07-05 14:43:50 +02:00
..
kmemcheck kmemcheck: Test the full object in kmemcheck_is_obj_initialized() 2010-02-17 21:39:08 +02:00
Makefile x86, pat: Migrate to rbtree only backend for pat memtype management 2010-02-18 15:41:36 -08:00
dump_pagetables.c x86: remove (null) in /sys kernel_page_tables 2009-04-14 11:50:22 +02:00
extable.c x86, 64-bit: Move K8 B step iret fixup to fault entry asm 2009-10-12 18:29:46 +02:00
fault.c Merge branch 'x86-debug-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2009-12-05 15:33:27 -08:00
gup.c x86, doc: Fix minor spelling error in arch/x86/mm/gup.c 2010-02-02 16:00:44 -08:00
highmem_32.c Merge branch 'kvm-updates/2.6.32' of git://git.kernel.org/pub/scm/virt/kvm/kvm 2009-09-14 17:43:43 -07:00
hugetlbpage.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
init.c Merge branch 'master' into export-slabh 2010-04-05 11:37:28 +09:00
init_32.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
init_64.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
iomap_32.c x86, pat: Add PAT reserve free to io_mapping* APIs 2009-08-26 15:41:16 -07:00
ioremap.c x86: Fix 'reservetop=' functionality 2010-04-30 12:19:53 +02:00
k8topology_64.c x86: Move find_smp_config() earlier and avoid bootmem usage 2009-11-24 12:10:51 +01:00
kmmio.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
memtest.c x86: memtest: use pointers of equal type for comparison 2009-06-11 16:26:35 +02:00
mmap.c x86: Use helpers for rlimits 2010-01-27 15:17:31 -08:00
mmio-mod.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
numa.c x86/mm: Remove unused DBG() macro 2010-05-31 10:01:53 +02:00
numa_32.c x86: Make 32bit support NO_BOOTMEM 2010-02-12 09:42:39 -08:00
numa_64.c numa: x86_64: use generic percpu var numa_node_id() implementation 2010-05-27 09:12:57 -07:00
pageattr-test.c x86: make sure the CPA test code's use of _PAGE_UNUSED1 is obvious 2008-09-05 17:09:57 +02:00
pageattr.c Merge branch 'drm-ttm-pool' into drm-core-next 2010-04-20 13:12:28 +10:00
pat.c x86, pat: Proper init of memtype subtree_max_end 2010-06-11 14:12:22 -07:00
pat_internal.h x86, pat: Fix memory leak in free_memtype 2010-05-26 11:26:04 -07:00
pat_rbtree.c rbtree: Undo augmented trees performance damage and regression 2010-07-05 14:43:50 +02:00
pf_in.c x86/mmiotrace: Remove redundant instruction prefix checks 2010-05-23 11:02:43 +02:00
pf_in.h x86 mmiotrace: move files into arch/x86/mm/. 2008-05-24 11:25:37 +02:00
pgtable.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
pgtable_32.c x86: remove last traces of quicklist usage 2010-05-24 13:33:31 -07:00
physaddr.c x86: split __phys_addr out into separate file 2009-09-10 11:48:55 -07:00
physaddr.h x86: split __phys_addr out into separate file 2009-09-10 11:48:55 -07:00
setup_nx.c x86, mm: Report state of NX protections during boot 2009-11-16 13:44:59 -08:00
srat_32.c x86: Fix checking of SRAT when node 0 ram is not from 0 2009-12-16 16:43:37 -08:00
srat_64.c Merge branch 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2010-05-18 09:17:50 -07:00
testmmiotrace.c testmmiotrace.c: Add and use pr_fmt(fmt) 2009-10-12 08:05:41 +02:00
tlb.c x86: Convert tlbstate_lock to raw_spinlock 2010-02-17 18:28:59 +01:00