linux/kernel/sched
Mel Gorman 37355bdc5a sched/numa: Migrate pages to local nodes quicker early in the lifetime of a task
Automatic NUMA Balancing uses a multi-stage pass to decide whether a page
should migrate to a local node. This filter avoids excessive ping-ponging
if a page is shared or used by threads that migrate cross-node frequently.

Threads inherit both page tables and the preferred node ID from the
parent. This means that threads can trigger hinting faults earlier than
a new task which delays scanning for a number of seconds. As it can be
load balanced very early in its lifetime there can be an unnecessary delay
before it starts migrating thread-local data. This patch migrates private
pages faster early in the lifetime of a thread using the sequence counter
as an identifier of new tasks.

With this patch applied, STREAM performance is the same as 4.17 even though
processes are not spread cross-node prematurely. Other workloads showed
a mix of minor gains and losses. This is somewhat expected most workloads
are not very sensitive to the starting conditions of a process.

                         4.19.0-rc5             4.19.0-rc5                 4.17.0
                         numab-v1r1       fastmigrate-v1r1                vanilla
MB/sec copy     43298.52 (   0.00%)    47335.46 (   9.32%)    47219.24 (   9.06%)
MB/sec scale    30115.06 (   0.00%)    32568.12 (   8.15%)    32527.56 (   8.01%)
MB/sec add      32825.12 (   0.00%)    36078.94 (   9.91%)    35928.02 (   9.45%)
MB/sec triad    32549.52 (   0.00%)    35935.94 (  10.40%)    35969.88 (  10.51%)

Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Reviewed-by: Rik van Riel <riel@surriel.com>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Jirka Hladky <jhladky@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Linux-MM <linux-mm@kvack.org>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20181001100525.29789-3-mgorman@techsingularity.net
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2018-10-02 11:31:33 +02:00
..
Makefile sched/pelt: Move PELT related code in a dedicated file 2018-07-15 23:51:20 +02:00
autogroup.c sched/autogroup: Fix possible Spectre-v1 indexing for sched_prio_to_weight[] 2018-05-05 08:34:42 +02:00
autogroup.h sched/headers: Simplify and clean up header usage in the scheduler 2018-03-04 12:39:29 +01:00
clock.c sched/clock: Disable interrupts when calling generic_sched_clock_init() 2018-07-30 19:33:35 +02:00
completion.c sched/Documentation: Update wake_up() & co. memory-barrier guarantees 2018-07-17 09:30:34 +02:00
core.c sched/numa: Pass destination CPU as a parameter to migrate_task_rq 2018-10-02 09:42:21 +02:00
cpuacct.c sched/headers: Simplify and clean up header usage in the scheduler 2018-03-04 12:39:29 +01:00
cpudeadline.c sched/headers: Simplify and clean up header usage in the scheduler 2018-03-04 12:39:29 +01:00
cpudeadline.h sched/headers: Simplify and clean up header usage in the scheduler 2018-03-04 12:39:29 +01:00
cpufreq.c sched/headers: Simplify and clean up header usage in the scheduler 2018-03-04 12:39:29 +01:00
cpufreq_schedutil.c sched/fair: Remove #ifdefs from scale_rt_capacity() 2018-07-25 11:41:05 +02:00
cpupri.c sched/headers: Simplify and clean up header usage in the scheduler 2018-03-04 12:39:29 +01:00
cpupri.h sched/headers: Simplify and clean up header usage in the scheduler 2018-03-04 12:39:29 +01:00
cputime.c sched/headers: Simplify and clean up header usage in the scheduler 2018-03-04 12:39:29 +01:00
deadline.c sched/numa: Pass destination CPU as a parameter to migrate_task_rq 2018-10-02 09:42:21 +02:00
debug.c sched/debug: Fix potential deadlock when writing to sched_features 2018-09-10 10:13:45 +02:00
fair.c sched/numa: Migrate pages to local nodes quicker early in the lifetime of a task 2018-10-02 11:31:33 +02:00
features.h sched/fair: Update util_est only on util_avg updates 2018-03-20 08:11:09 +01:00
idle.c sched: idle: Avoid retaining the tick when it has been stopped 2018-08-20 11:25:55 +02:00
isolation.c sched/headers: Simplify and clean up header usage in the scheduler 2018-03-04 12:39:29 +01:00
loadavg.c sched/headers: Simplify and clean up header usage in the scheduler 2018-03-04 12:39:29 +01:00
membarrier.c sched/headers: Simplify and clean up header usage in the scheduler 2018-03-04 12:39:29 +01:00
pelt.c sched/core: Use PELT for scale_rt_capacity() 2018-07-16 00:16:25 +02:00
pelt.h sched/irq: Add IRQ utilization tracking 2018-07-15 23:51:21 +02:00
rt.c Merge branch 'sched/urgent' into sched/core, to pick up fixes 2018-07-25 11:29:58 +02:00
sched-pelt.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
sched.h sched/numa: Pass destination CPU as a parameter to migrate_task_rq 2018-10-02 09:42:21 +02:00
stats.c proc: introduce proc_create_seq{,_data} 2018-05-16 07:23:35 +02:00
stats.h sched: Clean up and harmonize the coding style of the scheduler code base 2018-03-03 15:50:21 +01:00
stop_task.c sched: Clean up and harmonize the coding style of the scheduler code base 2018-03-03 15:50:21 +01:00
swait.c sched/swait: Rename to exclusive 2018-06-20 11:35:56 +02:00
topology.c sched/topology: Set correct NUMA topology type 2018-09-10 10:13:45 +02:00
wait.c sched/wait: assert the wait_queue_head lock is held in __wake_up_common 2018-08-22 10:52:47 -07:00
wait_bit.c sched/wait: Improve __var_waitqueue() code generation 2018-03-20 08:23:25 +01:00