linux/arch
Alexander Duyck 5470dea49f mm: use mm_zero_struct_page from SPARC on all 64b architectures
Patch series "Deferred page init improvements", v7.

This patchset is essentially a refactor of the page initialization logic
that is meant to provide for better code reuse while providing a
significant improvement in deferred page initialization performance.

In my testing on an x86_64 system with 384GB of RAM I have seen the
following.  In the case of regular memory initialization the deferred init
time was decreased from 3.75s to 1.38s on average.  This amounts to a 172%
improvement for the deferred memory initialization performance.

I have called out the improvement observed with each patch.

This patch (of 4):

Use the same approach that was already in use on Sparc on all the
architectures that support a 64b long.

This is mostly motivated by the fact that 7 to 10 store/move instructions
are likely always going to be faster than having to call into a function
that is not specialized for handling page init.

An added advantage to doing it this way is that the compiler can get away
with combining writes in the __init_single_page call.  As a result the
memset call will be reduced to only about 4 write operations, or at least
that is what I am seeing with GCC 6.2 as the flags, LRU pointers, and
count/mapcount seem to be cancelling out at least 4 of the 8 assignments
on my system.

One change I had to make to the function was to reduce the minimum page
size to 56 to support some powerpc64 configurations.

This change should introduce no change on SPARC since it already had this
code.  In the case of x86_64 I saw a reduction from 3.75s to 2.80s when
initializing 384GB of RAM per node.  Pavel Tatashin tested on a system
with Broadcom's Stingray CPU and 48GB of RAM and found that
__init_single_page() takes 19.30ns / 64-byte struct page before this patch
and with this patch it takes 17.33ns / 64-byte struct page.  Mike Rapoport
ran a similar test on a OpenPower (S812LC 8348-21C) with Power8 processor
and 128GB or RAM.  His results per 64-byte struct page were 4.68ns before,
and 4.59ns after this patch.

Link: http://lkml.kernel.org/r/20190405221213.12227.9392.stgit@localhost.localdomain
Signed-off-by: Alexander Duyck <alexander.h.duyck@linux.intel.com>
Reviewed-by: Pavel Tatashin <pavel.tatashin@microsoft.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@linux.ibm.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Cc: David S. Miller <davem@davemloft.net>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Khalid Aziz <khalid.aziz@oracle.com>
Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Cc: Laurent Dufour <ldufour@linux.vnet.ibm.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Mel Gorman <mgorman@techsingularity.net>
Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
Cc: Pavel Tatashin <pasha.tatashin@soleen.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: <yi.z.zhang@linux.intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2019-05-14 09:47:49 -07:00
..
alpha init: provide a generic free_initmem implementation 2019-05-14 09:47:47 -07:00
arc init: provide a generic free_initmem implementation 2019-05-14 09:47:47 -07:00
arm initramfs: move the legacy keepinitrd parameter to core code 2019-05-14 09:47:47 -07:00
arm64 hugetlb: allow to free gigantic pages regardless of the configuration 2019-05-14 09:47:47 -07:00
c6x init: provide a generic free_initmem implementation 2019-05-14 09:47:47 -07:00
csky DMA mapping updates for 5.2 2019-05-09 08:40:55 -07:00
h8300 init: provide a generic free_initmem implementation 2019-05-14 09:47:47 -07:00
hexagon hexagon: switch over to generic free_initmem() 2019-05-14 09:47:47 -07:00
ia64 Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next 2019-05-07 22:03:58 -07:00
m68k initramfs: provide a generic free_initrd_mem implementation 2019-05-14 09:47:47 -07:00
microblaze init: provide a generic free_initmem implementation 2019-05-14 09:47:47 -07:00
mips initramfs: poison freed initrd memory 2019-05-14 09:47:47 -07:00
nds32 init: provide a generic free_initmem implementation 2019-05-14 09:47:47 -07:00
nios2 init: provide a generic free_initmem implementation 2019-05-14 09:47:47 -07:00
openrisc init: provide a generic free_initmem implementation 2019-05-14 09:47:47 -07:00
parisc initramfs: provide a generic free_initrd_mem implementation 2019-05-14 09:47:47 -07:00
powerpc hugetlb: allow to free gigantic pages regardless of the configuration 2019-05-14 09:47:47 -07:00
riscv riscv: switch over to generic free_initmem() 2019-05-14 09:47:47 -07:00
s390 hugetlb: allow to free gigantic pages regardless of the configuration 2019-05-14 09:47:47 -07:00
sh hugetlb: allow to free gigantic pages regardless of the configuration 2019-05-14 09:47:47 -07:00
sparc mm: use mm_zero_struct_page from SPARC on all 64b architectures 2019-05-14 09:47:49 -07:00
um initramfs: provide a generic free_initrd_mem implementation 2019-05-14 09:47:47 -07:00
unicore32 init: provide a generic free_initmem implementation 2019-05-14 09:47:47 -07:00
x86 hugetlb: allow to free gigantic pages regardless of the configuration 2019-05-14 09:47:47 -07:00
xtensa init: provide a generic free_initmem implementation 2019-05-14 09:47:47 -07:00
.gitignore
Kconfig initramfs: move the legacy keepinitrd parameter to core code 2019-05-14 09:47:47 -07:00