Pull voluntary preemption fixes from Ingo Molnar:
"This tree contains a speedup which is achieved through better
might_sleep()/might_fault() preemption point annotations for uaccess
functions, by Michael S Tsirkin:
1. The only reason uaccess routines might sleep is if they fault.
Make this explicit for all architectures.
2. A voluntary preemption point in uaccess functions means compiler
can't inline them efficiently, this breaks assumptions that they
are very fast and small that e.g. net code seems to make. Remove
this preemption point so behaviour matches with what callers
assume.
3. Accesses (e.g through socket ops) to kernel memory with KERNEL_DS
like net/sunrpc does will never sleep. Remove an unconditinal
might_sleep() in the might_fault() inline in kernel.h (used when
PROVE_LOCKING is not set).
4. Accesses with pagefault_disable() return EFAULT but won't cause
caller to sleep. Check for that and thus avoid might_sleep() when
PROVE_LOCKING is set.
These changes offer a nice speedup for CONFIG_PREEMPT_VOLUNTARY=y
kernels, here's a network bandwidth measurement between a virtual
machine and the host:
before:
incoming: 7122.77 Mb/s
outgoing: 8480.37 Mb/s
after:
incoming: 8619.24 Mb/s [ +21.0% ]
outgoing: 9455.42 Mb/s [ +11.5% ]
I kept these changes in a separate tree, separate from scheduler
changes, because it's a mixed MM and scheduler topic"
* 'sched-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
mm, sched: Allow uaccess in atomic with pagefault_disable()
mm, sched: Drop voluntary schedule from might_fault()
x86: uaccess s/might_sleep/might_fault/
tile: uaccess s/might_sleep/might_fault/
powerpc: uaccess s/might_sleep/might_fault/
mn10300: uaccess s/might_sleep/might_fault/
microblaze: uaccess s/might_sleep/might_fault/
m32r: uaccess s/might_sleep/might_fault/
frv: uaccess s/might_sleep/might_fault/
arm64: uaccess s/might_sleep/might_fault/
asm-generic: uaccess s/might_sleep/might_fault/
arch/microblaze/include/asm/uaccess.h:101:3:
warning: cast removes address space of expression
arch/microblaze/include/asm/uaccess.h:107:2:
warning: cast removes address space of expression
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
The only reason uaccess routines might sleep
is if they fault. Make this explicit.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1369577426-26721-5-git-send-email-mst@redhat.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Fix access_ok macro no to permit
case where user will try to access
the last address space which is equal
to segment address.
Example:
segment addr = 0xbfff ffff
address = 0xbfff fff0
size = 0x10
Current wrong implementation
0xbfff ffff >= (0xbfff fff0 | 0x10 | (0xbfff fff0 + 0x10))
0xbfff ffff >= (0xbfff fff0 | 0xc000 0000)
0xbfff ffff >= 0xf000 0000
return 0 which is access failed even the combination is valid.
because get_fs().seq returns the last valid address.
This patch fix this problem.
Size equals to zero is valid access.
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
All files which uses user unified macros from uaccess.h
(get_user/put_user/clear_user/copy_tofrom_user/
strnlen_user and strncpy_user) generate this
warning messages:
Assembler messages:
Warning: ignoring changed section attributes for .discard
Setting up discard executable section flang for __EX_TABLE_SECTION
macro removed all these warnings.
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
__pu_val must be volatile to ensure that the value is not lost.
It was causing the problem with timerfd syscall
where using inline asm at the end of function call doesn't
save u64 bit value to the stack.
In comparison both cases you can find out this fragment
where you can see the first part which is saved u64
value to stack and then using it in __put_user_asm_8 macro.
Origin broken implementation misses the first two swi instructions.
swi r22, r1, 28 /* missing without volatile */
swi r23, r1, 32
...
addik r4, r1, 28
lwi r3, r4, 0
swi r3, r25, 0
lwi r3, r4, 4
swi r3, r25, 4
addk r3, r0, r0
NOTE: Moving __put_val initialization after declaration
has not impact on this bug. It is just coding style issue.
Signed-off-by: Michal Simek <monstr@monstr.eu>
Use CONFIG_LOWMEM_SIZE if system has larger ram size.
For system with larger ram size, enable HIGMEM support.
Also setup limitation for memblock and use memblock
allocation in lowmem region.
Signed-off-by: Michal Simek <monstr@monstr.eu>
There is the problem with bit OR (|) because for
some combination is addr | size | addr+size equal
to seq.
For standard kernel setting (kernel starts at 0xC0000000)
is seq for user space 0xBFFFFFFF and everything below
this limit is fine.
But even address 0xBFFFFFFF is fine because it
is below kernel space.
Signed-off-by: Andrew Fedonczuk <andrew.fedonczuk@ericsson.com>
Signed-off-by: Michal Simek <monstr@monstr.eu>
A userland read of more than PAGE_SIZE bytes from /dev/zero results in
(a) not all of the bytes returned being zero, and
(b) memory corruption due to zeroing of bytes beyond the user buffer.
This is caused by improper constraints on the assembly __clear_user function.
The constrints don't indicate to the compiler that the pointer argument is
modified. Since the function is inline, this results in double-incrementing
of the pointer when __clear_user() is invoked through a multi-page read() of
/dev/zero.
Signed-off-by: Steven J. Magnani <steve@digidescorp.com>
Acked-by: Michal Simek <monstr@monstr.eu>
CC: stable@kernel.org
__copy_to/from_user_inatomic should call __copy_to/from_user
because there is not necessary to check access because of kernel function.
might_sleep in copy_to/from_user macros is causing problems
in debug sessions too (CONFIG_DEBUG_SPINLOCK_SLEEP).
BUG: sleeping function called from invalid context at
.../arch/microblaze/include/asm/uaccess.h:388
in_atomic(): 1, irqs_disabled(): 0, pid: 1, name: swapper
1 lock held by swapper/1:
#0: (&p->cred_guard_mutex){......}, at: [<c00d4b90>] prepare_bprm_creds+0x2c/0x88
Kernel Stack:
...
Call Trace:
[<c0006bd4>] microblaze_unwind+0x7c/0x94
[<c0006684>] show_stack+0xf4/0x190
[<c0006730>] dump_stack+0x10/0x30
[<c00103a0>] __might_sleep+0x12c/0x160
[<c0090de4>] file_read_actor+0x1d8/0x2a8
[<c0091568>] generic_file_aio_read+0x6b4/0xa64
[<c00cd778>] do_sync_read+0xac/0x110
[<c00ce254>] vfs_read+0xc8/0x160
[<c00d585c>] kernel_read+0x38/0x64
[<c00d5984>] prepare_binprm+0xfc/0x130
[<c00d6430>] do_execve+0x228/0x370
[<c000614c>] microblaze_execve+0x58/0xa4
caused by file_read_actor (mm/filemap.c) which calls
__copy_to_user_inatomic.
Signed-off-by: Michal Simek <monstr@monstr.eu>
The Microblaze implementations of get_user() and (MMU) put_user() evaluate
the address argument more than once. This causes unexpected side-effects for
invocations that include increment operators, i.e. get_user(foo, bar++).
This patch also removes the distinction between MMU and noMMU put_user().
Without the patch:
$ echo 1234567890 > /proc/sys/kernel/core_pattern
$ cat /proc/sys/kernel/core_pattern
12345
Signed-off-by: Steven J. Magnani <steve@digidescorp.com>
Here is small regression on dhrystone tests and I think
that on all benchmarking tests. It is due to better checking
mechanism in put_user macro
Signed-off-by: Michal Simek <monstr@monstr.eu>
This is the first patch which does uaccess unification.
I choosed to do several patches to be able to use bisect
in future if any fault happens.
Signed-off-by: Michal Simek <monstr@monstr.eu>
This is first patch which clear part of uaccess.h.
uaccess.h will be clear later.
Signed-off-by: John Williams <john.williams@petalogix.com>
Signed-off-by: Michal Simek <monstr@monstr.eu>
For 64bits arguments gcc caused that put_user macro
works with wrong value because of optimalization.
Adding volatile caused that gcc not optimized it.
It is possible to use (as Blackfin do) two put_user
macros with 32bits arguments but there is one more
instruction which is due to duplication zero return
value which is called put_user_asm macro.
Signed-off-by: Michal Simek <monstr@monstr.eu>
This function was actually causing harm, by hiding
errors about invalid sized get_user/put_user accesses.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Michal Simek <monstr@monstr.eu>
Reviewed-by: Ingo Molnar <mingo@elte.hu>
Acked-by: Stephen Neuendorffer <stephen.neuendorffer@xilinx.com>
Acked-by: John Linn <john.linn@xilinx.com>
Acked-by: John Williams <john.williams@petalogix.com>
Signed-off-by: Michal Simek <monstr@monstr.eu>
Linus Torvalds