openkylin
/
linux
mirror of https://gitee.com/openkylin/linux.git
9
0
Fork 0
Code Issues Projects Releases Wiki Activity
linux/arch/arm/lib/copy_from_user.S

112 lines
2.1 KiB
ArmAsm
Raw Normal View History

[ARM] 2948/1: new preemption safe copy_{to|from}_user implementation Patch from Nicolas Pitre This patch provides a preemption safe implementation of copy_to_user and copy_from_user based on the copy template also used for memcpy. It is enabled unconditionally when CONFIG_PREEMPT=y. Otherwise if the configured architecture is not ARMv3 then it is enabled as well as it gives better performances at least on StrongARM and XScale cores. If ARMv3 is not too affected or if it doesn't matter too much then uaccess.S could be removed altogether. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2005-11-02 03:52:24 +08:00
/*
* linux/arch/arm/lib/copy_from_user.S
*
* Author: Nicolas Pitre
* Created: Sep 29, 2005
* Copyright: MontaVista Software, Inc.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*/
#include <linux/linkage.h>
#include <asm/assembler.h>
ARM: 8225/1: Add unwinding support for memory copy functions The memory copy functions(memcpy, __copy_from_user, __copy_to_user) never had unwinding annotations added. Currently, when accessing invalid pointer by these functions occurs the backtrace shown will stop at these functions or some completely unrelated function. Add unwinding annotations in hopes of getting a more useful backtrace in following cases: 1. die on accessing invalid pointer by these functions 2. kprobe trapped at any instruction within these functions 3. interrupted at any instruction within these functions Signed-off-by: Lin Yongting <linyongting@gmail.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2014-11-26 21:38:33 +08:00
#include <asm/unwind.h>
[ARM] 2948/1: new preemption safe copy_{to|from}_user implementation Patch from Nicolas Pitre This patch provides a preemption safe implementation of copy_to_user and copy_from_user based on the copy template also used for memcpy. It is enabled unconditionally when CONFIG_PREEMPT=y. Otherwise if the configured architecture is not ARMv3 then it is enabled as well as it gives better performances at least on StrongARM and XScale cores. If ARMv3 is not too affected or if it doesn't matter too much then uaccess.S could be removed altogether. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2005-11-02 03:52:24 +08:00
/*
* Prototype:
*
ARM: uaccess: provide uaccess_save_and_enable() and uaccess_restore() Provide uaccess_save_and_enable() and uaccess_restore() to permit control of userspace visibility to the kernel, and hook these into the appropriate places in the kernel where we need to access userspace. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2015-08-19 18:02:28 +08:00
* size_t arm_copy_from_user(void *to, const void *from, size_t n)
[ARM] 2948/1: new preemption safe copy_{to|from}_user implementation Patch from Nicolas Pitre This patch provides a preemption safe implementation of copy_to_user and copy_from_user based on the copy template also used for memcpy. It is enabled unconditionally when CONFIG_PREEMPT=y. Otherwise if the configured architecture is not ARMv3 then it is enabled as well as it gives better performances at least on StrongARM and XScale cores. If ARMv3 is not too affected or if it doesn't matter too much then uaccess.S could be removed altogether. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2005-11-02 03:52:24 +08:00
*
* Purpose:
*
* copy a block to kernel memory from user memory
*
* Params:
*
* to = kernel memory
* from = user memory
* n = number of bytes to copy
*
* Return value:
*
* Number of bytes NOT copied.
*/
Thumb-2: Implement the unified arch/arm/lib functions This patch adds the ARM/Thumb-2 unified support for the arch/arm/lib/* files. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2009-07-24 19:32:57 +08:00
#ifndef CONFIG_THUMB2_KERNEL
#define LDR1W_SHIFT 0
#else
#define LDR1W_SHIFT 1
#endif
#define STR1W_SHIFT 0
[ARM] 2948/1: new preemption safe copy_{to|from}_user implementation Patch from Nicolas Pitre This patch provides a preemption safe implementation of copy_to_user and copy_from_user based on the copy template also used for memcpy. It is enabled unconditionally when CONFIG_PREEMPT=y. Otherwise if the configured architecture is not ARMv3 then it is enabled as well as it gives better performances at least on StrongARM and XScale cores. If ARMv3 is not too affected or if it doesn't matter too much then uaccess.S could be removed altogether. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2005-11-02 03:52:24 +08:00
.macro ldr1w ptr reg abort
Thumb-2: Implement the unified arch/arm/lib functions This patch adds the ARM/Thumb-2 unified support for the arch/arm/lib/* files. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2009-07-24 19:32:57 +08:00
ldrusr \reg, \ptr, 4, abort=\abort
[ARM] 2948/1: new preemption safe copy_{to|from}_user implementation Patch from Nicolas Pitre This patch provides a preemption safe implementation of copy_to_user and copy_from_user based on the copy template also used for memcpy. It is enabled unconditionally when CONFIG_PREEMPT=y. Otherwise if the configured architecture is not ARMv3 then it is enabled as well as it gives better performances at least on StrongARM and XScale cores. If ARMv3 is not too affected or if it doesn't matter too much then uaccess.S could be removed altogether. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2005-11-02 03:52:24 +08:00
.endm
.macro ldr4w ptr reg1 reg2 reg3 reg4 abort
ldr1w \ptr, \reg1, \abort
ldr1w \ptr, \reg2, \abort
ldr1w \ptr, \reg3, \abort
ldr1w \ptr, \reg4, \abort
.endm
.macro ldr8w ptr reg1 reg2 reg3 reg4 reg5 reg6 reg7 reg8 abort
ldr4w \ptr, \reg1, \reg2, \reg3, \reg4, \abort
ldr4w \ptr, \reg5, \reg6, \reg7, \reg8, \abort
.endm
.macro ldr1b ptr reg cond=al abort
Thumb-2: Implement the unified arch/arm/lib functions This patch adds the ARM/Thumb-2 unified support for the arch/arm/lib/* files. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2009-07-24 19:32:57 +08:00
ldrusr \reg, \ptr, 1, \cond, abort=\abort
[ARM] 2948/1: new preemption safe copy_{to|from}_user implementation Patch from Nicolas Pitre This patch provides a preemption safe implementation of copy_to_user and copy_from_user based on the copy template also used for memcpy. It is enabled unconditionally when CONFIG_PREEMPT=y. Otherwise if the configured architecture is not ARMv3 then it is enabled as well as it gives better performances at least on StrongARM and XScale cores. If ARMv3 is not too affected or if it doesn't matter too much then uaccess.S could be removed altogether. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2005-11-02 03:52:24 +08:00
.endm
.macro str1w ptr reg abort
Thumb-2: Implement the unified arch/arm/lib functions This patch adds the ARM/Thumb-2 unified support for the arch/arm/lib/* files. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2009-07-24 19:32:57 +08:00
W(str) \reg, [\ptr], #4
[ARM] 2948/1: new preemption safe copy_{to|from}_user implementation Patch from Nicolas Pitre This patch provides a preemption safe implementation of copy_to_user and copy_from_user based on the copy template also used for memcpy. It is enabled unconditionally when CONFIG_PREEMPT=y. Otherwise if the configured architecture is not ARMv3 then it is enabled as well as it gives better performances at least on StrongARM and XScale cores. If ARMv3 is not too affected or if it doesn't matter too much then uaccess.S could be removed altogether. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2005-11-02 03:52:24 +08:00
.endm
.macro str8w ptr reg1 reg2 reg3 reg4 reg5 reg6 reg7 reg8 abort
stmia \ptr!, {\reg1, \reg2, \reg3, \reg4, \reg5, \reg6, \reg7, \reg8}
.endm
.macro str1b ptr reg cond=al abort
str\cond\()b \reg, [\ptr], #1
.endm
.macro enter reg1 reg2
mov r3, #0
stmdb sp!, {r0, r2, r3, \reg1, \reg2}
.endm
ARM: 8225/1: Add unwinding support for memory copy functions The memory copy functions(memcpy, __copy_from_user, __copy_to_user) never had unwinding annotations added. Currently, when accessing invalid pointer by these functions occurs the backtrace shown will stop at these functions or some completely unrelated function. Add unwinding annotations in hopes of getting a more useful backtrace in following cases: 1. die on accessing invalid pointer by these functions 2. kprobe trapped at any instruction within these functions 3. interrupted at any instruction within these functions Signed-off-by: Lin Yongting <linyongting@gmail.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2014-11-26 21:38:33 +08:00
.macro usave reg1 reg2
UNWIND( .save {r0, r2, r3, \reg1, \reg2} )
.endm
[ARM] 2948/1: new preemption safe copy_{to|from}_user implementation Patch from Nicolas Pitre This patch provides a preemption safe implementation of copy_to_user and copy_from_user based on the copy template also used for memcpy. It is enabled unconditionally when CONFIG_PREEMPT=y. Otherwise if the configured architecture is not ARMv3 then it is enabled as well as it gives better performances at least on StrongARM and XScale cores. If ARMv3 is not too affected or if it doesn't matter too much then uaccess.S could be removed altogether. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2005-11-02 03:52:24 +08:00
.macro exit reg1 reg2
add sp, sp, #8
ldmfd sp!, {r0, \reg1, \reg2}
.endm
.text
ARM: uaccess: provide uaccess_save_and_enable() and uaccess_restore() Provide uaccess_save_and_enable() and uaccess_restore() to permit control of userspace visibility to the kernel, and hook these into the appropriate places in the kernel where we need to access userspace. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2015-08-19 18:02:28 +08:00
ENTRY(arm_copy_from_user)
ARM: spectre-v1: mitigate user accesses Spectre variant 1 attacks are about this sequence of pseudo-code: index = load(user-manipulated pointer); access(base + index * stride); In order for the cache side-channel to work, the access() must me made to memory which userspace can detect whether cache lines have been loaded. On 32-bit ARM, this must be either user accessible memory, or a kernel mapping of that same user accessible memory. The problem occurs when the load() speculatively loads privileged data, and the subsequent access() is made to user accessible memory. Any load() which makes use of a user-maniplated pointer is a potential problem if the data it has loaded is used in a subsequent access. This also applies for the access() if the data loaded by that access is used by a subsequent access. Harden the get_user() accessors against Spectre attacks by forcing out of bounds addresses to a NULL pointer. This prevents get_user() being used as the load() step above. As a side effect, put_user() will also be affected even though it isn't implicated. Also harden copy_from_user() by redoing the bounds check within the arm_copy_from_user() code, and NULLing the pointer if out of bounds. Acked-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2018-05-14 16:40:24 +08:00
#ifdef CONFIG_CPU_SPECTRE
get_thread_info r3
ldr r3, [r3, #TI_ADDR_LIMIT]
ARM: 8796/1: spectre-v1,v1.1: provide helpers for address sanitization Introduce C and asm helpers to sanitize user address, taking the address range they target into account. Use asm helper for existing sanitization in __copy_from_user(). Signed-off-by: Julien Thierry <julien.thierry@arm.com> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2018-09-11 17:14:50 +08:00
uaccess_mask_range_ptr r1, r2, r3, ip
ARM: spectre-v1: mitigate user accesses Spectre variant 1 attacks are about this sequence of pseudo-code: index = load(user-manipulated pointer); access(base + index * stride); In order for the cache side-channel to work, the access() must me made to memory which userspace can detect whether cache lines have been loaded. On 32-bit ARM, this must be either user accessible memory, or a kernel mapping of that same user accessible memory. The problem occurs when the load() speculatively loads privileged data, and the subsequent access() is made to user accessible memory. Any load() which makes use of a user-maniplated pointer is a potential problem if the data it has loaded is used in a subsequent access. This also applies for the access() if the data loaded by that access is used by a subsequent access. Harden the get_user() accessors against Spectre attacks by forcing out of bounds addresses to a NULL pointer. This prevents get_user() being used as the load() step above. As a side effect, put_user() will also be affected even though it isn't implicated. Also harden copy_from_user() by redoing the bounds check within the arm_copy_from_user() code, and NULLing the pointer if out of bounds. Acked-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2018-05-14 16:40:24 +08:00
#endif
[ARM] 2948/1: new preemption safe copy_{to|from}_user implementation Patch from Nicolas Pitre This patch provides a preemption safe implementation of copy_to_user and copy_from_user based on the copy template also used for memcpy. It is enabled unconditionally when CONFIG_PREEMPT=y. Otherwise if the configured architecture is not ARMv3 then it is enabled as well as it gives better performances at least on StrongARM and XScale cores. If ARMv3 is not too affected or if it doesn't matter too much then uaccess.S could be removed altogether. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2005-11-02 03:52:24 +08:00
#include "copy_template.S"
ARM: uaccess: provide uaccess_save_and_enable() and uaccess_restore() Provide uaccess_save_and_enable() and uaccess_restore() to permit control of userspace visibility to the kernel, and hook these into the appropriate places in the kernel where we need to access userspace. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2015-08-19 18:02:28 +08:00
ENDPROC(arm_copy_from_user)
[ARM] 5227/1: Add the ENDPROC declarations to the .S files This declaration specifies the "function" type and size for various assembly functions, mainly needed for generating the correct branch instructions in Thumb-2. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-08-28 18:22:32 +08:00
ARM: fix build error in arch/arm/kernel/process.c /tmp/ccJ3ssZW.s: Assembler messages: /tmp/ccJ3ssZW.s:1952: Error: can't resolve `.text' {.text section} - `.LFB1077' This is caused because: .section .data .section .text .section .text .previous does not return us to the .text section, but the .data section; this makes use of .previous dangerous if the ordering of previous sections is not known. Fix up the other users of .previous; .pushsection and .popsection are a safer pairing to use than .section and .previous. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-04-19 17:15:03 +08:00
.pushsection .fixup,"ax"
[ARM] 2948/1: new preemption safe copy_{to|from}_user implementation Patch from Nicolas Pitre This patch provides a preemption safe implementation of copy_to_user and copy_from_user based on the copy template also used for memcpy. It is enabled unconditionally when CONFIG_PREEMPT=y. Otherwise if the configured architecture is not ARMv3 then it is enabled as well as it gives better performances at least on StrongARM and XScale cores. If ARMv3 is not too affected or if it doesn't matter too much then uaccess.S could be removed altogether. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2005-11-02 03:52:24 +08:00
.align 0
copy_abort_preamble
arm: don't zero in __copy_from_user_inatomic()/__copy_from_user() adjust copy_from_user(), obviously Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2016-09-11 04:44:03 +08:00
ldmfd sp!, {r1, r2, r3}
sub r0, r0, r1
rsb r0, r0, r2
[ARM] 2948/1: new preemption safe copy_{to|from}_user implementation Patch from Nicolas Pitre This patch provides a preemption safe implementation of copy_to_user and copy_from_user based on the copy template also used for memcpy. It is enabled unconditionally when CONFIG_PREEMPT=y. Otherwise if the configured architecture is not ARMv3 then it is enabled as well as it gives better performances at least on StrongARM and XScale cores. If ARMv3 is not too affected or if it doesn't matter too much then uaccess.S could be removed altogether. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2005-11-02 03:52:24 +08:00
copy_abort_end
ARM: fix build error in arch/arm/kernel/process.c /tmp/ccJ3ssZW.s: Assembler messages: /tmp/ccJ3ssZW.s:1952: Error: can't resolve `.text' {.text section} - `.LFB1077' This is caused because: .section .data .section .text .section .text .previous does not return us to the .text section, but the .data section; this makes use of .previous dangerous if the ordering of previous sections is not known. Fix up the other users of .previous; .pushsection and .popsection are a safer pairing to use than .section and .previous. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-04-19 17:15:03 +08:00
.popsection
[ARM] 2948/1: new preemption safe copy_{to|from}_user implementation Patch from Nicolas Pitre This patch provides a preemption safe implementation of copy_to_user and copy_from_user based on the copy template also used for memcpy. It is enabled unconditionally when CONFIG_PREEMPT=y. Otherwise if the configured architecture is not ARMv3 then it is enabled as well as it gives better performances at least on StrongARM and XScale cores. If ARMv3 is not too affected or if it doesn't matter too much then uaccess.S could be removed altogether. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2005-11-02 03:52:24 +08:00