openkylin
/
linux
mirror of https://gitee.com/openkylin/linux.git
9
0
Fork 0
Code Issues Projects Releases Wiki Activity
linux/arch/arm/kernel/entry-header.S

401 lines
11 KiB
ArmAsm
Raw Normal View History

[PATCH] ARM: make entry*.S includes more logical Move common includes to entry-header, and file specific includes to the relevant file. Signed-off-by: Russell King <rmk@arm.linux.org.uk>
2005-04-26 22:21:02 +08:00
#include <linux/init.h>
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
#include <linux/linkage.h>
#include <asm/assembler.h>
kbuild: arm - use generic asm-offsets.h support Delete obsoleted stuff from arch Makefile and rename constants.h to asm-offsets.h Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
2005-09-10 03:08:59 +08:00
#include <asm/asm-offsets.h>
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
#include <asm/errno.h>
[PATCH] ARM: make entry*.S includes more logical Move common includes to entry-header, and file specific includes to the relevant file. Signed-off-by: Russell King <rmk@arm.linux.org.uk>
2005-04-26 22:21:02 +08:00
#include <asm/thread_info.h>
ARM: ARMv7-M: Add support for exception handling This patch implements the exception handling for the ARMv7-M architecture (pretty different from the A or R profiles). It bases on work done earlier by Catalin for 2.6.33 but was nearly completely rewritten to use a pt_regs layout compatible to the A profile. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Reviewed-by: Jonathan Austin <jonathan.austin@arm.com> Tested-by: Jonathan Austin <jonathan.austin@arm.com> Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
2010-05-22 01:06:42 +08:00
#include <asm/v7m.h>
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
@ Bad Abort numbers
@ -----------------
@
#define BAD_PREFETCH 0
#define BAD_DATA 1
#define BAD_ADDREXCPTN 2
#define BAD_IRQ 3
#define BAD_UNDEFINSTR 4
@
[PATCH] ARM: pt_regs offsets Generate pt_regs S_xx offsets from the structure itself instead of #defining them. Signed-off-by: Russell King <rmk@arm.linux.org.uk>
2005-04-26 22:18:59 +08:00
@ Most of the stack format comes from struct pt_regs, but with
@ the addition of 8 bytes for storing syscall args 5 and 6.
[ARM] 3102/1: ARM EABI: stack pointer must be 64-bit aligned after a CPU exception Patch from Nicolas Pitre The ARM EABI says that the stack pointer has to be 64-bit aligned for reasons already mentioned in patch #3101 when calling C functions. We therefore must verify and adjust sp accordingly when taking an exception from kernel mode since sp might not necessarily be 64-bit aligned if the exception occurs in the middle of a kernel function. If the exception occurs while in user mode then no sp fixup is needed as long as sizeof(struct pt_regs) as well as any additional syscall data stack space remain multiples of 8. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2006-01-15 00:18:08 +08:00
@ This _must_ remain a multiple of 8 for EABI.
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
@
#define S_OFF 8
[PATCH] ARM: pt_regs offsets Generate pt_regs S_xx offsets from the structure itself instead of #defining them. Signed-off-by: Russell King <rmk@arm.linux.org.uk>
2005-04-26 22:18:59 +08:00
/*
* The SWI code relies on the fact that R0 is at the bottom of the stack
* (due to slow/fast restore user regs).
*/
#if S_R0 != 0
#error "Please fix"
#endif
[PATCH] ARM: make entry*.S includes more logical Move common includes to entry-header, and file specific includes to the relevant file. Signed-off-by: Russell King <rmk@arm.linux.org.uk>
2005-04-26 22:21:02 +08:00
.macro zero_fp
#ifdef CONFIG_FRAME_POINTER
mov fp, #0
#endif
.endm
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
#ifdef CONFIG_ALIGNMENT_TRAP
ARM: Avoid writing to control register on every exception If we are not changing the control register value, avoid writing to it. Writes to the control register can be very expensive, taking around a hundred cycles or so. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2014-08-28 20:08:14 +08:00
#define ATRAP(x...) x
#else
#define ATRAP(x...)
#endif
.macro alignment_trap, rtmp1, rtmp2, label
#ifdef CONFIG_ALIGNMENT_TRAP
mrc p15, 0, \rtmp2, c1, c0, 0
ldr \rtmp1, \label
ldr \rtmp1, [\rtmp1]
teq \rtmp1, \rtmp2
mcrne p15, 0, \rtmp1, c1, c0, 0
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
#endif
.endm
ARM: ARMv7-M: Add support for exception handling This patch implements the exception handling for the ARMv7-M architecture (pretty different from the A or R profiles). It bases on work done earlier by Catalin for 2.6.33 but was nearly completely rewritten to use a pt_regs layout compatible to the A profile. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Reviewed-by: Jonathan Austin <jonathan.austin@arm.com> Tested-by: Jonathan Austin <jonathan.austin@arm.com> Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
2010-05-22 01:06:42 +08:00
#ifdef CONFIG_CPU_V7M
/*
* ARMv7-M exception entry/exit macros.
*
* xPSR, ReturnAddress(), LR (R14), R12, R3, R2, R1, and R0 are
* automatically saved on the current stack (32 words) before
* switching to the exception stack (SP_main).
*
* If exception is taken while in user mode, SP_main is
* empty. Otherwise, SP_main is aligned to 64 bit automatically
* (CCR.STKALIGN set).
*
* Linux assumes that the interrupts are disabled when entering an
* exception handler and it may BUG if this is not the case. Interrupts
* are disabled during entry and reenabled in the exit macro.
*
* v7m_exception_slow_exit is used when returning from SVC or PendSV.
* When returning to kernel mode, we don't return from exception.
*/
.macro v7m_exception_entry
@ determine the location of the registers saved by the core during
@ exception entry. Depending on the mode the cpu was in when the
@ exception happend that is either on the main or the process stack.
@ Bit 2 of EXC_RETURN stored in the lr register specifies which stack
@ was used.
tst lr, #EXC_RET_STACK_MASK
mrsne r12, psp
moveq r12, sp
@ we cannot rely on r0-r3 and r12 matching the value saved in the
@ exception frame because of tail-chaining. So these have to be
@ reloaded.
ldmia r12!, {r0-r3}
@ Linux expects to have irqs off. Do it here before taking stack space
cpsid i
sub sp, #S_FRAME_SIZE-S_IP
stmdb sp!, {r0-r11}
@ load saved r12, lr, return address and xPSR.
@ r0-r7 are used for signals and never touched from now on. Clobbering
@ r8-r12 is OK.
mov r9, r12
ldmia r9!, {r8, r10-r12}
@ calculate the original stack pointer value.
@ r9 currently points to the memory location just above the auto saved
@ xPSR.
@ The cpu might automatically 8-byte align the stack. Bit 9
@ of the saved xPSR specifies if stack aligning took place. In this case
@ another 32-bit value is included in the stack.
tst r12, V7M_xPSR_FRAMEPTRALIGN
addne r9, r9, #4
@ store saved r12 using str to have a register to hold the base for stm
str r8, [sp, #S_IP]
add r8, sp, #S_SP
@ store r13-r15, xPSR
stmia r8!, {r9-r12}
@ store old_r0
str r0, [r8]
.endm
/*
* PENDSV and SVCALL are configured to have the same exception
* priorities. As a kernel thread runs at SVCALL execution priority it
* can never be preempted and so we will never have to return to a
* kernel thread here.
*/
.macro v7m_exception_slow_exit ret_r0
cpsid i
ldr lr, =EXC_RET_THREADMODE_PROCESSSTACK
@ read original r12, sp, lr, pc and xPSR
add r12, sp, #S_IP
ldmia r12, {r1-r5}
@ an exception frame is always 8-byte aligned. To tell the hardware if
@ the sp to be restored is aligned or not set bit 9 of the saved xPSR
@ accordingly.
tst r2, #4
subne r2, r2, #4
orrne r5, V7M_xPSR_FRAMEPTRALIGN
biceq r5, V7M_xPSR_FRAMEPTRALIGN
ARM: 8064/1: fix v7-M signal return According to the ARM ARM, the behaviour is UNPREDICTABLE if the PC read from the exception return stack is not half word aligned. See the pseudo code for ExceptionReturn() and PopStack(). The signal handler's address has the bit 0 set, and setup_return() directly writes this to regs->ARM_pc. Current hardware happens to discard this bit, but QEMU's emulation doesn't and this makes processes crash. Mask out bit 0 before the exception return in order to get predictable behaviour. Fixes: 19c4d593f0b4 ("ARM: ARMv7-M: Add support for exception handling") Cc: stable@kernel.org Acked-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de> Signed-off-by: Rabin Vincent <rabin@rab.in> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2014-05-25 00:38:01 +08:00
@ ensure bit 0 is cleared in the PC, otherwise behaviour is
@ unpredictable
bic r4, #1
ARM: ARMv7-M: Add support for exception handling This patch implements the exception handling for the ARMv7-M architecture (pretty different from the A or R profiles). It bases on work done earlier by Catalin for 2.6.33 but was nearly completely rewritten to use a pt_regs layout compatible to the A profile. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Reviewed-by: Jonathan Austin <jonathan.austin@arm.com> Tested-by: Jonathan Austin <jonathan.austin@arm.com> Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
2010-05-22 01:06:42 +08:00
@ write basic exception frame
stmdb r2!, {r1, r3-r5}
ldmia sp, {r1, r3-r5}
.if \ret_r0
stmdb r2!, {r0, r3-r5}
.else
stmdb r2!, {r1, r3-r5}
.endif
@ restore process sp
msr psp, r2
@ restore original r4-r11
ldmia sp!, {r0-r11}
@ restore main sp
add sp, sp, #S_FRAME_SIZE-S_IP
cpsie i
bx lr
.endm
#endif /* CONFIG_CPU_V7M */
Thumb-2: Implementation of the unified start-up and exceptions code This patch implements the ARM/Thumb-2 unified kernel start-up and exception handling code. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2009-07-24 19:32:54 +08:00
@
@ Store/load the USER SP and LR registers by switching to the SYS
@ mode. Useful in Thumb-2 mode where "stm/ldm rd, {sp, lr}^" is not
@ available. Should only be called from SVC mode
@
.macro store_user_sp_lr, rd, rtemp, offset = 0
mrs \rtemp, cpsr
eor \rtemp, \rtemp, #(SVC_MODE ^ SYSTEM_MODE)
msr cpsr_c, \rtemp @ switch to the SYS mode
str sp, [\rd, #\offset] @ save sp_usr
str lr, [\rd, #\offset + 4] @ save lr_usr
eor \rtemp, \rtemp, #(SVC_MODE ^ SYSTEM_MODE)
msr cpsr_c, \rtemp @ switch back to the SVC mode
.endm
.macro load_user_sp_lr, rd, rtemp, offset = 0
mrs \rtemp, cpsr
eor \rtemp, \rtemp, #(SVC_MODE ^ SYSTEM_MODE)
msr cpsr_c, \rtemp @ switch to the SYS mode
ldr sp, [\rd, #\offset] @ load sp_usr
ldr lr, [\rd, #\offset + 4] @ load lr_usr
eor \rtemp, \rtemp, #(SVC_MODE ^ SYSTEM_MODE)
msr cpsr_c, \rtemp @ switch back to the SVC mode
.endm
#ifndef CONFIG_THUMB2_KERNEL
ARM: entry: move IRQ tracing exit into svc_exit The IRQ tracing exit path is much the same between all SVC mode exits, so move this into the svc_exit macro. Use a macro parameter to identify the IRQ case, which is the only different case there is. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-03-28 20:57:40 +08:00
.macro svc_exit, rpsr, irq = 0
.if \irq != 0
ARM: entry: move disable_irq_notrace into svc_exit All svc exit paths need IRQs off. Rather than placing this before every user of svc_exit, combine it into this macro. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-03-28 22:36:05 +08:00
@ IRQs already off
ARM: entry: move IRQ tracing exit into svc_exit The IRQ tracing exit path is much the same between all SVC mode exits, so move this into the svc_exit macro. Use a macro parameter to identify the IRQ case, which is the only different case there is. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-03-28 20:57:40 +08:00
#ifdef CONFIG_TRACE_IRQFLAGS
@ The parent context IRQs must have been enabled to get here in
@ the first place, so there's no point checking the PSR I bit.
bl trace_hardirqs_on
#endif
.else
ARM: entry: move disable_irq_notrace into svc_exit All svc exit paths need IRQs off. Rather than placing this before every user of svc_exit, combine it into this macro. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-03-28 22:36:05 +08:00
@ IRQs off again before pulling preserved data off the stack
disable_irq_notrace
ARM: entry: move IRQ tracing exit into svc_exit The IRQ tracing exit path is much the same between all SVC mode exits, so move this into the svc_exit macro. Use a macro parameter to identify the IRQ case, which is the only different case there is. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-03-28 20:57:40 +08:00
#ifdef CONFIG_TRACE_IRQFLAGS
tst \rpsr, #PSR_I_BIT
bleq trace_hardirqs_on
tst \rpsr, #PSR_I_BIT
blne trace_hardirqs_off
#endif
.endif
Thumb-2: Implementation of the unified start-up and exceptions code This patch implements the ARM/Thumb-2 unified kernel start-up and exception handling code. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2009-07-24 19:32:54 +08:00
msr spsr_cxsf, \rpsr
ARM: 8129/1: errata: work around Cortex-A15 erratum 830321 using dummy strex On revisions of Cortex-A15 prior to r3p3, a CLREX instruction at PL1 may falsely trigger a watchpoint exception, leading to potential data aborts during exception return and/or livelock. This patch resolves the issue in the following ways: - Replacing our uses of CLREX with a dummy STREX sequence instead (as we did for v6 CPUs). - Removing the clrex code from v7_exit_coherency_flush and derivatives, since this only exists as a minor performance improvement when non-cached exclusives are in use (Linux doesn't use these). Benchmarking on a variety of ARM cores revealed no measurable performance difference with this change applied, so the change is performed unconditionally and no new Kconfig entry is added. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com> Cc: stable@vger.kernel.org Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2014-08-15 19:11:50 +08:00
#if defined(CONFIG_CPU_V6) || defined(CONFIG_CPU_32v6K)
@ We must avoid clrex due to Cortex-A15 erratum #830321
sub r0, sp, #4 @ uninhabited address
strex r1, r2, [r0] @ clear the exclusive monitor
Clear the exclusive monitor when returning from an exception The patch adds a CLREX or dummy STREX to the exception return path. This is needed because several atomic/locking operations use a pair of LDREX/STREXEQ and the EQ condition may not always be satisfied. This would leave the exclusive monitor status set and may cause problems with atomic/locking operations in the interrupted code. With this patch, the atomic_set() operation can be a simple STR instruction (on SMP systems, the global exclusive monitor is cleared by STR anyway). Clearing the exclusive monitor during context switch is no longer needed as this is handled by the exception return path anyway. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Reported-by: Jamie Lokier <jamie@shareable.org>
2009-09-19 06:27:05 +08:00
#endif
ARM: 8129/1: errata: work around Cortex-A15 erratum 830321 using dummy strex On revisions of Cortex-A15 prior to r3p3, a CLREX instruction at PL1 may falsely trigger a watchpoint exception, leading to potential data aborts during exception return and/or livelock. This patch resolves the issue in the following ways: - Replacing our uses of CLREX with a dummy STREX sequence instead (as we did for v6 CPUs). - Removing the clrex code from v7_exit_coherency_flush and derivatives, since this only exists as a minor performance improvement when non-cached exclusives are in use (Linux doesn't use these). Benchmarking on a variety of ARM cores revealed no measurable performance difference with this change applied, so the change is performed unconditionally and no new Kconfig entry is added. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com> Cc: stable@vger.kernel.org Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2014-08-15 19:11:50 +08:00
ldmia sp, {r0 - pc}^ @ load r0 - pc, cpsr
Thumb-2: Implementation of the unified start-up and exceptions code This patch implements the ARM/Thumb-2 unified kernel start-up and exception handling code. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2009-07-24 19:32:54 +08:00
.endm
ARM: 8150/3: fiq: Replace default FIQ handler This patch introduces a new default FIQ handler that is structured in a similar way to the existing ARM exception handler and result in the FIQ being handled by C code running on the SVC stack (despite this code run in the FIQ handler is subject to severe limitations with respect to locking making normal interaction with the kernel impossible). This default handler allows concepts that on x86 would be handled using NMIs to be realized on ARM. Credit: This patch is a near complete re-write of a patch originally provided by Anton Vorontsov. Today only a couple of small fragments survive, however without Anton's work to build from this patch would not exist. Thanks also to Russell King for spoonfeeding me a variety of fixes during the review cycle. Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Nicolas Pitre <nico@linaro.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2014-09-18 00:12:06 +08:00
@
@ svc_exit_via_fiq - like svc_exit but switches to FIQ mode before exit
@
@ This macro acts in a similar manner to svc_exit but switches to FIQ
@ mode to restore the final part of the register state.
@
@ We cannot use the normal svc_exit procedure because that would
@ clobber spsr_svc (FIQ could be delivered during the first few
@ instructions of vector_swi meaning its contents have not been
@ saved anywhere).
@
@ Note that, unlike svc_exit, this macro also does not allow a caller
@ supplied rpsr. This is because the FIQ exceptions are not re-entrant
@ and the handlers cannot call into the scheduler (meaning the value
@ on the stack remains correct).
@
.macro svc_exit_via_fiq
mov r0, sp
ldmib r0, {r1 - r14} @ abort is deadly from here onward (it will
@ clobber state restored below)
msr cpsr_c, #FIQ_MODE | PSR_I_BIT | PSR_F_BIT
add r8, r0, #S_PC
ldr r9, [r0, #S_PSR]
msr spsr_cxsf, r9
ldr r0, [r0, #S_R0]
ldmia r8, {pc}^
.endm
Thumb-2: Implementation of the unified start-up and exceptions code This patch implements the ARM/Thumb-2 unified kernel start-up and exception handling code. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2009-07-24 19:32:54 +08:00
.macro restore_user_regs, fast = 0, offset = 0
ldr r1, [sp, #\offset + S_PSR] @ get calling cpsr
ldr lr, [sp, #\offset + S_PC]! @ get pc
msr spsr_cxsf, r1 @ save in spsr_svc
ARM: 8129/1: errata: work around Cortex-A15 erratum 830321 using dummy strex On revisions of Cortex-A15 prior to r3p3, a CLREX instruction at PL1 may falsely trigger a watchpoint exception, leading to potential data aborts during exception return and/or livelock. This patch resolves the issue in the following ways: - Replacing our uses of CLREX with a dummy STREX sequence instead (as we did for v6 CPUs). - Removing the clrex code from v7_exit_coherency_flush and derivatives, since this only exists as a minor performance improvement when non-cached exclusives are in use (Linux doesn't use these). Benchmarking on a variety of ARM cores revealed no measurable performance difference with this change applied, so the change is performed unconditionally and no new Kconfig entry is added. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com> Cc: stable@vger.kernel.org Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2014-08-15 19:11:50 +08:00
#if defined(CONFIG_CPU_V6) || defined(CONFIG_CPU_32v6K)
@ We must avoid clrex due to Cortex-A15 erratum #830321
Clear the exclusive monitor when returning from an exception The patch adds a CLREX or dummy STREX to the exception return path. This is needed because several atomic/locking operations use a pair of LDREX/STREXEQ and the EQ condition may not always be satisfied. This would leave the exclusive monitor status set and may cause problems with atomic/locking operations in the interrupted code. With this patch, the atomic_set() operation can be a simple STR instruction (on SMP systems, the global exclusive monitor is cleared by STR anyway). Clearing the exclusive monitor during context switch is no longer needed as this is handled by the exception return path anyway. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Reported-by: Jamie Lokier <jamie@shareable.org>
2009-09-19 06:27:05 +08:00
strex r1, r2, [sp] @ clear the exclusive monitor
#endif
Thumb-2: Implementation of the unified start-up and exceptions code This patch implements the ARM/Thumb-2 unified kernel start-up and exception handling code. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2009-07-24 19:32:54 +08:00
.if \fast
ldmdb sp, {r1 - lr}^ @ get calling r1 - lr
.else
ldmdb sp, {r0 - lr}^ @ get calling r0 - lr
.endif
ARM: 5991/1: Fix regression in restore_user_regs macro ARMv5T and earlier require that a ldm {}^ instruction is not followed by an instruction that accesses banked registers. This patch restores the nop that was lost in commit b86040a59feb255a8193173caa4d5199464433d5. Signed-off-by: Anders Grafström <grfstrm@users.sourceforge.net> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-03-15 23:04:14 +08:00
mov r0, r0 @ ARMv5T and earlier require a nop
@ after ldm {}^
Thumb-2: Implementation of the unified start-up and exceptions code This patch implements the ARM/Thumb-2 unified kernel start-up and exception handling code. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2009-07-24 19:32:54 +08:00
add sp, sp, #S_FRAME_SIZE - S_PC
movs pc, lr @ return & move spsr_svc into cpsr
.endm
#else /* CONFIG_THUMB2_KERNEL */
ARM: entry: move IRQ tracing exit into svc_exit The IRQ tracing exit path is much the same between all SVC mode exits, so move this into the svc_exit macro. Use a macro parameter to identify the IRQ case, which is the only different case there is. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-03-28 20:57:40 +08:00
.macro svc_exit, rpsr, irq = 0
.if \irq != 0
ARM: entry: move disable_irq_notrace into svc_exit All svc exit paths need IRQs off. Rather than placing this before every user of svc_exit, combine it into this macro. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-03-28 22:36:05 +08:00
@ IRQs already off
ARM: entry: move IRQ tracing exit into svc_exit The IRQ tracing exit path is much the same between all SVC mode exits, so move this into the svc_exit macro. Use a macro parameter to identify the IRQ case, which is the only different case there is. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-03-28 20:57:40 +08:00
#ifdef CONFIG_TRACE_IRQFLAGS
@ The parent context IRQs must have been enabled to get here in
@ the first place, so there's no point checking the PSR I bit.
bl trace_hardirqs_on
#endif
.else
ARM: entry: move disable_irq_notrace into svc_exit All svc exit paths need IRQs off. Rather than placing this before every user of svc_exit, combine it into this macro. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-03-28 22:36:05 +08:00
@ IRQs off again before pulling preserved data off the stack
disable_irq_notrace
ARM: entry: move IRQ tracing exit into svc_exit The IRQ tracing exit path is much the same between all SVC mode exits, so move this into the svc_exit macro. Use a macro parameter to identify the IRQ case, which is the only different case there is. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-03-28 20:57:40 +08:00
#ifdef CONFIG_TRACE_IRQFLAGS
tst \rpsr, #PSR_I_BIT
bleq trace_hardirqs_on
tst \rpsr, #PSR_I_BIT
blne trace_hardirqs_off
#endif
.endif
ARM: Thumb-2: Fix exception return sequence to restore stack correctly The implementation of svc_exit didn't take into account any stack hole created by svc_entry; as happens with the undef handler when kprobes are configured. The fix is to read the saved value of SP rather than trying to calculate it. Signed-off-by: Jon Medhurst <tixy@yxit.co.uk> Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
2011-03-19 01:32:44 +08:00
ldr lr, [sp, #S_SP] @ top of the stack
ldrd r0, r1, [sp, #S_LR] @ calling lr and pc
ARM: 8129/1: errata: work around Cortex-A15 erratum 830321 using dummy strex On revisions of Cortex-A15 prior to r3p3, a CLREX instruction at PL1 may falsely trigger a watchpoint exception, leading to potential data aborts during exception return and/or livelock. This patch resolves the issue in the following ways: - Replacing our uses of CLREX with a dummy STREX sequence instead (as we did for v6 CPUs). - Removing the clrex code from v7_exit_coherency_flush and derivatives, since this only exists as a minor performance improvement when non-cached exclusives are in use (Linux doesn't use these). Benchmarking on a variety of ARM cores revealed no measurable performance difference with this change applied, so the change is performed unconditionally and no new Kconfig entry is added. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com> Cc: stable@vger.kernel.org Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2014-08-15 19:11:50 +08:00
@ We must avoid clrex due to Cortex-A15 erratum #830321
strex r2, r1, [sp, #S_LR] @ clear the exclusive monitor
ARM: Thumb-2: Fix exception return sequence to restore stack correctly The implementation of svc_exit didn't take into account any stack hole created by svc_entry; as happens with the undef handler when kprobes are configured. The fix is to read the saved value of SP rather than trying to calculate it. Signed-off-by: Jon Medhurst <tixy@yxit.co.uk> Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
2011-03-19 01:32:44 +08:00
stmdb lr!, {r0, r1, \rpsr} @ calling lr and rfe context
Thumb-2: Implementation of the unified start-up and exceptions code This patch implements the ARM/Thumb-2 unified kernel start-up and exception handling code. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2009-07-24 19:32:54 +08:00
ldmia sp, {r0 - r12}
ARM: Thumb-2: Fix exception return sequence to restore stack correctly The implementation of svc_exit didn't take into account any stack hole created by svc_entry; as happens with the undef handler when kprobes are configured. The fix is to read the saved value of SP rather than trying to calculate it. Signed-off-by: Jon Medhurst <tixy@yxit.co.uk> Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
2011-03-19 01:32:44 +08:00
mov sp, lr
ldr lr, [sp], #4
Thumb-2: Implementation of the unified start-up and exceptions code This patch implements the ARM/Thumb-2 unified kernel start-up and exception handling code. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2009-07-24 19:32:54 +08:00
rfeia sp!
.endm
ARM: 8150/3: fiq: Replace default FIQ handler This patch introduces a new default FIQ handler that is structured in a similar way to the existing ARM exception handler and result in the FIQ being handled by C code running on the SVC stack (despite this code run in the FIQ handler is subject to severe limitations with respect to locking making normal interaction with the kernel impossible). This default handler allows concepts that on x86 would be handled using NMIs to be realized on ARM. Credit: This patch is a near complete re-write of a patch originally provided by Anton Vorontsov. Today only a couple of small fragments survive, however without Anton's work to build from this patch would not exist. Thanks also to Russell King for spoonfeeding me a variety of fixes during the review cycle. Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Nicolas Pitre <nico@linaro.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2014-09-18 00:12:06 +08:00
@
@ svc_exit_via_fiq - like svc_exit but switches to FIQ mode before exit
@
@ For full details see non-Thumb implementation above.
@
.macro svc_exit_via_fiq
add r0, sp, #S_R2
ldr lr, [sp, #S_LR]
ldr sp, [sp, #S_SP] @ abort is deadly from here onward (it will
@ clobber state restored below)
ldmia r0, {r2 - r12}
mov r1, #FIQ_MODE | PSR_I_BIT | PSR_F_BIT
msr cpsr_c, r1
sub r0, #S_R2
add r8, r0, #S_PC
ldmia r0, {r0 - r1}
rfeia r8
.endm
ARM: ARMv7-M: Add support for exception handling This patch implements the exception handling for the ARMv7-M architecture (pretty different from the A or R profiles). It bases on work done earlier by Catalin for 2.6.33 but was nearly completely rewritten to use a pt_regs layout compatible to the A profile. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Reviewed-by: Jonathan Austin <jonathan.austin@arm.com> Tested-by: Jonathan Austin <jonathan.austin@arm.com> Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
2010-05-22 01:06:42 +08:00
#ifdef CONFIG_CPU_V7M
/*
* Note we don't need to do clrex here as clearing the local monitor is
* part of each exception entry and exit sequence.
*/
.macro restore_user_regs, fast = 0, offset = 0
.if \offset
add sp, #\offset
.endif
v7m_exception_slow_exit ret_r0 = \fast
.endm
#else /* ifdef CONFIG_CPU_V7M */
Thumb-2: Implementation of the unified start-up and exceptions code This patch implements the ARM/Thumb-2 unified kernel start-up and exception handling code. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2009-07-24 19:32:54 +08:00
.macro restore_user_regs, fast = 0, offset = 0
mov r2, sp
load_user_sp_lr r2, r3, \offset + S_SP @ calling sp, lr
ldr r1, [sp, #\offset + S_PSR] @ get calling cpsr
ldr lr, [sp, #\offset + S_PC] @ get pc
add sp, sp, #\offset + S_SP
msr spsr_cxsf, r1 @ save in spsr_svc
ARM: 8129/1: errata: work around Cortex-A15 erratum 830321 using dummy strex On revisions of Cortex-A15 prior to r3p3, a CLREX instruction at PL1 may falsely trigger a watchpoint exception, leading to potential data aborts during exception return and/or livelock. This patch resolves the issue in the following ways: - Replacing our uses of CLREX with a dummy STREX sequence instead (as we did for v6 CPUs). - Removing the clrex code from v7_exit_coherency_flush and derivatives, since this only exists as a minor performance improvement when non-cached exclusives are in use (Linux doesn't use these). Benchmarking on a variety of ARM cores revealed no measurable performance difference with this change applied, so the change is performed unconditionally and no new Kconfig entry is added. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com> Cc: stable@vger.kernel.org Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2014-08-15 19:11:50 +08:00
@ We must avoid clrex due to Cortex-A15 erratum #830321
strex r1, r2, [sp] @ clear the exclusive monitor
Thumb-2: Implementation of the unified start-up and exceptions code This patch implements the ARM/Thumb-2 unified kernel start-up and exception handling code. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2009-07-24 19:32:54 +08:00
.if \fast
ldmdb sp, {r1 - r12} @ get calling r1 - r12
.else
ldmdb sp, {r0 - r12} @ get calling r0 - r12
.endif
add sp, sp, #S_FRAME_SIZE - S_SP
movs pc, lr @ return & move spsr_svc into cpsr
.endm
ARM: ARMv7-M: Add support for exception handling This patch implements the exception handling for the ARMv7-M architecture (pretty different from the A or R profiles). It bases on work done earlier by Catalin for 2.6.33 but was nearly completely rewritten to use a pt_regs layout compatible to the A profile. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Reviewed-by: Jonathan Austin <jonathan.austin@arm.com> Tested-by: Jonathan Austin <jonathan.austin@arm.com> Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
2010-05-22 01:06:42 +08:00
#endif /* ifdef CONFIG_CPU_V7M / else */
Thumb-2: Implementation of the unified start-up and exceptions code This patch implements the ARM/Thumb-2 unified kernel start-up and exception handling code. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2009-07-24 19:32:54 +08:00
#endif /* !CONFIG_THUMB2_KERNEL */
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
ARM: 7688/1: add support for context tracking subsystem commit 91d1aa43 (context_tracking: New context tracking susbsystem) generalized parts of the RCU userspace extended quiescent state into the context tracking subsystem. Context tracking is then used to implement adaptive tickless (a.k.a extended nohz) To support the new context tracking subsystem on ARM, the user/kernel boundary transtions need to be instrumented. For exceptions and IRQs in usermode, the existing usr_entry macro is used to instrument the user->kernel transition. For the return to usermode path, the ret_to_user* path is instrumented. Using the usr_entry macro, this covers interrupts in userspace, data abort and prefetch abort exceptions in userspace as well as undefined exceptions in userspace (which is where FP emulation and VFP are handled.) For syscalls, the slow return path is covered by instrumenting the ret_to_user path. In addition, the syscall entry point is instrumented which covers the user->kernel transition for both fast and slow syscalls, and an additional instrumentation point is added for the fast syscall return path (ret_fast_syscall). Cc: Mats Liljegren <mats.liljegren@enea.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Signed-off-by: Kevin Hilman <khilman@linaro.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-03-29 05:54:40 +08:00
/*
* Context tracking subsystem. Used to instrument transitions
* between user and kernel mode.
*/
.macro ct_user_exit, save = 1
#ifdef CONFIG_CONTEXT_TRACKING
.if \save
stmdb sp!, {r0-r3, ip, lr}
arm: Fix build error with context tracking calls ad65782fba50 (context_tracking: Optimize main APIs off case with static key) converted context tracking main APIs to inline function and left ARM asm callers behind. This can be easily fixed by making ARM calling the post static keys context tracking function. We just need to replicate the static key checks there. We'll remove these later when ARM will support the context tracking static keys. Reported-by: Guenter Roeck <linux@roeck-us.net> Reported-by: Russell King <linux@arm.linux.org.uk> Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com> Tested-by: Kevin Hilman <khilman@linaro.org> Cc: Nicolas Pitre <nicolas.pitre@linaro.org> Cc: Anil Kumar <anilk4.v@gmail.com> Cc: Tony Lindgren <tony@atomide.com> Cc: Benoit Cousson <b-cousson@ti.com> Cc: Guenter Roeck <linux@roeck-us.net> Cc: Russell King <linux@arm.linux.org.uk> Cc: Kevin Hilman <khilman@linaro.org>
2013-09-10 06:54:17 +08:00
bl context_tracking_user_exit
ARM: 7688/1: add support for context tracking subsystem commit 91d1aa43 (context_tracking: New context tracking susbsystem) generalized parts of the RCU userspace extended quiescent state into the context tracking subsystem. Context tracking is then used to implement adaptive tickless (a.k.a extended nohz) To support the new context tracking subsystem on ARM, the user/kernel boundary transtions need to be instrumented. For exceptions and IRQs in usermode, the existing usr_entry macro is used to instrument the user->kernel transition. For the return to usermode path, the ret_to_user* path is instrumented. Using the usr_entry macro, this covers interrupts in userspace, data abort and prefetch abort exceptions in userspace as well as undefined exceptions in userspace (which is where FP emulation and VFP are handled.) For syscalls, the slow return path is covered by instrumenting the ret_to_user path. In addition, the syscall entry point is instrumented which covers the user->kernel transition for both fast and slow syscalls, and an additional instrumentation point is added for the fast syscall return path (ret_fast_syscall). Cc: Mats Liljegren <mats.liljegren@enea.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Signed-off-by: Kevin Hilman <khilman@linaro.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-03-29 05:54:40 +08:00
ldmia sp!, {r0-r3, ip, lr}
.else
arm: Fix build error with context tracking calls ad65782fba50 (context_tracking: Optimize main APIs off case with static key) converted context tracking main APIs to inline function and left ARM asm callers behind. This can be easily fixed by making ARM calling the post static keys context tracking function. We just need to replicate the static key checks there. We'll remove these later when ARM will support the context tracking static keys. Reported-by: Guenter Roeck <linux@roeck-us.net> Reported-by: Russell King <linux@arm.linux.org.uk> Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com> Tested-by: Kevin Hilman <khilman@linaro.org> Cc: Nicolas Pitre <nicolas.pitre@linaro.org> Cc: Anil Kumar <anilk4.v@gmail.com> Cc: Tony Lindgren <tony@atomide.com> Cc: Benoit Cousson <b-cousson@ti.com> Cc: Guenter Roeck <linux@roeck-us.net> Cc: Russell King <linux@arm.linux.org.uk> Cc: Kevin Hilman <khilman@linaro.org>
2013-09-10 06:54:17 +08:00
bl context_tracking_user_exit
ARM: 7688/1: add support for context tracking subsystem commit 91d1aa43 (context_tracking: New context tracking susbsystem) generalized parts of the RCU userspace extended quiescent state into the context tracking subsystem. Context tracking is then used to implement adaptive tickless (a.k.a extended nohz) To support the new context tracking subsystem on ARM, the user/kernel boundary transtions need to be instrumented. For exceptions and IRQs in usermode, the existing usr_entry macro is used to instrument the user->kernel transition. For the return to usermode path, the ret_to_user* path is instrumented. Using the usr_entry macro, this covers interrupts in userspace, data abort and prefetch abort exceptions in userspace as well as undefined exceptions in userspace (which is where FP emulation and VFP are handled.) For syscalls, the slow return path is covered by instrumenting the ret_to_user path. In addition, the syscall entry point is instrumented which covers the user->kernel transition for both fast and slow syscalls, and an additional instrumentation point is added for the fast syscall return path (ret_fast_syscall). Cc: Mats Liljegren <mats.liljegren@enea.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Signed-off-by: Kevin Hilman <khilman@linaro.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-03-29 05:54:40 +08:00
.endif
#endif
.endm
.macro ct_user_enter, save = 1
#ifdef CONFIG_CONTEXT_TRACKING
.if \save
stmdb sp!, {r0-r3, ip, lr}
arm: Fix build error with context tracking calls ad65782fba50 (context_tracking: Optimize main APIs off case with static key) converted context tracking main APIs to inline function and left ARM asm callers behind. This can be easily fixed by making ARM calling the post static keys context tracking function. We just need to replicate the static key checks there. We'll remove these later when ARM will support the context tracking static keys. Reported-by: Guenter Roeck <linux@roeck-us.net> Reported-by: Russell King <linux@arm.linux.org.uk> Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com> Tested-by: Kevin Hilman <khilman@linaro.org> Cc: Nicolas Pitre <nicolas.pitre@linaro.org> Cc: Anil Kumar <anilk4.v@gmail.com> Cc: Tony Lindgren <tony@atomide.com> Cc: Benoit Cousson <b-cousson@ti.com> Cc: Guenter Roeck <linux@roeck-us.net> Cc: Russell King <linux@arm.linux.org.uk> Cc: Kevin Hilman <khilman@linaro.org>
2013-09-10 06:54:17 +08:00
bl context_tracking_user_enter
ARM: 7688/1: add support for context tracking subsystem commit 91d1aa43 (context_tracking: New context tracking susbsystem) generalized parts of the RCU userspace extended quiescent state into the context tracking subsystem. Context tracking is then used to implement adaptive tickless (a.k.a extended nohz) To support the new context tracking subsystem on ARM, the user/kernel boundary transtions need to be instrumented. For exceptions and IRQs in usermode, the existing usr_entry macro is used to instrument the user->kernel transition. For the return to usermode path, the ret_to_user* path is instrumented. Using the usr_entry macro, this covers interrupts in userspace, data abort and prefetch abort exceptions in userspace as well as undefined exceptions in userspace (which is where FP emulation and VFP are handled.) For syscalls, the slow return path is covered by instrumenting the ret_to_user path. In addition, the syscall entry point is instrumented which covers the user->kernel transition for both fast and slow syscalls, and an additional instrumentation point is added for the fast syscall return path (ret_fast_syscall). Cc: Mats Liljegren <mats.liljegren@enea.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Signed-off-by: Kevin Hilman <khilman@linaro.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-03-29 05:54:40 +08:00
ldmia sp!, {r0-r3, ip, lr}
.else
arm: Fix build error with context tracking calls ad65782fba50 (context_tracking: Optimize main APIs off case with static key) converted context tracking main APIs to inline function and left ARM asm callers behind. This can be easily fixed by making ARM calling the post static keys context tracking function. We just need to replicate the static key checks there. We'll remove these later when ARM will support the context tracking static keys. Reported-by: Guenter Roeck <linux@roeck-us.net> Reported-by: Russell King <linux@arm.linux.org.uk> Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com> Tested-by: Kevin Hilman <khilman@linaro.org> Cc: Nicolas Pitre <nicolas.pitre@linaro.org> Cc: Anil Kumar <anilk4.v@gmail.com> Cc: Tony Lindgren <tony@atomide.com> Cc: Benoit Cousson <b-cousson@ti.com> Cc: Guenter Roeck <linux@roeck-us.net> Cc: Russell King <linux@arm.linux.org.uk> Cc: Kevin Hilman <khilman@linaro.org>
2013-09-10 06:54:17 +08:00
bl context_tracking_user_enter
ARM: 7688/1: add support for context tracking subsystem commit 91d1aa43 (context_tracking: New context tracking susbsystem) generalized parts of the RCU userspace extended quiescent state into the context tracking subsystem. Context tracking is then used to implement adaptive tickless (a.k.a extended nohz) To support the new context tracking subsystem on ARM, the user/kernel boundary transtions need to be instrumented. For exceptions and IRQs in usermode, the existing usr_entry macro is used to instrument the user->kernel transition. For the return to usermode path, the ret_to_user* path is instrumented. Using the usr_entry macro, this covers interrupts in userspace, data abort and prefetch abort exceptions in userspace as well as undefined exceptions in userspace (which is where FP emulation and VFP are handled.) For syscalls, the slow return path is covered by instrumenting the ret_to_user path. In addition, the syscall entry point is instrumented which covers the user->kernel transition for both fast and slow syscalls, and an additional instrumentation point is added for the fast syscall return path (ret_fast_syscall). Cc: Mats Liljegren <mats.liljegren@enea.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Signed-off-by: Kevin Hilman <khilman@linaro.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-03-29 05:54:40 +08:00
.endif
#endif
.endm
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
/*
* These are the registers used in the syscall handler, and allow us to
* have in theory up to 7 arguments to a function - r0 to r6.
*
* r7 is reserved for the system call number for thumb mode.
*
* Note that tbl == why is intentional.
*
* We must set at least "tsk" and "why" when calling ret_with_reschedule.
*/
scno .req r7 @ syscall number
tbl .req r8 @ syscall table pointer
why .req r8 @ Linux syscall (!= 0)
tsk .req r9 @ current thread_info