openkylin
/
linux
mirror of https://gitee.com/openkylin/linux.git
9
0
Fork 0
Code Issues Projects Releases Wiki Activity
linux/arch/x86/kernel
History
Denys Vlasenko 29722cd4ef x86/asm/entry/64: Save R11 into pt_regs->flags on SYSCALL64 fastpath 
Before this patch, R11 was saved in pt_regs->r11.

Which looks natural, but requires messy shuffling to/from iret
frame whenever ptrace or e.g. sys_iopl() wants to modify flags -
because that's how this register is used by SYSCALL/SYSRET.

This patch saves R11 in pt_regs->flags, and uses that value for
the SYSRET64 instruction. Shuffling is eliminated.

FIXUP/RESTORE_TOP_OF_STACK are simplified.

stub_iopl is no longer needed: pt_regs->flags needs no fixing up.

Testing shows that syscall fast path is ~54.3 ns before
and after the patch (on 2.7 GHz Sandy Bridge CPU).

Signed-off-by: Denys Vlasenko <dvlasenk@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Will Drewry <wad@chromium.org>
Link: http://lkml.kernel.org/r/1425926364-9526-2-git-send-email-dvlasenk@redhat.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
 		2015-03-10 13:56:10 +01:00
..
acpi Merge branches 'core-urgent-for-linus' and 'irq-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2015-02-21 10:36:06 -08:00
apic Revert "x86/apic: Only disable CPU x2apic mode when necessary" 2015-02-13 10:26:18 -08:00
cpu x86/asm/entry: Replace this_cpu_sp0() with current_top_of_stack() and fix it on x86_32 2015-03-07 09:34:03 +01:00
kprobes Merge branch 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2015-03-01 11:56:13 -08:00
.gitignore
…
Makefile Linux 4.0-rc2 2015-03-04 06:35:43 +01:00
alternative.c x86/alternatives: Use optimized NOPs for padding 2015-02-23 13:44:12 +01:00
amd_gart_64.c x86: enable DMA CMA with swiotlb 2014-06-04 16:53:57 -07:00
amd_nb.c x86, amd_nb: Add device IDs to NB tables for F15h M60h 2014-10-20 14:18:45 +02:00
apb_timer.c x86/platform: Remove unused function from apb_timer.c 2014-12-23 10:43:35 +01:00
aperture_64.c x86/gart: Tidy messages and add bridge device info 2014-05-23 10:47:19 -06:00
apm_32.c cpuidle: Invert CPUIDLE_FLAG_TIME_VALID logic 2014-11-12 21:17:27 +01:00
asm-offsets.c sched, x86: Provide a per-cpu preempt_count implementation 2013-09-25 14:07:57 +02:00
asm-offsets_32.c x86/asm: Guard against building the 32/64-bit versions of the asm-offsets*.c file directly 2014-12-11 11:43:56 +01:00
asm-offsets_64.c x86/asm/entry/64/compat: Change the 32-bit sysenter code to use sp0 2015-03-06 08:32:58 +01:00
audit_64.c x86: hook up execveat system call 2014-12-13 12:42:51 -08:00
bootflag.c
…
check.c x86/mm: memblock: switch to use NUMA_NO_NODE 2014-01-21 16:19:47 -08:00
cpuid.c x86, cpuid: Use PTR_ERR_OR_ZERO 2014-10-17 13:40:51 -07:00
crash.c x86, irq: Move IOAPIC related declarations from hw_irq.h into io_apic.h 2014-12-16 14:08:17 +01:00
crash_dump_32.c
…
crash_dump_64.c
…
devicetree.c x86, irq, devicetree: Release IOAPIC pin when PCI device is disabled 2014-06-21 23:05:44 +02:00
doublefault.c x86: Delete non-required instances of include <linux/init.h> 2014-01-06 21:25:18 -08:00
dumpstack.c x86_64: add KASan support 2015-02-13 21:21:41 -08:00
dumpstack_32.c Merge branch 'x86-threadinfo-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2014-04-01 10:17:18 -07:00
dumpstack_64.c x86_64, traps: Stop using IST for #SS 2014-11-23 13:56:19 -08:00
e820.c x86, e820: Clean up sanitize_e820_map() users 2015-01-23 16:14:27 +01:00
early-quirks.c drm/i915/skl: Add the additional graphics stolen sizes 2014-09-24 14:47:39 +02:00
early_printk.c Specify PCI based UART for earlyprintk 2015-02-02 10:11:27 -08:00
entry_32.S x86/asm: Introduce push/pop macros which generate CFI_REL_OFFSET and CFI_RESTORE 2015-03-04 22:50:49 +01:00
entry_64.S x86/asm/entry/64: Save R11 into pt_regs->flags on SYSCALL64 fastpath 2015-03-10 13:56:10 +01:00
espfix_64.c x86, espfix: Remove stale ptemask 2014-11-11 17:57:46 +01:00
ftrace.c module: remove mod arg from module_free, rename module_memfree(). 2015-01-20 11:38:33 +10:30
head.c
…
head32.c x86: Store a per-cpu shadow copy of CR4 2015-02-04 12:10:42 +01:00
head64.c Merge branch 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2015-02-16 14:58:12 -08:00
head_32.S x86/asm/boot: Use already defined KEEP_SEGMENTS macro in head_{32,64}.S 2015-02-19 10:05:04 +01:00
head_64.S x86/asm: Optimize unnecessarily wide TEST instructions 2015-03-07 11:12:43 +01:00
hpet.c kernel.h: remove ancient __FUNCTION__ hack 2015-02-12 18:54:13 -08:00
hw_breakpoint.c perf/x86: Remove get_hbp_len and replace with bp_len 2014-12-03 15:14:30 +01:00
i386_ksyms_32.c sched, x86: Optimize the preempt_schedule() call 2013-09-25 14:23:07 +02:00
i387.c Merge branch 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2015-02-16 14:58:12 -08:00
i8237.c
…
i8253.c
…
i8259.c x86/irq: Fix XT-PIC-XT-PIC in /proc/interrupts 2014-10-28 12:01:08 +01:00
io_delay.c
…
ioport.c x86/asm/entry: Rename 'init_tss' to 'cpu_tss' 2015-03-06 08:32:58 +01:00
iosf_mbi.c x86/platform/intel/iosf: Add debugfs config option for IOSF 2014-09-19 13:08:43 +02:00
irq.c x86/irq: Check for valid irq descriptor in check_irq_vectors_for_cpu_disable() 2015-02-18 15:01:42 +01:00
irq_32.c x86: Clean up current_stack_pointer 2015-01-02 10:22:46 -08:00
irq_64.c x86: Replace __get_cpu_var uses 2014-08-26 13:45:49 -04:00
irq_work.c x86: Tell irq work about self IPI support 2014-09-13 18:38:29 +02:00
irqinit.c x86, irq: Move local APIC related code from io_apic.c into vector.c 2014-12-16 14:08:16 +01:00
jump_label.c x86/jump_label: expect default_nop if static_key gets enabled on boot-up 2013-10-19 19:45:35 -04:00
kdebugfs.c
…
kexec-bzimage64.c kexec-bzimage64: fix sparse warnings 2014-10-14 02:18:21 +02:00
kgdb.c x86: Delete non-required instances of include <linux/init.h> 2014-01-06 21:25:18 -08:00
ksysfs.c x86: ksysfs.c build fix 2014-01-03 14:37:13 +00:00
kvm.c x86/spinlocks/paravirt: Fix memory corruption on unlock 2015-02-18 14:53:49 +01:00
kvmclock.c x86, kvm: Clear paravirt_enabled on KVM guests for espfix32's benefit 2014-12-10 12:49:39 +01:00
ldt.c Revert "x86-64, modify_ldt: Make support for 16-bit segments a runtime option" 2014-05-21 10:22:59 -07:00
livepatch.c livepatch: kernel: add support for live patching 2014-12-22 15:40:49 +01:00
machine_kexec_32.c x86, irq: Move IOAPIC related declarations from hw_irq.h into io_apic.h 2014-12-16 14:08:17 +01:00
machine_kexec_64.c x86, irq: Move IOAPIC related declarations from hw_irq.h into io_apic.h 2014-12-16 14:08:17 +01:00
mcount_64.S ftrace/fgraph/x86: Have prepare_ftrace_return() take ip as first parameter 2014-12-01 14:08:58 -05:00
mmconf-fam10h_64.c x86: delete __cpuinit usage from all x86 files 2013-07-14 19:36:56 -04:00
module.c Merge branch 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2015-02-21 10:41:29 -08:00
mpparse.c x86, apic: Remove mps_oem_check callback 2014-07-31 08:05:42 -07:00
msr.c x86, msr: Use seek definitions instead of hard-coded values 2014-10-17 13:40:55 -07:00
nmi.c kprobes, x86: Use NOKPROBE_SYMBOL() instead of __kprobes annotation 2014-04-24 10:26:38 +02:00
nmi_selftest.c
…
paravirt-spinlocks.c x86, ticketlock: Add slowpath logic 2013-08-09 07:54:00 -07:00
paravirt.c kprobes, x86: Prohibit probing on native_set_debugreg()/load_idt() 2014-04-24 10:02:58 +02:00
paravirt_patch_32.c
…
paravirt_patch_64.c x86_64/entry/xen: Do not invoke espfix64 on Xen 2014-07-28 15:25:40 -07:00
pci-calgary_64.c x86, calgary: Use 8M TCE table size by default 2014-04-10 19:51:32 -07:00
pci-dma.c arch/x86/kernel/pci-dma.c: fix dma_generic_alloc_coherent() when CONFIG_DMA_CMA is enabled 2014-06-04 16:53:57 -07:00
pci-iommu_table.c
…
pci-nommu.c x86: Delete non-required instances of include <linux/init.h> 2014-01-06 21:25:18 -08:00
pci-swiotlb.c x86: enable DMA CMA with swiotlb 2014-06-04 16:53:57 -07:00
pcspeaker.c
…
perf_regs.c perf/x86_64: Improve user regs sampling 2015-01-09 11:12:29 +01:00
pmc_atom.c x86: pmc_atom: Expose contents of PSS 2015-01-20 12:50:14 +01:00
probe_roms.c
…
process.c x86/asm/entry: Remove INIT_TSS and fold the definitions into 'cpu_tss' 2015-03-06 08:32:58 +01:00
process_32.c x86/asm/entry: Replace this_cpu_sp0() with current_top_of_stack() and fix it on x86_32 2015-03-07 09:34:03 +01:00
process_64.c x86/asm/entry: Delay loading sp0 slightly on task switch 2015-03-07 09:34:03 +01:00
ptrace.c x86/ptrace: Remove checks for TIF_IA32 when changing CS and SS 2015-02-25 08:27:49 +01:00
pvclock.c hung_task: add method to reset detector 2013-11-06 09:49:02 +02:00
quirks.c x86: HPET force enable for e6xx based systems 2014-09-15 17:53:35 -07:00
reboot.c x86, irq: Move IOAPIC related declarations from hw_irq.h into io_apic.h 2014-12-16 14:08:17 +01:00
reboot_fixups_32.c
…
relocate_kernel_32.S x86/asm: Optimize unnecessarily wide TEST instructions 2015-03-07 11:12:43 +01:00
relocate_kernel_64.S x86/asm: Optimize unnecessarily wide TEST instructions 2015-03-07 11:12:43 +01:00
resource.c x86: don't exclude low BIOS area when allocating address space for non-PCI cards 2014-07-16 12:29:36 -06:00
rtc.c kernel.h: remove ancient __FUNCTION__ hack 2015-02-12 18:54:13 -08:00
setup.c Merge branch 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2015-02-21 10:41:29 -08:00
setup_percpu.c x86: Convert a few more per-CPU items to read-mostly ones 2014-11-04 20:13:28 +01:00
signal.c all arches, signal: move restart_block to struct task_struct 2015-02-12 18:54:12 -08:00
smp.c asmlinkage, x86: Add explicit __visible to arch/x86/* 2014-05-05 16:07:44 -07:00
smpboot.c x86/asm/entry: Replace this_cpu_sp0() with current_top_of_stack() and fix it on x86_32 2015-03-07 09:34:03 +01:00
stacktrace.c
…
step.c
…
sys_x86_64.c x86 get_unmapped_area: Access mmap_legacy_base through mm_struct member 2013-08-22 10:19:35 -07:00
syscall_32.c x86/compat: Merge native and compat 32-bit syscall tables 2015-03-04 06:16:21 +01:00
syscall_64.c x86, asmlinkage: Make syscall tables visible 2013-08-06 14:20:18 -07:00
sysfb.c x86/sysfb: Use PTR_ERR_OR_ZERO 2014-10-17 13:40:52 -07:00
sysfb_efi.c x86: sysfb: move EFI quirks from efifb to sysfb 2013-08-02 16:17:47 -07:00
sysfb_simplefb.c x86/simplefb: Use PTR_ERR_OR_ZERO 2014-10-17 13:40:52 -07:00
tboot.c x86 / tboot / ACPI: Fail extended mode reduced hardware sleep 2013-07-31 14:25:51 +02:00
tce_64.c
…
test_nx.c
…
test_rodata.c
…
time.c x86_64/vdso: Remove jiffies from the vvar page 2014-10-28 11:22:13 +01:00
tls.c x86, tls: Interpret an all-zero struct user_desc as "no segment" 2015-01-22 21:45:07 +01:00
tls.h
…
topology.c hotplug, powerpc, x86: Remove cpu_hotplug_driver_lock() 2013-09-30 19:55:51 +02:00
trace_clock.c
…
tracepoint.c x86: Make sure IDT is page aligned 2013-07-16 15:14:48 -07:00
traps.c x86/asm/entry: Replace this_cpu_sp0() with current_top_of_stack() and fix it on x86_32 2015-03-07 09:34:03 +01:00
tsc.c x86/tsc: Change Fast TSC calibration failed from error to info 2015-01-23 10:53:52 +01:00
tsc_msr.c x86: tsc: Add missing Baytrail frequency to the table 2014-02-19 17:12:24 +01:00
tsc_sync.c x86: Delete non-required instances of include <linux/init.h> 2014-01-06 21:25:18 -08:00
uprobes.c uprobes/x86: Fix 2-byte opcode table 2015-02-18 20:55:53 +01:00
verify_cpu.S
…
vm86_32.c x86/asm/entry: Rename 'init_tss' to 'cpu_tss' 2015-03-06 08:32:58 +01:00
vmlinux.lds.S x86-64: Use RIP-relative addressing for most per-CPU accesses 2014-11-04 20:43:14 +01:00
vsmp_64.c x86/apic/vsmp: Make is_vsmp_box() static 2014-08-01 15:09:45 -07:00
vsyscall_64.c x86_64/vsyscall: Restore orig_ax after vsyscall seccomp 2014-11-10 10:46:35 +01:00
vsyscall_emu_64.S
…
vsyscall_gtod.c timekeeping: Create struct tk_read_base and use it in struct timekeeper 2014-07-23 15:01:53 -07:00
vsyscall_trace.h
…
x86_init.c Revert "PCI: Add x86_msi.msi_mask_irq() and msix_mask_irq()" 2014-11-11 15:14:30 -07:00
x8664_ksyms_64.c x86_64: kasan: add interceptors for memset/memmove/memcpy functions 2015-02-13 21:21:41 -08:00
xsave.c x86: Clean up cr4 manipulation 2015-02-04 12:10:41 +01:00