openkylin
/
linux
mirror of https://gitee.com/openkylin/linux.git
9
0
Fork 0
Code Issues Projects Releases Wiki Activity
linux/arch/powerpc/kernel/kvm_emul.S

355 lines
8.0 KiB
ArmAsm
Raw Normal View History

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 266 Based on 1 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license version 2 as published by the free software foundation this program is distributed in the hope that it will be useful but without any warranty without even the implied warranty of merchantability or fitness for a particular purpose see the gnu general public license for more details you should have received a copy of the gnu general public license along with this program if not write to the free software foundation 51 franklin street fifth floor boston ma 02110 1301 usa extracted by the scancode license scanner the SPDX license identifier GPL-2.0-only has been chosen to replace the boilerplate/reference in 67 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Allison Randal <allison@lohutok.net> Reviewed-by: Richard Fontana <rfontana@redhat.com> Reviewed-by: Alexios Zavras <alexios.zavras@intel.com> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190529141333.953658117@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-05-29 22:12:40 +08:00
/* SPDX-License-Identifier: GPL-2.0-only */
KVM: PPC: Generic KVM PV guest support We have all the hypervisor pieces in place now, but the guest parts are still missing. This patch implements basic awareness of KVM when running Linux as guest. It doesn't do anything with it yet though. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:47:57 +08:00
/*
*
* Copyright SUSE Linux Products GmbH 2010
KVM: PPC: booke: Paravirtualize wrtee Also fix wrteei 1 paravirt to check for a pending interrupt. Signed-off-by: Scott Wood <scottwood@freescale.com> Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2011-11-09 08:23:28 +08:00
* Copyright 2010-2011 Freescale Semiconductor, Inc.
KVM: PPC: Generic KVM PV guest support We have all the hypervisor pieces in place now, but the guest parts are still missing. This patch implements basic awareness of KVM when running Linux as guest. It doesn't do anything with it yet though. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:47:57 +08:00
*
* Authors: Alexander Graf <agraf@suse.de>
*/
#include <asm/ppc_asm.h>
#include <asm/kvm_asm.h>
#include <asm/reg.h>
#include <asm/page.h>
#include <asm/asm-offsets.h>
powerpc: move ASM_CONST and stringify_in_c() into asm-const.h This patch moves ASM_CONST() and stringify_in_c() into dedicated asm-const.h, then cleans all related inclusions. Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr> [mpe: asm-compat.h should include asm-const.h] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-07-06 00:24:57 +08:00
#include <asm/asm-compat.h>
KVM: PPC: Generic KVM PV guest support We have all the hypervisor pieces in place now, but the guest parts are still missing. This patch implements basic awareness of KVM when running Linux as guest. It doesn't do anything with it yet though. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:47:57 +08:00
#define KVM_MAGIC_PAGE (-4096)
KVM: PPC: PV assembler helpers When we hook an instruction we need to make sure we don't clobber any of the registers at that point. So we write them out to scratch space in the magic page. To make sure we don't fall into a race with another piece of hooked code, we need to disable interrupts. To make the later patches and code in general easier readable, let's introduce a set of defines that save and restore r30, r31 and cr. Let's also define some helpers to read the lower 32 bits of a 64 bit field on 32 bit systems. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:48:03 +08:00
#ifdef CONFIG_64BIT
#define LL64(reg, offs, reg2) ld reg, (offs)(reg2)
#define STL64(reg, offs, reg2) std reg, (offs)(reg2)
#else
#define LL64(reg, offs, reg2) lwz reg, (offs + 4)(reg2)
#define STL64(reg, offs, reg2) stw reg, (offs + 4)(reg2)
#endif
#define SCRATCH_SAVE \
/* Enable critical section. We are critical if \
shared->critical == r1 */ \
STL64(r1, KVM_MAGIC_PAGE + KVM_MAGIC_CRITICAL, 0); \
\
/* Save state */ \
PPC_STL r31, (KVM_MAGIC_PAGE + KVM_MAGIC_SCRATCH1)(0); \
PPC_STL r30, (KVM_MAGIC_PAGE + KVM_MAGIC_SCRATCH2)(0); \
mfcr r31; \
stw r31, (KVM_MAGIC_PAGE + KVM_MAGIC_SCRATCH3)(0);
#define SCRATCH_RESTORE \
/* Restore state */ \
PPC_LL r31, (KVM_MAGIC_PAGE + KVM_MAGIC_SCRATCH1)(0); \
lwz r30, (KVM_MAGIC_PAGE + KVM_MAGIC_SCRATCH3)(0); \
mtcr r30; \
PPC_LL r30, (KVM_MAGIC_PAGE + KVM_MAGIC_SCRATCH2)(0); \
\
/* Disable critical section. We are critical if \
shared->critical == r1 and r2 is always != r1 */ \
STL64(r2, KVM_MAGIC_PAGE + KVM_MAGIC_CRITICAL, 0);
KVM: PPC: PV mtmsrd L=1 The PowerPC ISA has a special instruction for mtmsr that only changes the EE and RI bits, namely the L=1 form. Since that one is reasonably often occuring and simple to implement, let's go with this first. Writing EE=0 is always just a store. Doing EE=1 also requires us to check for pending interrupts and if necessary exit back to the hypervisor. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:48:04 +08:00
KVM: PPC: Avoid patching paravirt template code Currently we patch the whole code include paravirt template code. This isn't safe for scratch area and has impact to performance. Signed-off-by: Liu Yu <yu.liu@freescale.com> Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2011-12-02 04:22:53 +08:00
.global kvm_template_start
kvm_template_start:
KVM: PPC: PV mtmsrd L=1 The PowerPC ISA has a special instruction for mtmsr that only changes the EE and RI bits, namely the L=1 form. Since that one is reasonably often occuring and simple to implement, let's go with this first. Writing EE=0 is always just a store. Doing EE=1 also requires us to check for pending interrupts and if necessary exit back to the hypervisor. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:48:04 +08:00
.global kvm_emulate_mtmsrd
kvm_emulate_mtmsrd:
SCRATCH_SAVE
/* Put MSR & ~(MSR_EE|MSR_RI) in r31 */
LL64(r31, KVM_MAGIC_PAGE + KVM_MAGIC_MSR, 0)
lis r30, (~(MSR_EE | MSR_RI))@h
ori r30, r30, (~(MSR_EE | MSR_RI))@l
and r31, r31, r30
/* OR the register's (MSR_EE|MSR_RI) on MSR */
kvm_emulate_mtmsrd_reg:
KVM: PPC: Make PV mtmsrd L=1 work with r30 and r31 We had an arbitrary limitation in mtmsrd L=1 that kept us from using r30 and r31 as input registers. Let's get rid of that and get more potential speedups! Signed-off-by: Alexander Graf <agraf@suse.de>
2010-08-05 21:44:41 +08:00
ori r30, r0, 0
andi. r30, r30, (MSR_EE|MSR_RI)
KVM: PPC: PV mtmsrd L=1 The PowerPC ISA has a special instruction for mtmsr that only changes the EE and RI bits, namely the L=1 form. Since that one is reasonably often occuring and simple to implement, let's go with this first. Writing EE=0 is always just a store. Doing EE=1 also requires us to check for pending interrupts and if necessary exit back to the hypervisor. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:48:04 +08:00
or r31, r31, r30
/* Put MSR back into magic page */
STL64(r31, KVM_MAGIC_PAGE + KVM_MAGIC_MSR, 0)
/* Check if we have to fetch an interrupt */
lwz r31, (KVM_MAGIC_PAGE + KVM_MAGIC_INT)(0)
cmpwi r31, 0
beq+ no_check
/* Check if we may trigger an interrupt */
andi. r30, r30, MSR_EE
beq no_check
SCRATCH_RESTORE
/* Nag hypervisor */
KVM: PPC: Make PV mtmsrd L=1 work with r30 and r31 We had an arbitrary limitation in mtmsrd L=1 that kept us from using r30 and r31 as input registers. Let's get rid of that and get more potential speedups! Signed-off-by: Alexander Graf <agraf@suse.de>
2010-08-05 21:44:41 +08:00
kvm_emulate_mtmsrd_orig_ins:
KVM: PPC: PV mtmsrd L=1 The PowerPC ISA has a special instruction for mtmsr that only changes the EE and RI bits, namely the L=1 form. Since that one is reasonably often occuring and simple to implement, let's go with this first. Writing EE=0 is always just a store. Doing EE=1 also requires us to check for pending interrupts and if necessary exit back to the hypervisor. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:48:04 +08:00
tlbsync
b kvm_emulate_mtmsrd_branch
no_check:
SCRATCH_RESTORE
/* Go back to caller */
kvm_emulate_mtmsrd_branch:
b .
kvm_emulate_mtmsrd_end:
.global kvm_emulate_mtmsrd_branch_offs
kvm_emulate_mtmsrd_branch_offs:
.long (kvm_emulate_mtmsrd_branch - kvm_emulate_mtmsrd) / 4
.global kvm_emulate_mtmsrd_reg_offs
kvm_emulate_mtmsrd_reg_offs:
.long (kvm_emulate_mtmsrd_reg - kvm_emulate_mtmsrd) / 4
KVM: PPC: Make PV mtmsrd L=1 work with r30 and r31 We had an arbitrary limitation in mtmsrd L=1 that kept us from using r30 and r31 as input registers. Let's get rid of that and get more potential speedups! Signed-off-by: Alexander Graf <agraf@suse.de>
2010-08-05 21:44:41 +08:00
.global kvm_emulate_mtmsrd_orig_ins_offs
kvm_emulate_mtmsrd_orig_ins_offs:
.long (kvm_emulate_mtmsrd_orig_ins - kvm_emulate_mtmsrd) / 4
KVM: PPC: PV mtmsrd L=1 The PowerPC ISA has a special instruction for mtmsr that only changes the EE and RI bits, namely the L=1 form. Since that one is reasonably often occuring and simple to implement, let's go with this first. Writing EE=0 is always just a store. Doing EE=1 also requires us to check for pending interrupts and if necessary exit back to the hypervisor. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:48:04 +08:00
.global kvm_emulate_mtmsrd_len
kvm_emulate_mtmsrd_len:
.long (kvm_emulate_mtmsrd_end - kvm_emulate_mtmsrd) / 4
KVM: PPC: PV mtmsrd L=0 and mtmsr There is also a form of mtmsr where all bits need to be addressed. While the PPC64 Linux kernel behaves resonably well here, on PPC32 we do not have an L=1 form. It does mtmsr even for simple things like only changing EE. So we need to hook into that one as well and check for a mask of bits that we deem safe to change from within guest context. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:48:05 +08:00
KVM: PPC: Not optimizing MSR_CE and MSR_ME with paravirt. If there is pending critical or machine check interrupt then guest would like to capture it when guest enable MSR.CE and MSR_ME respectively. Also as mostly MSR_CE and MSR_ME are updated with rfi/rfci/rfmii which anyway traps so removing the the paravirt optimization for MSR.CE and MSR.ME. Signed-off-by: Bharat Bhushan <bharat.bhushan@freescale.com> Signed-off-by: Alexander Graf <agraf@suse.de>
2012-05-21 07:21:53 +08:00
#define MSR_SAFE_BITS (MSR_EE | MSR_RI)
KVM: PPC: PV mtmsrd L=0 and mtmsr There is also a form of mtmsr where all bits need to be addressed. While the PPC64 Linux kernel behaves resonably well here, on PPC32 we do not have an L=1 form. It does mtmsr even for simple things like only changing EE. So we need to hook into that one as well and check for a mask of bits that we deem safe to change from within guest context. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:48:05 +08:00
#define MSR_CRITICAL_BITS ~MSR_SAFE_BITS
.global kvm_emulate_mtmsr
kvm_emulate_mtmsr:
SCRATCH_SAVE
/* Fetch old MSR in r31 */
LL64(r31, KVM_MAGIC_PAGE + KVM_MAGIC_MSR, 0)
/* Find the changed bits between old and new MSR */
kvm_emulate_mtmsr_reg1:
KVM: PPC: Make PV mtmsr work with r30 and r31 So far we've been restricting ourselves to r0-r29 as registers an mtmsr instruction could use. This was bad, as there are some code paths in Linux actually using r30. So let's instead handle all registers gracefully and get rid of that stupid limitation Signed-off-by: Alexander Graf <agraf@suse.de>
2010-08-05 17:26:04 +08:00
ori r30, r0, 0
xor r31, r30, r31
KVM: PPC: PV mtmsrd L=0 and mtmsr There is also a form of mtmsr where all bits need to be addressed. While the PPC64 Linux kernel behaves resonably well here, on PPC32 we do not have an L=1 form. It does mtmsr even for simple things like only changing EE. So we need to hook into that one as well and check for a mask of bits that we deem safe to change from within guest context. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:48:05 +08:00
/* Check if we need to really do mtmsr */
LOAD_REG_IMMEDIATE(r30, MSR_CRITICAL_BITS)
and. r31, r31, r30
/* No critical bits changed? Maybe we can stay in the guest. */
beq maybe_stay_in_guest
do_mtmsr:
SCRATCH_RESTORE
/* Just fire off the mtmsr if it's critical */
kvm_emulate_mtmsr_orig_ins:
mtmsr r0
b kvm_emulate_mtmsr_branch
maybe_stay_in_guest:
KVM: PPC: Make PV mtmsr work with r30 and r31 So far we've been restricting ourselves to r0-r29 as registers an mtmsr instruction could use. This was bad, as there are some code paths in Linux actually using r30. So let's instead handle all registers gracefully and get rid of that stupid limitation Signed-off-by: Alexander Graf <agraf@suse.de>
2010-08-05 17:26:04 +08:00
/* Get the target register in r30 */
kvm_emulate_mtmsr_reg2:
ori r30, r0, 0
PPC: Fix race in mtmsr paravirt implementation The current implementation of mtmsr and mtmsrd are racy in that it does: * check (int_pending == 0) ---> host sets int_pending = 1 <--- * write shared page * done while instead we should check for int_pending after the shared page is written. Signed-off-by: Bharat Bhushan <bharat.bhushan@freescale.com> Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2011-10-13 17:47:08 +08:00
/* Put MSR into magic page because we don't call mtmsr */
STL64(r30, KVM_MAGIC_PAGE + KVM_MAGIC_MSR, 0)
KVM: PPC: PV mtmsrd L=0 and mtmsr There is also a form of mtmsr where all bits need to be addressed. While the PPC64 Linux kernel behaves resonably well here, on PPC32 we do not have an L=1 form. It does mtmsr even for simple things like only changing EE. So we need to hook into that one as well and check for a mask of bits that we deem safe to change from within guest context. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:48:05 +08:00
/* Check if we have to fetch an interrupt */
lwz r31, (KVM_MAGIC_PAGE + KVM_MAGIC_INT)(0)
cmpwi r31, 0
beq+ no_mtmsr
/* Check if we may trigger an interrupt */
KVM: PPC: Make PV mtmsr work with r30 and r31 So far we've been restricting ourselves to r0-r29 as registers an mtmsr instruction could use. This was bad, as there are some code paths in Linux actually using r30. So let's instead handle all registers gracefully and get rid of that stupid limitation Signed-off-by: Alexander Graf <agraf@suse.de>
2010-08-05 17:26:04 +08:00
andi. r31, r30, MSR_EE
PPC: Fix race in mtmsr paravirt implementation The current implementation of mtmsr and mtmsrd are racy in that it does: * check (int_pending == 0) ---> host sets int_pending = 1 <--- * write shared page * done while instead we should check for int_pending after the shared page is written. Signed-off-by: Bharat Bhushan <bharat.bhushan@freescale.com> Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2011-10-13 17:47:08 +08:00
bne do_mtmsr
KVM: PPC: PV mtmsrd L=0 and mtmsr There is also a form of mtmsr where all bits need to be addressed. While the PPC64 Linux kernel behaves resonably well here, on PPC32 we do not have an L=1 form. It does mtmsr even for simple things like only changing EE. So we need to hook into that one as well and check for a mask of bits that we deem safe to change from within guest context. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:48:05 +08:00
no_mtmsr:
SCRATCH_RESTORE
/* Go back to caller */
kvm_emulate_mtmsr_branch:
b .
kvm_emulate_mtmsr_end:
.global kvm_emulate_mtmsr_branch_offs
kvm_emulate_mtmsr_branch_offs:
.long (kvm_emulate_mtmsr_branch - kvm_emulate_mtmsr) / 4
.global kvm_emulate_mtmsr_reg1_offs
kvm_emulate_mtmsr_reg1_offs:
.long (kvm_emulate_mtmsr_reg1 - kvm_emulate_mtmsr) / 4
.global kvm_emulate_mtmsr_reg2_offs
kvm_emulate_mtmsr_reg2_offs:
.long (kvm_emulate_mtmsr_reg2 - kvm_emulate_mtmsr) / 4
.global kvm_emulate_mtmsr_orig_ins_offs
kvm_emulate_mtmsr_orig_ins_offs:
.long (kvm_emulate_mtmsr_orig_ins - kvm_emulate_mtmsr) / 4
.global kvm_emulate_mtmsr_len
kvm_emulate_mtmsr_len:
.long (kvm_emulate_mtmsr_end - kvm_emulate_mtmsr) / 4
KVM: PPC: PV wrteei On BookE the preferred way to write the EE bit is the wrteei instruction. It already encodes the EE bit in the instruction. So in order to get BookE some speedups as well, let's also PV'nize thati instruction. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:48:06 +08:00
powerpc/kvm: Add ifdefs around template code Some of the templates used for KVM patching are only used on certain platforms, but currently they are always built-in, fix that. Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190911115746.12433-4-mpe@ellerman.id.au
2019-09-11 19:57:46 +08:00
#ifdef CONFIG_BOOKE
KVM: PPC: booke: Paravirtualize wrtee Also fix wrteei 1 paravirt to check for a pending interrupt. Signed-off-by: Scott Wood <scottwood@freescale.com> Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2011-11-09 08:23:28 +08:00
/* also used for wrteei 1 */
.global kvm_emulate_wrtee
kvm_emulate_wrtee:
KVM: PPC: PV wrteei On BookE the preferred way to write the EE bit is the wrteei instruction. It already encodes the EE bit in the instruction. So in order to get BookE some speedups as well, let's also PV'nize thati instruction. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:48:06 +08:00
KVM: PPC: booke: Paravirtualize wrtee Also fix wrteei 1 paravirt to check for a pending interrupt. Signed-off-by: Scott Wood <scottwood@freescale.com> Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2011-11-09 08:23:28 +08:00
SCRATCH_SAVE
/* Fetch old MSR in r31 */
LL64(r31, KVM_MAGIC_PAGE + KVM_MAGIC_MSR, 0)
KVM: PPC: PV wrteei On BookE the preferred way to write the EE bit is the wrteei instruction. It already encodes the EE bit in the instruction. So in order to get BookE some speedups as well, let's also PV'nize thati instruction. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:48:06 +08:00
KVM: PPC: booke: Paravirtualize wrtee Also fix wrteei 1 paravirt to check for a pending interrupt. Signed-off-by: Scott Wood <scottwood@freescale.com> Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2011-11-09 08:23:28 +08:00
/* Insert new MSR[EE] */
kvm_emulate_wrtee_reg:
ori r30, r0, 0
rlwimi r31, r30, 0, MSR_EE
/*
* If MSR[EE] is now set, check for a pending interrupt.
* We could skip this if MSR[EE] was already on, but that
* should be rare, so don't bother.
*/
andi. r30, r30, MSR_EE
KVM: PPC: PV wrteei On BookE the preferred way to write the EE bit is the wrteei instruction. It already encodes the EE bit in the instruction. So in order to get BookE some speedups as well, let's also PV'nize thati instruction. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:48:06 +08:00
KVM: PPC: booke: Paravirtualize wrtee Also fix wrteei 1 paravirt to check for a pending interrupt. Signed-off-by: Scott Wood <scottwood@freescale.com> Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2011-11-09 08:23:28 +08:00
/* Put MSR into magic page because we don't call wrtee */
STL64(r31, KVM_MAGIC_PAGE + KVM_MAGIC_MSR, 0)
beq no_wrtee
/* Check if we have to fetch an interrupt */
lwz r30, (KVM_MAGIC_PAGE + KVM_MAGIC_INT)(0)
cmpwi r30, 0
bne do_wrtee
no_wrtee:
SCRATCH_RESTORE
/* Go back to caller */
kvm_emulate_wrtee_branch:
b .
do_wrtee:
SCRATCH_RESTORE
/* Just fire off the wrtee if it's critical */
kvm_emulate_wrtee_orig_ins:
wrtee r0
b kvm_emulate_wrtee_branch
kvm_emulate_wrtee_end:
.global kvm_emulate_wrtee_branch_offs
kvm_emulate_wrtee_branch_offs:
.long (kvm_emulate_wrtee_branch - kvm_emulate_wrtee) / 4
.global kvm_emulate_wrtee_reg_offs
kvm_emulate_wrtee_reg_offs:
.long (kvm_emulate_wrtee_reg - kvm_emulate_wrtee) / 4
.global kvm_emulate_wrtee_orig_ins_offs
kvm_emulate_wrtee_orig_ins_offs:
.long (kvm_emulate_wrtee_orig_ins - kvm_emulate_wrtee) / 4
.global kvm_emulate_wrtee_len
kvm_emulate_wrtee_len:
.long (kvm_emulate_wrtee_end - kvm_emulate_wrtee) / 4
.global kvm_emulate_wrteei_0
kvm_emulate_wrteei_0:
KVM: PPC: PV wrteei On BookE the preferred way to write the EE bit is the wrteei instruction. It already encodes the EE bit in the instruction. So in order to get BookE some speedups as well, let's also PV'nize thati instruction. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:48:06 +08:00
SCRATCH_SAVE
/* Fetch old MSR in r31 */
LL64(r31, KVM_MAGIC_PAGE + KVM_MAGIC_MSR, 0)
/* Remove MSR_EE from old MSR */
KVM: PPC: booke: Paravirtualize wrtee Also fix wrteei 1 paravirt to check for a pending interrupt. Signed-off-by: Scott Wood <scottwood@freescale.com> Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2011-11-09 08:23:28 +08:00
rlwinm r31, r31, 0, ~MSR_EE
KVM: PPC: PV wrteei On BookE the preferred way to write the EE bit is the wrteei instruction. It already encodes the EE bit in the instruction. So in order to get BookE some speedups as well, let's also PV'nize thati instruction. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:48:06 +08:00
/* Write new MSR value back */
STL64(r31, KVM_MAGIC_PAGE + KVM_MAGIC_MSR, 0)
SCRATCH_RESTORE
/* Go back to caller */
KVM: PPC: booke: Paravirtualize wrtee Also fix wrteei 1 paravirt to check for a pending interrupt. Signed-off-by: Scott Wood <scottwood@freescale.com> Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2011-11-09 08:23:28 +08:00
kvm_emulate_wrteei_0_branch:
KVM: PPC: PV wrteei On BookE the preferred way to write the EE bit is the wrteei instruction. It already encodes the EE bit in the instruction. So in order to get BookE some speedups as well, let's also PV'nize thati instruction. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:48:06 +08:00
b .
KVM: PPC: booke: Paravirtualize wrtee Also fix wrteei 1 paravirt to check for a pending interrupt. Signed-off-by: Scott Wood <scottwood@freescale.com> Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2011-11-09 08:23:28 +08:00
kvm_emulate_wrteei_0_end:
KVM: PPC: PV wrteei On BookE the preferred way to write the EE bit is the wrteei instruction. It already encodes the EE bit in the instruction. So in order to get BookE some speedups as well, let's also PV'nize thati instruction. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-07-29 20:48:06 +08:00
KVM: PPC: booke: Paravirtualize wrtee Also fix wrteei 1 paravirt to check for a pending interrupt. Signed-off-by: Scott Wood <scottwood@freescale.com> Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2011-11-09 08:23:28 +08:00
.global kvm_emulate_wrteei_0_branch_offs
kvm_emulate_wrteei_0_branch_offs:
.long (kvm_emulate_wrteei_0_branch - kvm_emulate_wrteei_0) / 4
KVM: PPC: Add mtsrin PV code This is the guest side of the mtsr acceleration. Using this a guest can now call mtsrin with almost no overhead as long as it ensures that it only uses it with (MSR_IR|MSR_DR) == 0. Linux does that, so we're good. Signed-off-by: Alexander Graf <agraf@suse.de>
2010-08-03 16:39:35 +08:00
KVM: PPC: booke: Paravirtualize wrtee Also fix wrteei 1 paravirt to check for a pending interrupt. Signed-off-by: Scott Wood <scottwood@freescale.com> Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2011-11-09 08:23:28 +08:00
.global kvm_emulate_wrteei_0_len
kvm_emulate_wrteei_0_len:
.long (kvm_emulate_wrteei_0_end - kvm_emulate_wrteei_0) / 4
KVM: PPC: Add mtsrin PV code This is the guest side of the mtsr acceleration. Using this a guest can now call mtsrin with almost no overhead as long as it ensures that it only uses it with (MSR_IR|MSR_DR) == 0. Linux does that, so we're good. Signed-off-by: Alexander Graf <agraf@suse.de>
2010-08-03 16:39:35 +08:00
powerpc/kvm: Add ifdefs around template code Some of the templates used for KVM patching are only used on certain platforms, but currently they are always built-in, fix that. Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190911115746.12433-4-mpe@ellerman.id.au
2019-09-11 19:57:46 +08:00
#endif /* CONFIG_BOOKE */
#ifdef CONFIG_PPC_BOOK3S_32
KVM: PPC: Add mtsrin PV code This is the guest side of the mtsr acceleration. Using this a guest can now call mtsrin with almost no overhead as long as it ensures that it only uses it with (MSR_IR|MSR_DR) == 0. Linux does that, so we're good. Signed-off-by: Alexander Graf <agraf@suse.de>
2010-08-03 16:39:35 +08:00
.global kvm_emulate_mtsrin
kvm_emulate_mtsrin:
SCRATCH_SAVE
LL64(r31, KVM_MAGIC_PAGE + KVM_MAGIC_MSR, 0)
andi. r31, r31, MSR_DR | MSR_IR
beq kvm_emulate_mtsrin_reg1
SCRATCH_RESTORE
kvm_emulate_mtsrin_orig_ins:
nop
b kvm_emulate_mtsrin_branch
kvm_emulate_mtsrin_reg1:
/* rX >> 26 */
rlwinm r30,r0,6,26,29
kvm_emulate_mtsrin_reg2:
stw r0, (KVM_MAGIC_PAGE + KVM_MAGIC_SR)(r30)
SCRATCH_RESTORE
/* Go back to caller */
kvm_emulate_mtsrin_branch:
b .
kvm_emulate_mtsrin_end:
.global kvm_emulate_mtsrin_branch_offs
kvm_emulate_mtsrin_branch_offs:
.long (kvm_emulate_mtsrin_branch - kvm_emulate_mtsrin) / 4
.global kvm_emulate_mtsrin_reg1_offs
kvm_emulate_mtsrin_reg1_offs:
.long (kvm_emulate_mtsrin_reg1 - kvm_emulate_mtsrin) / 4
.global kvm_emulate_mtsrin_reg2_offs
kvm_emulate_mtsrin_reg2_offs:
.long (kvm_emulate_mtsrin_reg2 - kvm_emulate_mtsrin) / 4
.global kvm_emulate_mtsrin_orig_ins_offs
kvm_emulate_mtsrin_orig_ins_offs:
.long (kvm_emulate_mtsrin_orig_ins - kvm_emulate_mtsrin) / 4
.global kvm_emulate_mtsrin_len
kvm_emulate_mtsrin_len:
.long (kvm_emulate_mtsrin_end - kvm_emulate_mtsrin) / 4
KVM: PPC: Avoid patching paravirt template code Currently we patch the whole code include paravirt template code. This isn't safe for scratch area and has impact to performance. Signed-off-by: Liu Yu <yu.liu@freescale.com> Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2011-12-02 04:22:53 +08:00
powerpc/kvm: Add ifdefs around template code Some of the templates used for KVM patching are only used on certain platforms, but currently they are always built-in, fix that. Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190911115746.12433-4-mpe@ellerman.id.au
2019-09-11 19:57:46 +08:00
#endif /* CONFIG_PPC_BOOK3S_32 */
powerpc/kvm: Move kvm_tmp into .text, shrink to 64K In some configurations of KVM, guests binary patch themselves to avoid/reduce trapping into the hypervisor. For some instructions this requires replacing one instruction with a sequence of instructions. For those cases we need to write the sequence of instructions somewhere and then patch the location of the original instruction to branch to the sequence. That requires that the location of the sequence be within 32MB of the original instruction. The current solution for this is that we create a 1MB array in BSS, write sequences into there, and then free the remainder of the array. This has a few problems: - it confuses kmemleak. - it confuses lockdep. - it requires mapping kvm_tmp executable, which can cause adjacent areas to also be mapped executable if we're using 16M pages for the linear mapping. - the 32MB limit can be exceeded if the kernel is big enough, especially with STRICT_KERNEL_RWX enabled, which then prevents the patching from working at all. We can fix all those problems by making kvm_tmp just a region of regular .text. However currently it's 1MB in size, and we don't want to waste 1MB of text. In practice however I only see ~30KB of kvm_tmp being used even for an allyes_config. So shrink kvm_tmp to 64K, which ought to be enough for everyone, and move it into .text. Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190911115746.12433-1-mpe@ellerman.id.au
2019-09-11 19:57:43 +08:00
.balign 4
.global kvm_tmp
kvm_tmp:
.space (64 * 1024)
.global kvm_tmp_end
kvm_tmp_end:
KVM: PPC: Avoid patching paravirt template code Currently we patch the whole code include paravirt template code. This isn't safe for scratch area and has impact to performance. Signed-off-by: Liu Yu <yu.liu@freescale.com> Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2011-12-02 04:22:53 +08:00
.global kvm_template_end
kvm_template_end: