2015-10-19 22:50:37 +08:00
|
|
|
/*
|
|
|
|
|
* Copyright (C) 2012-2015 - ARM Ltd
|
|
|
|
|
* Author: Marc Zyngier <marc.zyngier@arm.com>
|
|
|
|
|
*
|
|
|
|
|
* This program is free software; you can redistribute it and/or modify
|
|
|
|
|
* it under the terms of the GNU General Public License version 2 as
|
|
|
|
|
* published by the Free Software Foundation.
|
|
|
|
|
*
|
|
|
|
|
* This program is distributed in the hope that it will be useful,
|
|
|
|
|
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
|
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
|
|
|
* GNU General Public License for more details.
|
|
|
|
|
*
|
|
|
|
|
* You should have received a copy of the GNU General Public License
|
|
|
|
|
* along with this program. If not, see <http://www.gnu.org/licenses/>.
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
#include <linux/compiler.h>
|
|
|
|
|
#include <linux/irqchip/arm-gic.h>
|
|
|
|
|
#include <linux/kvm_host.h>
|
arm64: vgic-v2: Fix proxying of cpuif access
Proxying the cpuif accesses at EL2 makes use of vcpu_data_guest_to_host
and co, which check the endianness, which call into vcpu_read_sys_reg...
which isn't mapped at EL2 (it was inlined before, and got moved OoL
with the VHE optimizations).
The result is of course a nice panic. Let's add some specialized
cruft to keep the broken platforms that require this hack alive.
But, this code used vcpu_data_guest_to_host(), which expected us to
write the value to host memory, instead we have trapped the guest's
read or write to an mmio-device, and are about to replay it using the
host's readl()/writel() which also perform swabbing based on the host
endianness. This goes wrong when both host and guest are big-endian,
as readl()/writel() will undo the guest's swabbing, causing the
big-endian value to be written to device-memory.
What needs doing?
A big-endian guest will have pre-swabbed data before storing, undo this.
If its necessary for the host, writel() will re-swab it.
For a read a big-endian guest expects to swab the data after the load.
The hosts's readl() will correct for host endianness, giving us the
device-memory's value in the register. For a big-endian guest, swab it
as if we'd only done the load.
For a little-endian guest, nothing needs doing as readl()/writel() leave
the correct device-memory value in registers.
Tested on Juno with that rarest of things: a big-endian 64K host.
Based on a patch from Marc Zyngier.
Reported-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Fixes: bf8feb39642b ("arm64: KVM: vgic-v2: Add GICV access from HYP")
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
2018-05-04 23:19:24 +08:00
|
|
|
#include <linux/swab.h>
|
2015-10-19 22:50:37 +08:00
|
|
|
|
2016-09-06 16:28:46 +08:00
|
|
|
#include <asm/kvm_emulate.h>
|
2016-01-28 21:44:07 +08:00
|
|
|
#include <asm/kvm_hyp.h>
|
2017-10-24 00:11:14 +08:00
|
|
|
#include <asm/kvm_mmu.h>
|
2015-10-19 22:50:37 +08:00
|
|
|
|
arm64: vgic-v2: Fix proxying of cpuif access
Proxying the cpuif accesses at EL2 makes use of vcpu_data_guest_to_host
and co, which check the endianness, which call into vcpu_read_sys_reg...
which isn't mapped at EL2 (it was inlined before, and got moved OoL
with the VHE optimizations).
The result is of course a nice panic. Let's add some specialized
cruft to keep the broken platforms that require this hack alive.
But, this code used vcpu_data_guest_to_host(), which expected us to
write the value to host memory, instead we have trapped the guest's
read or write to an mmio-device, and are about to replay it using the
host's readl()/writel() which also perform swabbing based on the host
endianness. This goes wrong when both host and guest are big-endian,
as readl()/writel() will undo the guest's swabbing, causing the
big-endian value to be written to device-memory.
What needs doing?
A big-endian guest will have pre-swabbed data before storing, undo this.
If its necessary for the host, writel() will re-swab it.
For a read a big-endian guest expects to swab the data after the load.
The hosts's readl() will correct for host endianness, giving us the
device-memory's value in the register. For a big-endian guest, swab it
as if we'd only done the load.
For a little-endian guest, nothing needs doing as readl()/writel() leave
the correct device-memory value in registers.
Tested on Juno with that rarest of things: a big-endian 64K host.
Based on a patch from Marc Zyngier.
Reported-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Fixes: bf8feb39642b ("arm64: KVM: vgic-v2: Add GICV access from HYP")
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
2018-05-04 23:19:24 +08:00
|
|
|
static bool __hyp_text __is_be(struct kvm_vcpu *vcpu)
|
|
|
|
|
{
|
|
|
|
|
if (vcpu_mode_is_32bit(vcpu))
|
2018-07-05 22:16:53 +08:00
|
|
|
return !!(read_sysreg_el2(spsr) & PSR_AA32_E_BIT);
|
arm64: vgic-v2: Fix proxying of cpuif access
Proxying the cpuif accesses at EL2 makes use of vcpu_data_guest_to_host
and co, which check the endianness, which call into vcpu_read_sys_reg...
which isn't mapped at EL2 (it was inlined before, and got moved OoL
with the VHE optimizations).
The result is of course a nice panic. Let's add some specialized
cruft to keep the broken platforms that require this hack alive.
But, this code used vcpu_data_guest_to_host(), which expected us to
write the value to host memory, instead we have trapped the guest's
read or write to an mmio-device, and are about to replay it using the
host's readl()/writel() which also perform swabbing based on the host
endianness. This goes wrong when both host and guest are big-endian,
as readl()/writel() will undo the guest's swabbing, causing the
big-endian value to be written to device-memory.
What needs doing?
A big-endian guest will have pre-swabbed data before storing, undo this.
If its necessary for the host, writel() will re-swab it.
For a read a big-endian guest expects to swab the data after the load.
The hosts's readl() will correct for host endianness, giving us the
device-memory's value in the register. For a big-endian guest, swab it
as if we'd only done the load.
For a little-endian guest, nothing needs doing as readl()/writel() leave
the correct device-memory value in registers.
Tested on Juno with that rarest of things: a big-endian 64K host.
Based on a patch from Marc Zyngier.
Reported-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Fixes: bf8feb39642b ("arm64: KVM: vgic-v2: Add GICV access from HYP")
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
2018-05-04 23:19:24 +08:00
|
|
|
|
|
|
|
|
return !!(read_sysreg(SCTLR_EL1) & SCTLR_ELx_EE);
|
|
|
|
|
}
|
|
|
|
|
|
2016-09-06 21:02:17 +08:00
|
|
|
/*
|
|
|
|
|
* __vgic_v2_perform_cpuif_access -- perform a GICV access on behalf of the
|
|
|
|
|
* guest.
|
|
|
|
|
*
|
|
|
|
|
* @vcpu: the offending vcpu
|
|
|
|
|
*
|
|
|
|
|
* Returns:
|
|
|
|
|
* 1: GICV access successfully performed
|
|
|
|
|
* 0: Not a GICV access
|
|
|
|
|
* -1: Illegal GICV access
|
|
|
|
|
*/
|
|
|
|
|
int __hyp_text __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu)
|
arm64: KVM: vgic-v2: Add the GICV emulation infrastructure
In order to efficiently perform the GICV access on behalf of the
guest, we need to be able to avoid going back all the way to
the host kernel.
For this, we introduce a new hook in the world switch code,
conveniently placed just after populating the fault info.
At that point, we only have saved/restored the GP registers,
and we can quickly perform all the required checks (data abort,
translation fault, valid faulting syndrome, not an external
abort, not a PTW).
Coming back from the emulation code, we need to skip the emulated
instruction. This involves an additional bit of save/restore in
order to be able to access the guest's PC (and possibly CPSR if
this is a 32bit guest).
At this stage, no emulation code is provided.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
2016-09-06 16:28:45 +08:00
|
|
|
{
|
2016-09-06 16:28:46 +08:00
|
|
|
struct kvm *kvm = kern_hyp_va(vcpu->kvm);
|
|
|
|
|
struct vgic_dist *vgic = &kvm->arch.vgic;
|
|
|
|
|
phys_addr_t fault_ipa;
|
|
|
|
|
void __iomem *addr;
|
|
|
|
|
int rd;
|
|
|
|
|
|
|
|
|
|
/* Build the full address */
|
|
|
|
|
fault_ipa = kvm_vcpu_get_fault_ipa(vcpu);
|
|
|
|
|
fault_ipa |= kvm_vcpu_get_hfar(vcpu) & GENMASK(11, 0);
|
|
|
|
|
|
|
|
|
|
/* If not for GICV, move on */
|
|
|
|
|
if (fault_ipa < vgic->vgic_cpu_base ||
|
|
|
|
|
fault_ipa >= (vgic->vgic_cpu_base + KVM_VGIC_V2_CPU_SIZE))
|
2016-09-06 21:02:17 +08:00
|
|
|
return 0;
|
2016-09-06 16:28:46 +08:00
|
|
|
|
|
|
|
|
/* Reject anything but a 32bit access */
|
|
|
|
|
if (kvm_vcpu_dabt_get_as(vcpu) != sizeof(u32))
|
2016-09-06 21:02:17 +08:00
|
|
|
return -1;
|
2016-09-06 16:28:46 +08:00
|
|
|
|
|
|
|
|
/* Not aligned? Don't bother */
|
|
|
|
|
if (fault_ipa & 3)
|
2016-09-06 21:02:17 +08:00
|
|
|
return -1;
|
2016-09-06 16:28:46 +08:00
|
|
|
|
|
|
|
|
rd = kvm_vcpu_dabt_get_rd(vcpu);
|
2017-12-05 00:43:23 +08:00
|
|
|
addr = hyp_symbol_addr(kvm_vgic_global_state)->vcpu_hyp_va;
|
2016-09-06 16:28:46 +08:00
|
|
|
addr += fault_ipa - vgic->vgic_cpu_base;
|
|
|
|
|
|
|
|
|
|
if (kvm_vcpu_dabt_iswrite(vcpu)) {
|
arm64: vgic-v2: Fix proxying of cpuif access
Proxying the cpuif accesses at EL2 makes use of vcpu_data_guest_to_host
and co, which check the endianness, which call into vcpu_read_sys_reg...
which isn't mapped at EL2 (it was inlined before, and got moved OoL
with the VHE optimizations).
The result is of course a nice panic. Let's add some specialized
cruft to keep the broken platforms that require this hack alive.
But, this code used vcpu_data_guest_to_host(), which expected us to
write the value to host memory, instead we have trapped the guest's
read or write to an mmio-device, and are about to replay it using the
host's readl()/writel() which also perform swabbing based on the host
endianness. This goes wrong when both host and guest are big-endian,
as readl()/writel() will undo the guest's swabbing, causing the
big-endian value to be written to device-memory.
What needs doing?
A big-endian guest will have pre-swabbed data before storing, undo this.
If its necessary for the host, writel() will re-swab it.
For a read a big-endian guest expects to swab the data after the load.
The hosts's readl() will correct for host endianness, giving us the
device-memory's value in the register. For a big-endian guest, swab it
as if we'd only done the load.
For a little-endian guest, nothing needs doing as readl()/writel() leave
the correct device-memory value in registers.
Tested on Juno with that rarest of things: a big-endian 64K host.
Based on a patch from Marc Zyngier.
Reported-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Fixes: bf8feb39642b ("arm64: KVM: vgic-v2: Add GICV access from HYP")
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
2018-05-04 23:19:24 +08:00
|
|
|
u32 data = vcpu_get_reg(vcpu, rd);
|
|
|
|
|
if (__is_be(vcpu)) {
|
|
|
|
|
/* guest pre-swabbed data, undo this for writel() */
|
|
|
|
|
data = swab32(data);
|
|
|
|
|
}
|
2016-09-06 16:28:46 +08:00
|
|
|
writel_relaxed(data, addr);
|
|
|
|
|
} else {
|
|
|
|
|
u32 data = readl_relaxed(addr);
|
arm64: vgic-v2: Fix proxying of cpuif access
Proxying the cpuif accesses at EL2 makes use of vcpu_data_guest_to_host
and co, which check the endianness, which call into vcpu_read_sys_reg...
which isn't mapped at EL2 (it was inlined before, and got moved OoL
with the VHE optimizations).
The result is of course a nice panic. Let's add some specialized
cruft to keep the broken platforms that require this hack alive.
But, this code used vcpu_data_guest_to_host(), which expected us to
write the value to host memory, instead we have trapped the guest's
read or write to an mmio-device, and are about to replay it using the
host's readl()/writel() which also perform swabbing based on the host
endianness. This goes wrong when both host and guest are big-endian,
as readl()/writel() will undo the guest's swabbing, causing the
big-endian value to be written to device-memory.
What needs doing?
A big-endian guest will have pre-swabbed data before storing, undo this.
If its necessary for the host, writel() will re-swab it.
For a read a big-endian guest expects to swab the data after the load.
The hosts's readl() will correct for host endianness, giving us the
device-memory's value in the register. For a big-endian guest, swab it
as if we'd only done the load.
For a little-endian guest, nothing needs doing as readl()/writel() leave
the correct device-memory value in registers.
Tested on Juno with that rarest of things: a big-endian 64K host.
Based on a patch from Marc Zyngier.
Reported-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Fixes: bf8feb39642b ("arm64: KVM: vgic-v2: Add GICV access from HYP")
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
2018-05-04 23:19:24 +08:00
|
|
|
if (__is_be(vcpu)) {
|
|
|
|
|
/* guest expects swabbed data */
|
|
|
|
|
data = swab32(data);
|
|
|
|
|
}
|
|
|
|
|
vcpu_set_reg(vcpu, rd, data);
|
2016-09-06 16:28:46 +08:00
|
|
|
}
|
|
|
|
|
|
2016-09-06 21:02:17 +08:00
|
|
|
return 1;
|
arm64: KVM: vgic-v2: Add the GICV emulation infrastructure
In order to efficiently perform the GICV access on behalf of the
guest, we need to be able to avoid going back all the way to
the host kernel.
For this, we introduce a new hook in the world switch code,
conveniently placed just after populating the fault info.
At that point, we only have saved/restored the GP registers,
and we can quickly perform all the required checks (data abort,
translation fault, valid faulting syndrome, not an external
abort, not a PTW).
Coming back from the emulation code, we need to skip the emulated
instruction. This involves an additional bit of save/restore in
order to be able to access the guest's PC (and possibly CPSR if
this is a 32bit guest).
At this stage, no emulation code is provided.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
2016-09-06 16:28:45 +08:00
|
|
|
}
|