Commit 66d0d5a854 ("riscv: bpf: eliminate zero extension code-gen")
added the new zero-extension optimization for some BPF ALU operations.
Since then, bugs in the JIT that have been fixed in the bpf tree require
this optimization to be added to other operations: commit 1e692f09e0
("bpf, riscv: clear high 32 bits for ALU32 add/sub/neg/lsh/rsh/arsh"),
and commit fe121ee531 ("bpf, riscv: clear target register high 32-bits
for and/or/xor on ALU32").
Now that these have been merged to bpf-next, the zext optimization can
be enabled for the fixed operations.
Signed-off-by: Luke Nelson <luke.r.nels@gmail.com>
Cc: Song Liu <liu.song.a23@gmail.com>
Cc: Jiong Wang <jiong.wang@netronome.com>
Cc: Xi Wang <xi.wang@gmail.com>
Acked-by: Björn Töpel <bjorn.topel@gmail.com>
Acked-by: Jiong Wang <jiong.wang@netronome.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
In BPF, 32-bit ALU operations should zero-extend their results into
the 64-bit registers.
The current BPF JIT on RISC-V emits incorrect instructions that perform
sign extension only (e.g., addw, subw) on 32-bit add, sub, lsh, rsh,
arsh, and neg. This behavior diverges from the interpreter and JITs
for other architectures.
This patch fixes the bugs by performing zero extension on the destination
register of 32-bit ALU operations.
Fixes: 2353ecc6f9 ("bpf, riscv: add BPF JIT for RV64G")
Cc: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luke Nelson <luke.r.nels@gmail.com>
Acked-by: Song Liu <songliubraving@fb.com>
Acked-by: Björn Töpel <bjorn.topel@gmail.com>
Reviewed-by: Palmer Dabbelt <palmer@sifive.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Cc: Björn Töpel <bjorn.topel@gmail.com>
Acked-by: Björn Töpel <bjorn.topel@gmail.com>
Tested-by: Björn Töpel <bjorn.topel@gmail.com>
Signed-off-by: Jiong Wang <jiong.wang@netronome.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
When using 32-bit subregisters (ALU32), the RISC-V JIT would not clear
the high 32-bits of the target register and therefore generate
incorrect code.
E.g., in the following code:
$ cat test.c
unsigned int f(unsigned long long a,
unsigned int b)
{
return (unsigned int)a & b;
}
$ clang-9 -target bpf -O2 -emit-llvm -S test.c -o - | \
llc-9 -mattr=+alu32 -mcpu=v3
.text
.file "test.c"
.globl f
.p2align 3
.type f,@function
f:
r0 = r1
w0 &= w2
exit
.Lfunc_end0:
.size f, .Lfunc_end0-f
The JIT would not clear the high 32-bits of r0 after the
and-operation, which in this case might give an incorrect return
value.
After this patch, that is not the case, and the upper 32-bits are
cleared.
Reported-by: Jiong Wang <jiong.wang@netronome.com>
Fixes: 2353ecc6f9 ("bpf, riscv: add BPF JIT for RV64G")
Signed-off-by: Björn Töpel <bjorn.topel@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
This commit adds a BPF JIT for RV64G.
The JIT is a two-pass JIT, and has a dynamic prolog/epilogue (similar
to the MIPS64 BPF JIT) instead of static ones (e.g. x86_64).
At the moment the RISC-V Linux port does not support
CONFIG_HAVE_KPROBES, which means that CONFIG_BPF_EVENTS is not
supported. Thus, no tests involving BPF_PROG_TYPE_TRACEPOINT,
BPF_PROG_TYPE_PERF_EVENT, BPF_PROG_TYPE_KPROBE and
BPF_PROG_TYPE_RAW_TRACEPOINT passes.
The implementation does not support "far branching" (>4KiB).
Test results:
# modprobe test_bpf
test_bpf: Summary: 378 PASSED, 0 FAILED, [366/366 JIT'ed]
# echo 1 > /proc/sys/kernel/unprivileged_bpf_disabled
# ./test_verifier
...
Summary: 761 PASSED, 507 SKIPPED, 2 FAILED
Note that "test_verifier" was run with one build with
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y and one without, otherwise
many of the the tests that require unaligned access were skipped.
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y:
# echo 1 > /proc/sys/kernel/unprivileged_bpf_disabled
# ./test_verifier | grep -c 'NOTE.*unknown align'
0
No CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS:
# echo 1 > /proc/sys/kernel/unprivileged_bpf_disabled
# ./test_verifier | grep -c 'NOTE.*unknown align'
59
The two failing test_verifier tests are:
"ld_abs: vlan + abs, test 1"
"ld_abs: jump around ld_abs"
This is due to that "far branching" involved in those tests.
All tests where done on QEMU (QEMU emulator version 3.1.50
(v3.1.0-688-g8ae951fbc106)).
Signed-off-by: Björn Töpel <bjorn.topel@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Luke Nelson