Alexei Starovoitov 8c290e60fa bpf: fix hashmap extra_elems logic ... In both kmalloc and prealloc mode the bpf_map_update_elem() is using per-cpu extra_elems to do atomic update when the map is full. There are two issues with it. The logic can be misused, since it allows max_entries+num_cpus elements to be present in the map. And alloc_extra_elems() at map creation time can fail percpu alloc for large map values with a warn: WARNING: CPU: 3 PID: 2752 at ../mm/percpu.c:892 pcpu_alloc+0x119/0xa60 illegal size (32824) or align (8) for percpu allocation The fixes for both of these issues are different for kmalloc and prealloc modes. For prealloc mode allocate extra num_possible_cpus elements and store their pointers into extra_elems array instead of actual elements. Hence we can use these hidden(spare) elements not only when the map is full but during bpf_map_update_elem() that replaces existing element too. That also improves performance, since pcpu_freelist_pop/push is avoided. Unfortunately this approach cannot be used for kmalloc mode which needs to kfree elements after rcu grace period. Therefore switch it back to normal kmalloc even when full and old element exists like it was prior to commit 6c90598174 ("bpf: pre-allocate hash map elements"). Add tests to check for over max_entries and large map values. Reported-by: Dave Jones <davej@codemonkey.org.uk> Fixes: 6c90598174 ("bpf: pre-allocate hash map elements") Signed-off-by: Alexei Starovoitov <ast@kernel.org> Acked-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Martin KaFai Lau <kafai@fb.com> Signed-off-by: David S. Miller <davem@davemloft.net>		2017-03-22 14:12:18 -07:00
..
Makefile	bpf: add a longest prefix match trie map implementation	2017-01-23 16:10:38 -05:00
arraymap.c	bpf: mark all registered map/prog types as __ro_after_init	2017-02-17 13:40:04 -05:00
bpf_lru_list.c	bpf: Make unnecessarily global functions static	2017-01-10 21:00:59 -05:00
bpf_lru_list.h	bpf: Add percpu LRU list	2016-11-15 11:50:20 -05:00
cgroup.c	bpf: introduce BPF_F_ALLOW_OVERRIDE flag	2017-02-12 21:52:19 -05:00
core.c	bpf: make jited programs visible in traces	2017-02-17 13:40:05 -05:00
hashtab.c	bpf: fix hashmap extra_elems logic	2017-03-22 14:12:18 -07:00
helpers.c	bpf: rename ARG_PTR_TO_STACK	2017-01-09 16:56:27 -05:00
inode.c	bpf: add initial bpf tracepoints	2017-01-25 13:17:47 -05:00
lpm_trie.c	bpf: add get_next_key callback to LPM map	2017-03-05 17:55:29 -08:00
percpu_freelist.c	bpf: introduce percpu_freelist	2016-03-08 15:28:31 -05:00
percpu_freelist.h	bpf: introduce percpu_freelist	2016-03-08 15:28:31 -05:00
stackmap.c	bpf: mark all registered map/prog types as __ro_after_init	2017-02-17 13:40:04 -05:00
syscall.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/signal.h>	2017-03-02 08:42:29 +01:00
verifier.c	bpf: update the comment about the length of analysis	2017-03-01 14:56:50 -08:00