linux/net/ipv4
Shawn Bohrer 421b3885bf udp: ipv4: Add udp early demux
The removal of the routing cache introduced a performance regression for
some UDP workloads since a dst lookup must be done for each packet.
This change caches the dst per socket in a similar manner to what we do
for TCP by implementing early_demux.

For UDP multicast we can only cache the dst if there is only one
receiving socket on the host.  Since caching only works when there is
one receiving socket we do the multicast socket lookup using RCU.

For UDP unicast we only demux sockets with an exact match in order to
not break forwarding setups.  Additionally since the hash chains may be
long we only check the first socket to see if it is a match and not
waste extra time searching the whole chain when we might not find an
exact match.

Benchmark results from a netperf UDP_RR test:
Before 87961.22 transactions/s
After  89789.68 transactions/s

Benchmark results from a fio 1 byte UDP multicast pingpong test
(Multicast one way unicast response):
Before 12.97us RTT
After  12.63us RTT

Signed-off-by: Shawn Bohrer <sbohrer@rgmadvisors.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2013-10-08 16:27:33 -04:00
..
netfilter netfilter: synproxy: fix BUG_ON triggered by corrupt TCP packets 2013-09-30 12:44:38 +02:00
Kconfig net: neighbour: Remove CONFIG_ARPD 2013-09-03 21:41:43 -04:00
Makefile net: gre: move GSO functions to gre_offload 2013-07-03 14:37:39 -07:00
af_inet.c udp: ipv4: Add udp early demux 2013-10-08 16:27:33 -04:00
ah4.c ipv4: properly refresh rtable entries on pmtu/redirect events 2013-06-03 00:07:42 -07:00
arp.c net: neighbour: Remove CONFIG_ARPD 2013-09-03 21:41:43 -04:00
cipso_ipv4.c cipso: don't follow a NULL pointer when setsockopt() is called 2012-07-18 09:01:12 -07:00
datagram.c ipv4: Add a socket release callback for datagram sockets 2013-01-21 14:17:05 -05:00
devinet.c net: igmp: Allow user-space configuration of igmp unsolicited report interval 2013-08-09 11:27:46 -07:00
esp4.c net: esp{4,6}: fix potential MTU calculation overflows 2013-08-05 12:26:50 -07:00
fib_frontend.c netlink: fix splat in skb_clone with large messages 2013-06-27 22:44:16 -07:00
fib_lookup.h ipv4: Fix nexthop caching wrt. scoping. 2011-03-24 18:06:47 -07:00
fib_rules.c fib_rules: fix suppressor names and default values 2013-08-03 10:40:23 -07:00
fib_semantics.c ipv4: use next hop exceptions also for input routes 2013-06-28 21:27:47 -07:00
fib_trie.c fib_trie: avoid a redundant bit judgement in inflate 2013-10-02 16:37:15 -04:00
gre_demux.c net: gre: move GSO functions to gre_offload 2013-07-03 14:37:39 -07:00
gre_offload.c gso: Update tunnel segmentation to support Tx checksum offload 2013-07-11 12:18:49 -07:00
icmp.c ipv4: processing ancillary IP_TOS or IP_TTL 2013-09-28 15:21:52 -07:00
igmp.c ipv4 igmp: use in_dev_put in timer handlers instead of __in_dev_put 2013-09-30 22:28:56 -07:00
inet_connection_sock.c inet: consolidate INET_TW_MATCH 2013-10-03 15:33:35 -04:00
inet_diag.c tcp: shrink tcp6_timewait_sock by one cache line 2013-10-03 17:43:39 -04:00
inet_fragment.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next 2013-07-09 18:24:39 -07:00
inet_hashtables.c net ipv4: Convert ipv4.ip_local_port_range to be per netns v3 2013-09-30 21:59:38 -07:00
inet_lro.c ipv4: replace ip_fast_csum with csum_replace2 2013-03-15 09:12:25 -04:00
inet_timewait_sock.c tcp: shrink tcp6_timewait_sock by one cache line 2013-10-03 17:43:39 -04:00
inetpeer.c ip: generate unique IP identificator if local fragmentation is allowed 2013-09-19 14:11:15 -04:00
ip_forward.c ipv4: introduce rt_uses_gateway 2012-10-08 17:42:36 -04:00
ip_fragment.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2013-04-22 20:32:51 -04:00
ip_gre.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2013-08-16 15:37:26 -07:00
ip_input.c net: add SNMP counters tracking incoming ECN bits 2013-08-08 22:24:59 -07:00
ip_options.c net/ipv4: Ensure that location of timestamp option is stored 2013-03-12 05:35:39 -04:00
ip_output.c ipv4: processing ancillary IP_TOS or IP_TTL 2013-09-28 15:21:52 -07:00
ip_sockglue.c ipv4: IP_TOS and IP_TTL can be specified as ancillary data 2013-09-28 15:21:51 -07:00
ip_tunnel.c ip_tunnel: Remove double unregister of the fallback device 2013-10-01 12:42:16 -04:00
ip_tunnel_core.c ip_tunnel_core: Change __skb_push back to skb_push 2013-10-01 12:42:16 -04:00
ip_vti.c {ipv4,xfrm}: Introduce xfrm_tunnel_notifier for xfrm tunnel mode callback 2013-08-28 09:22:17 +02:00
ipcomp.c ipv4: properly refresh rtable entries on pmtu/redirect events 2013-06-03 00:07:42 -07:00
ipconfig.c ipconfig: add informative timeout messages while waiting for carrier 2013-04-02 14:35:33 -04:00
ipip.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2013-09-05 14:58:52 -04:00
ipmr.c ip: generate unique IP identificator if local fragmentation is allowed 2013-09-19 14:11:15 -04:00
netfilter.c netfilter: add my copyright statements 2013-04-18 20:27:55 +02:00
ping.c net ipv4: Convert ipv4.ip_local_port_range to be per netns v3 2013-09-30 21:59:38 -07:00
proc.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2013-08-16 15:37:26 -07:00
protocol.c ipv4: Disallow non-namespace aware protocols to register. 2013-02-05 14:42:23 -05:00
raw.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2013-10-01 17:06:14 -04:00
route.c ipv4: raise IP_MAX_MTU to theoretical limit 2013-08-20 15:05:04 -07:00
syncookies.c tcp: syncookies: reduce mss table to four values 2013-09-24 10:39:58 -04:00
sysctl_net_ipv4.c net ipv4: Convert ipv4.ip_local_port_range to be per netns v3 2013-09-30 21:59:38 -07:00
tcp.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2013-09-05 14:58:52 -04:00
tcp_bic.c tcp: fix undo after RTO for BIC 2012-01-20 14:17:26 -05:00
tcp_cong.c tcp: remove Appropriate Byte Count support 2013-02-05 14:51:16 -05:00
tcp_cubic.c tcp: cubic: fix bug in bictcp_acked() 2013-08-07 10:35:08 -07:00
tcp_diag.c inet_diag: Rename inet_diag_req into inet_diag_req_v2 2012-01-11 12:56:06 -08:00
tcp_fastopen.c tcp: add server ip to encrypt cookie in fast open 2013-08-10 00:35:33 -07:00
tcp_highspeed.c tcp: mark tcp_congestion_ops read_mostly 2011-03-10 00:40:17 -08:00
tcp_htcp.c tcp: mark tcp_congestion_ops read_mostly 2011-03-10 00:40:17 -08:00
tcp_hybla.c tcp: bool conversions 2012-05-17 14:59:59 -04:00
tcp_illinois.c net: fix divide by zero in tcp algorithm illinois 2012-11-01 11:55:59 -04:00
tcp_input.c net: ipv4: Change variable type to bool 2013-10-03 15:40:34 -04:00
tcp_ipv4.c tcp: sndbuf autotuning improvements 2013-10-02 16:45:17 -04:00
tcp_lp.c Fix common misspellings 2011-03-31 11:26:23 -03:00
tcp_memcontrol.c memcg: rename RESOURCE_MAX to RES_COUNTER_MAX 2013-09-12 15:38:02 -07:00
tcp_metrics.c tcp: fix RTO calculated from cached RTT 2013-09-17 19:08:08 -04:00
tcp_minisocks.c tcp: consolidate SYNACK RTT sampling 2013-07-22 17:53:42 -07:00
tcp_offload.c net: tcp: move GRO/GSO functions to tcp_offload 2013-06-07 14:39:05 -07:00
tcp_output.c tcp: TSQ can use a dynamic limit 2013-09-30 20:41:57 -07:00
tcp_probe.c tcp: Change return value of tcp_rcv_established() 2013-09-04 00:27:28 -04:00
tcp_scalable.c tcp: mark tcp_congestion_ops read_mostly 2011-03-10 00:40:17 -08:00
tcp_timer.c tcp: refactor F-RTO 2013-03-21 11:47:50 -04:00
tcp_vegas.c tcp: mark tcp_congestion_ops read_mostly 2011-03-10 00:40:17 -08:00
tcp_vegas.h
tcp_veno.c tcp: mark tcp_congestion_ops read_mostly 2011-03-10 00:40:17 -08:00
tcp_westwood.c tcp: refactor F-RTO 2013-03-21 11:47:50 -04:00
tcp_yeah.c Fix common misspellings 2011-03-31 11:26:23 -03:00
tunnel4.c net: Convert printks to pr_<level> 2012-03-11 23:42:51 -07:00
udp.c udp: ipv4: Add udp early demux 2013-10-08 16:27:33 -04:00
udp_diag.c netlink: rename ssk to sk in struct netlink_skb_params 2013-04-19 14:57:56 -04:00
udp_impl.h ipv4: fix checkpatch errors 2012-04-15 12:37:19 -04:00
udp_offload.c net: udp4: move GSO functions to udp_offload 2013-06-12 00:47:25 -07:00
udplite.c net: ipv4: Standardize prefixes for message logging 2012-03-12 17:05:21 -07:00
xfrm4_input.c net: Add skb_unclone() helper function. 2013-02-15 15:10:37 -05:00
xfrm4_mode_beet.c ipsec: be careful of non existing mac headers 2012-02-23 16:50:45 -05:00
xfrm4_mode_transport.c
xfrm4_mode_tunnel.c Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/klassert/ipsec-next 2013-09-30 15:24:57 -04:00
xfrm4_output.c xfrm: revert ipv4 mtu determination to dst_mtu 2013-08-26 12:40:53 +02:00
xfrm4_policy.c xfrm: make gc_thresh configurable in all namespaces 2013-02-06 11:36:29 +01:00
xfrm4_state.c xfrm: make local error reporting more robust 2013-08-14 13:07:12 +02:00
xfrm4_tunnel.c sit: add IPv4 over IPv4 support 2013-05-31 17:19:05 -07:00