linux/net
Mike Stroyan 18955cfcb2 [IPV4] tcp/route: Another look at hash table sizes
The tcp_ehash hash table gets too big on systems with really big memory.
It is worse on systems with pages larger than 4KB.  It wastes memory that
could be better used.  It also makes the netstat command slow because reading
/proc/net/tcp and /proc/net/tcp6 needs to go through the full hash table.

  The default value should not be larger for larger page sizes.  It seems
that the effect of page size is an unintended error dating back a long
time.  I also wonder if the default value really should be a larger
fraction of memory for systems with more memory.  While systems with
really big ram can afford more space for hash tables, it is not clear to
me that they benefit from increasing the allocation ratio for this table.

  The amount of memory allocated is determined by net/ipv4/tcp.c:tcp_init and
mm/page_alloc.c:alloc_large_system_hash.

tcp_init calls alloc_large_system_hash passing parameters-
    bucketsize=sizeof(struct tcp_ehash_bucket)
    numentries=thash_entries
    scale=(num_physpages >= 128 * 1024) ? (25-PAGE_SHIFT) : (27-PAGE_SHIFT)
    limit=0

On i386, PAGE_SHIFT is 12 for a page size of 4K
On ia64, PAGE_SHIFT defaults to 14 for a page size of 16K

The num_physpages test above makes the allocation take a larger fraction
of the total memory on systems with larger memory.  The threshold size
for a i386 system is 512MB.  For an ia64 system with 16KB pages the
threshold is 2GB.

For smaller memory systems-
On i386, scale = (27 - 12) = 15
On ia64, scale = (27 - 14) = 13
For larger memory systems-
On i386, scale = (25 - 12) = 13
On ia64, scale = (25 - 14) = 11

  For the rest of this discussion, I'll just track the larger memory case.

  The default behavior has numentries=thash_entries=0, so the allocated
size is determined by either scale or by the default limit of 1/16 of
total memory.

In alloc_large_system_hash-
|	numentries = (flags & HASH_HIGHMEM) ? nr_all_pages : nr_kernel_pages;
|	numentries += (1UL << (20 - PAGE_SHIFT)) - 1;
|	numentries >>= 20 - PAGE_SHIFT;
|	numentries <<= 20 - PAGE_SHIFT;

  At this point, numentries is pages for all of memory, rounded up to the
nearest megabyte boundary.

|	/* limit to 1 bucket per 2^scale bytes of low memory */
|	if (scale > PAGE_SHIFT)
|		numentries >>= (scale - PAGE_SHIFT);
|	else
|		numentries <<= (PAGE_SHIFT - scale);

On i386, numentries >>= (13 - 12), so numentries is 1/8196 of
bytes of total memory.
On ia64, numentries <<= (14 - 11), so numentries is 1/2048 of
bytes of total memory.

|        log2qty = long_log2(numentries);
|
|        do {
|                size = bucketsize << log2qty;

bucketsize is 16, so size is 16 times numentries, rounded
down to a power of two.

On i386, size is 1/512 of bytes of total memory.
On ia64, size is 1/128 of bytes of total memory.

For smaller systems the results are
On i386, size is 1/2048 of bytes of total memory.
On ia64, size is 1/512 of bytes of total memory.

  The large page effect can be removed by just replacing
the use of PAGE_SHIFT with a constant of 12 in the calls to
alloc_large_system_hash.  That makes them more like the other uses of
that function from fs/inode.c and fs/dcache.c

Signed-off-by: David S. Miller <davem@davemloft.net>
2005-11-29 16:12:55 -08:00
..
802 [NET]: kfree cleanup 2005-11-08 09:41:34 -08:00
8021q [8021Q]: Add endian annotations. 2005-09-19 15:41:28 -07:00
appletalk [APPLETALK]: Fix broadcast bug. 2005-09-27 16:11:29 -07:00
atm [PATCH] gfp flags annotations - part 1 2005-10-08 15:00:57 -07:00
ax25 [NET]: kfree cleanup 2005-11-08 09:41:34 -08:00
bluetooth [Bluetooth]: Remove the usage of /proc completely 2005-11-08 09:57:38 -08:00
bridge [BRIDGE]: recompute features when adding a new device 2005-11-23 19:04:08 -08:00
core [NET]: Reject socket filter if division by constant zero is attempted. 2005-11-20 13:41:34 -08:00
dccp [DCCP]: Add missing no_policy flag to struct net_protocol 2005-11-20 21:16:13 -08:00
decnet [DECNET]: fix SIGPIPE 2005-11-11 12:04:28 -08:00
econet [IPV4]: Replace __in_dev_get with __in_dev_get_rcu/rtnl 2005-10-03 14:35:55 -07:00
ethernet [NET]: kfree cleanup 2005-11-08 09:41:34 -08:00
ieee80211 ieee80211: cleanup crypto list handling, other minor cleanups. 2005-11-09 01:01:04 -05:00
ipv4 [IPV4] tcp/route: Another look at hash table sizes 2005-11-29 16:12:55 -08:00
ipv6 [IPV6]: Implement appropriate dummy rule 4 in ipv6_dev_get_saddr(). 2005-11-28 22:27:11 -08:00
ipx [NET]: Fix sparse warnings 2005-08-29 16:01:32 -07:00
irda [NET]: kfree cleanup 2005-11-08 09:41:34 -08:00
key [PATCH] gfp flags annotations - part 1 2005-10-08 15:00:57 -07:00
lapb [NET]: Kill skb->list 2005-08-29 15:31:14 -07:00
llc [LLC]: Fix compiler warnings introduced by TX window scaling changes. 2005-11-17 15:17:42 -08:00
netfilter [NETFILTER]: Fix nf_conntrack compilation with CONFIG_NETFILTER_DEBUG 2005-11-16 23:14:19 -08:00
netlink [NETLINK]: Use tgid instead of pid for nlmsg_pid 2005-11-22 14:41:50 -08:00
netrom [AX.25]: Fix packet socket crash 2005-10-05 12:16:04 -07:00
packet [NET]: Fix packet timestamping. 2005-10-03 13:57:23 -07:00
rose [NET]: kfree cleanup 2005-11-08 09:41:34 -08:00
rxrpc [NET]: Detect hardware rx checksum faults correctly 2005-11-10 13:01:24 -08:00
sched [PKT_SCHED]: sch_netem: correctly order packets to be sent simultaneously 2005-11-20 13:41:05 -08:00
sctp [SCTP]: Include ulpevents in socket receive buffer accounting. 2005-11-11 16:08:24 -08:00
sunrpc SUNRPC: Funny looking code in __rpc_purge_upcall 2005-11-25 17:11:30 -05:00
unix [PATCH] add a vfs_permission helper 2005-11-09 07:55:58 -08:00
wanrouter [NET]: kfree cleanup 2005-11-08 09:41:34 -08:00
x25 [TCP]: Move the tcp sock states to net/tcp_states.h 2005-08-29 15:41:54 -07:00
xfrm [XFRM]: Use generic netlink receive queue processor 2005-11-10 02:26:40 +01:00
Kconfig [NETFILTER] move nfnetlink options to right location in kconfig menu 2005-09-17 00:41:21 -07:00
Makefile [NETFILTER]: link 'netfilter' before ipv4 2005-11-14 15:25:59 -08:00
TUNABLE Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
compat.c [PATCH] Fix 32bit sendmsg() flaw 2005-09-08 08:14:11 -07:00
nonet.c Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
socket.c [NET]: Fix module reference counts for loadable protocol modules 2005-09-27 15:23:38 -07:00
sysctl_net.c [NET]: Fix "sysctl_net.c:36: error: 'core_table' undeclared here" 2005-10-03 14:16:34 -07:00