When some objects are allocated by one CPU but freed by another CPU we can
consume lot of cycles doing divides in obj_to_index().
(Typical load on a dual processor machine where network interrupts are
handled by one particular CPU (allocating skbufs), and the other CPU is
running the application (consuming and freeing skbufs))
Here on one production server (dual-core AMD Opteron 285), I noticed this
divide took 1.20 % of CPU_CLK_UNHALTED events in kernel. But Opteron are
quite modern cpus and the divide is much more expensive on oldest
architectures :
On a 200 MHz sparcv9 machine, the division takes 64 cycles instead of 1
cycle for a multiply.
Doing some math, we can use a reciprocal multiplication instead of a divide.
If we want to compute V = (A / B) (A and B being u32 quantities)
we can instead use :
V = ((u64)A * RECIPROCAL(B)) >> 32 ;
where RECIPROCAL(B) is precalculated to ((1LL << 32) + (B - 1)) / B
Note :
I wrote pure C code for clarity. gcc output for i386 is not optimal but
acceptable :
mull 0x14(%ebx)
mov %edx,%eax // part of the >> 32
xor %edx,%edx // useless
mov %eax,(%esp) // could be avoided
mov %edx,0x4(%esp) // useless
mov (%esp),%ebx
[akpm@osdl.org: small cleanups]
Signed-off-by: Eric Dumazet <dada1@cosmosbay.com>
Cc: Christoph Lameter <clameter@sgi.com>
Cc: David Miller <davem@davemloft.net>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Elaborate the API for calling cpuset_zone_allowed(), so that users have to
explicitly choose between the two variants:
cpuset_zone_allowed_hardwall()
cpuset_zone_allowed_softwall()
Until now, whether or not you got the hardwall flavor depended solely on
whether or not you or'd in the __GFP_HARDWALL gfp flag to the gfp_mask
argument.
If you didn't specify __GFP_HARDWALL, you implicitly got the softwall
version.
Unfortunately, this meant that users would end up with the softwall version
without thinking about it. Since only the softwall version might sleep,
this led to bugs with possible sleeping in interrupt context on more than
one occassion.
The hardwall version requires that the current tasks mems_allowed allows
the node of the specified zone (or that you're in interrupt or that
__GFP_THISNODE is set or that you're on a one cpuset system.)
The softwall version, depending on the gfp_mask, might allow a node if it
was allowed in the nearest enclusing cpuset marked mem_exclusive (which
requires taking the cpuset lock 'callback_mutex' to evaluate.)
This patch removes the cpuset_zone_allowed() call, and forces the caller to
explicitly choose between the hardwall and the softwall case.
If the caller wants the gfp_mask to determine this choice, they should (1)
be sure they can sleep or that __GFP_HARDWALL is set, and (2) invoke the
cpuset_zone_allowed_softwall() routine.
This adds another 100 or 200 bytes to the kernel text space, due to the few
lines of nearly duplicate code at the top of both cpuset_zone_allowed_*
routines. It should save a few instructions executed for the calls that
turned into calls of cpuset_zone_allowed_hardwall, thanks to not having to
set (before the call) then check (within the call) the __GFP_HARDWALL flag.
For the most critical call, from get_page_from_freelist(), the same
instructions are executed as before -- the old cpuset_zone_allowed()
routine it used to call is the same code as the
cpuset_zone_allowed_softwall() routine that it calls now.
Not a perfect win, but seems worth it, to reduce this chance of hitting a
sleeping with irq off complaint again.
Signed-off-by: Paul Jackson <pj@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
More cleanups for slab.h
1. Remove tabs from weird locations as suggested by Pekka
2. Drop the check for NUMA and SLAB_DEBUG from the fallback section
as suggested by Pekka.
3. Uses static inline for the fallback defs as also suggested by Pekka.
4. Make kmem_ptr_valid take a const * argument.
5. Separate the NUMA fallback definitions from the kmalloc_track fallback
definitions.
Signed-off-by: Christoph Lameter <clameter@sgi.com>
Cc: Pekka Enberg <penberg@cs.helsinki.fi>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This is a response to an earlier discussion on linux-mm about splitting
slab.h components per allocator. Patch is against 2.6.19-git11. See
http://marc.theaimsgroup.com/?l=linux-mm&m=116469577431008&w=2
This patch cleans up the slab header definitions. We define the common
functions of slob and slab in slab.h and put the extra definitions needed
for slab's kmalloc implementations in <linux/slab_def.h>. In order to get
a greater set of common functions we add several empty functions to slob.c
and also rename slob's kmalloc to __kmalloc.
Slob does not need any special definitions since we introduce a fallback
case. If there is no need for a slab implementation to provide its own
kmalloc mess^H^H^Hacros then we simply fall back to __kmalloc functions.
That is sufficient for SLOB.
Sort the function in slab.h according to their functionality. First the
functions operating on struct kmem_cache * then the kmalloc related
functions followed by special debug and fallback definitions.
Also redo a lot of comments.
Signed-off-by: Christoph Lameter <clameter@sgi.com>?
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Fields of struct pipe_buf_operations have not a precise layout (ie not
optimized to fit cache lines nor reduce cache line ping pongs)
The bufs[] array is *large* and is placed near the beginning of the
structure, so all following fields have a large offset. This is
unfortunate because many archs have smaller instructions when using small
offsets relative to a base register. On x86 for example, 7 bits offsets
have smaller instruction lengths.
Moving bufs[] at the end of pipe_buf_operations permits all fields to have
small offsets, and reduce text size, and icache pressure.
# size vmlinux.pre vmlinux
text data bss dec hex filename
3268989 664356 492196 4425541 438745 vmlinux.pre
3268765 664356 492196 4425317 438665 vmlinux
So this patch reduces text size by 224 bytes on my x86_64 machine. Similar
results on ia32.
Signed-off-by: Eric Dumazet <dada1@cosmosbay.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This reverts commit 373beb35cd.
No one is using this identifier yet. The purpose of this identifier is to
export nsproxy to user space which is wrong. nsproxy is an internal
implementation optimization, which should keep our fork times from getting
slower as we increase the number of global namespaces you don't have to
share.
Adding a global identifier like this is inappropriate because it makes
namespaces inherently non-recursive, greatly limiting what we can do with
them in the future.
Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
Cc: Cedric Le Goater <clg@fr.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
- pipe/splice should use const pipe_buf_operations and file_operations
- struct pipe_inode_info has an unused field "start" : get rid of it.
Signed-off-by: Eric Dumazet <dada1@cosmosbay.com>
Cc: Jens Axboe <jens.axboe@oracle.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
* 'release' of git://git.kernel.org/pub/scm/linux/kernel/git/aegl/linux-2.6:
[IA64] kprobe clears qp bits for special instructions
[IA64] enable trap code on slot 1
[IA64] Take defensive stance on ia64_pal_get_brand_info()
[IA64] fix possible XPC deadlock when disconnecting
[IA64] - Reduce overhead of FP exception logging messages
[IA64] fix arch/ia64/mm/contig.c:235: warning: unused variable `nid'
[IA64] s/termios/ktermios/ in simserial.c
[IA64] kexec/kdump: tidy up declaration of relocate_new_kernel_t
[IA64] Kexec/Kdump: honour non-zero crashkernel offset.
[IA64] CONFIG_KEXEC/CONFIG_CRASH_DUMP permutations
[IA64] Do not call SN_SAL_SET_CPU_NUMBER twice on cpu 0
* 'for-linus' of master.kernel.org:/pub/scm/linux/kernel/git/roland/infiniband:
IPoIB: Make sure struct ipoib_neigh.queue is always initialized
IB/iser: Use the new verbs DMA mapping functions
IB/srp: Use new verbs IB DMA mapping functions
IPoIB: Use the new verbs DMA mapping functions
IB/core: Use the new verbs DMA mapping functions
IB/ipath: Implement new verbs DMA mapping functions
IB: Add DMA mapping functions to allow device drivers to interpose
RDMA/cma: Export rdma cm interface to userspace
RDMA/cma: Add support for RDMA_PS_UDP
RDMA/cma: Allow early transition to RTS to handle lost CM messages
RDMA/cma: Report connect info with connect events
RDMA/cma: Remove unneeded qp_type parameter from rdma_cm
IB/ipath: Fix IRQ for PCI Express HCAs
RDMA/amso1100: Fix memory leak in c2_qp_modify()
IB/iser: Remove unused "write-only" variables
IB/ipath: Remove unused "write-only" variables
IB/fmr: ib_flush_fmr_pool() may wait too long
* git://git.kernel.org/pub/scm/linux/kernel/git/bunk/trivial:
Fix inotify maintainers entry
Fix typo in new debug options.
Jon needs a new shift key.
fs: Convert kmalloc() + memset() to kzalloc() in fs/.
configfs.h: Remove dead macro definitions.
kconfig: Standardize "depends" -> "depends on" in Kconfig files
e100: replace kmalloc with kcalloc
um: replace kmalloc+memset with kzalloc
fix typo in net/ipv4/ip_fragment.c
include/linux/compiler.h: reject gcc 3 < gcc 3.2
Kconfig: fix spelling error in config KALLSYMS help text
Remove duplicate "have to" in comment
Fix small typo in drivers/serial/icom.c
Use consistent casing in help message
EXT{2,3,4}_FS: remove outdated part of the help text
Support for Core CPUs was broken in two ways in speedstep-lib: for x86_64,
we missed a MSR definition; for both x86_64 and i386, the FSB calculation
was wrong by four (it's a quad-pumped bus). Also increase the accuracy
of the calculation.
Signed-off-by: Dominik Brodowski <linux@dominikbrodowski.net>
Signed-off-by: Dave Jones <davej@redhat.com>
The QLogic InfiniPath HCAs use programmed I/O instead of HW DMA.
This patch allows a verbs device driver to interpose on DMA mapping
function calls in order to avoid relying on bus_to_virt() and
phys_to_virt() to undo the mappings created by dma_map_single(),
dma_map_sg(), etc.
Signed-off-by: Ralph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
Because slot 1 of one instr bundle crosses border of two consecutive
8-bytes, kprobe on slot 1 is disabled. This patch enables kprobe on
slot1, it only replaces higher 8-bytes of the instruction bundle and
changes the exception code to ignore the low 12 bits of the break
number (which is across the border in the lower 8-bytes of the bundle).
For those instructions which must execute regardless qp bits,
kprobe on slot 1 is still disabled.
Signed-off-by: bibo,mao <bibo.mao@intel.com>
Signed-off-by: Tony Luck <tony.luck@intel.com>
Export the rdma cm interfaces to userspace via a misc device.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
Allow the use of UD QPs through the rdma_cm, in order to provide
address translation services for resolving IB addresses for datagram
messages using SIDR.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
During connection establishment, the passive side of a connection can
receive messages from the active side before the connection event has
been delivered to the user. Allow the passive side to send messages
in response to received data before the event is delivered. To handle
the case where the connection messages are lost, a new rdma_notify()
function is added that users may invoke to force a connection into the
established state.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
Connection information was never given to the recipient of a
connection request or reply message. Only the event was delivered.
Report the connection data with the event to allows user to
reject the connection based on the requested parameters, or adjust
their resources to match the request.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
The qp_type parameter into the rdma_cm is unneeded, and can be
misleading. The QP type should be determined from the port space.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
This patch eliminates a potential deadlock that is possible when XPC
disconnects a channel to a partition that has gone down. This deadlock will
occur if at least one of the kthreads created by XPC for the purpose of making
callouts to the channel's registerer is detained in the registerer and will
not be returning back to XPC until some registerer request occurs on the now
downed partition. The potential for a deadlock is removed by ensuring that
there always is a kthread available to make the channel disconnecting callout
to the registerer.
Signed-off-by: Dean Nelson <dcn@sgi.com>
Signed-off-by: Tony Luck <tony.luck@intel.com>
Delete the __ATTR-related macro definitions since these are now
defined in include/linux/sysfs.h.
Signed-off-by: Robert P. J. Day <rpjday@mindspring.com>
Signed-off-by: Adrian Bunk <bunk@stusta.de>
The kernel doesn't compile with GCC <3.2, do not allow it to succeed if GCC
3.0.x or 3.1.x are used.
Signed-off-by: Alistair John Strachan <s0348365@sms.ed.ac.uk>
Signed-off-by: Adrian Bunk <bunk@stusta.de>
Introduced in commit 7cc13edc13.
Signed-off-by: Rolf Eike Beer <eike-kernel@sf-tec.de>
Acked-by: Eric W. Biederman <ebiederm@xmission.com>
Signed-off-by: Adrian Bunk <bunk@stusta.de>
* 'i2c-for-linus' of git://jdelvare.pck.nerim.net/jdelvare-2.6:
i2c: Fix OMAP clock prescaler to match the comment
i2c: Refactor a kfree in i2c-dev
i2c: Fix return value check in i2c-dev
i2c: Enable PEC on more i2c-i801 devices
i2c: Discard the i2c algo del_bus wrappers
i2c: New ARM Versatile/Realview bus driver
i2c: fix broken ds1337 initialization
i2c: i2c-i801 documentation update
i2c: Use the __ATTR macro where possible
i2c: Whitespace cleanups
i2c: Use put_user instead of copy_to_user where possible
i2c: New Atmel AT91 bus driver
i2c: Add support for nested i2c bus locking
i2c: Cleanups to the i2c-nforce2 bus driver
i2c: Add request/release_mem_region to i2c-ibm_iic bus driver
i2c: New Philips PNX bus driver
i2c: Delete the broken i2c-ite bus driver
i2c: Update the list of driver IDs
i2c: Fix documentation typos
seqlock_init() needs to use spin_lock_init() for dynamic locks, so that
lockdep is notified about the presence of a new lock.
(this is a fallout of the recent networking merge, which started using
the so-far unused seqlock_init() API.)
This fix solves the following lockdep-internal warning on current -git:
INFO: trying to register non-static key.
the code is fine but needs lockdep annotation.
turning off the locking correctness validator.
__lock_acquire+0x10c/0x9f9
lock_acquire+0x56/0x72
_spin_lock+0x35/0x42
neigh_destroy+0x9d/0x12e
neigh_periodic_timer+0x10a/0x15c
run_timer_softirq+0x126/0x18e
__do_softirq+0x6b/0xe6
do_softirq+0x64/0xd2
ksoftirqd+0x82/0x138
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
While working on bidi support at struct request level
I have found that blk_queue_activity_fn is actually never used.
The only user is in ide-probe.c with this code:
/* enable led activity for disk drives only */
if (drive->media == ide_disk && hwif->led_act)
blk_queue_activity_fn(q, hwif->led_act, drive);
And led_act is never initialized anywhere.
(Looking back at older kernels it was used in the PPC arch, but was removed around 2.6.18)
Unless it is all for future use off course.
(this patch is against linux-2.6-block.git as off 2006/12/4)
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
* 'for-linus' of git://www.atmel.no/~hskinnemoen/linux/kernel/avr32:
[AVR32] Add missing #include <linux/param.h> to delay.c
[AVR32] Pass dev parameter to dma_cache_sync()
[AVR32] Implement intc_get_pending()
[AVR32] Don't include <asm/delay.h>
[AVR32] Put the chip in "stop" mode when halting the system
[AVR32] Set flow handler for external interrupts
[AVR32] Remove unused file
[AVR32] Remove mii_phy_addr and eth_addr from eth_platform_data
[AVR32] Move ethernet tag parsing to board-specific code
[AVR32] Add macb1 platform_device
[AVR32] Portmux API update
* git://git.kernel.org/pub/scm/linux/kernel/git/paulus/powerpc: (36 commits)
[POWERPC] Generic BUG for powerpc
[PPC] Fix compile failure do to introduction of PHY_POLL
[POWERPC] Only export __mtdcr/__mfdcr if CONFIG_PPC_DCR is set
[POWERPC] Remove old dcr.S
[POWERPC] Fix SPU coredump code for max_fdset removal
[POWERPC] Fix irq routing on some 32-bit PowerMacs
[POWERPC] ps3: Add vuart support
[POWERPC] Support ibm,dynamic-reconfiguration-memory nodes
[POWERPC] dont allow pSeries_probe to succeed without initialising MMU
[POWERPC] micro optimise pSeries_probe
[POWERPC] Add SPURR SPR to sysfs
[POWERPC] Add DSCR SPR to sysfs
[POWERPC] Fix 440SPe CPU table entry
[POWERPC] Add support for FP emulation for the e300c2 core
[POWERPC] of_device_register: propagate device_create_file return code
[POWERPC] Fix mmap of PCI resource with hack for X
[POWERPC] iSeries: head_64.o needs to depend on lparmap.s
[POWERPC] cbe_thermal: Fix initialization of sysfs attribute_group
[POWERPC] Remove QE header files from lite5200.c
[POWERPC] of_platform_make_bus_id(): make `magic' int
...
Previously we haven't been doing anything with verbose BUG() reporting,
and we've been relying on the oops path for handling BUG()'s, which is
rather sub-optimal.
This switches BUG handling to use a fixed trapa vector (#0x3e) where we
construct a small bug frame post trapa instruction to get the context
right. This also makes it trivial to wire up a DIE_BUG for the atomic
die chain, which we couldn't really do before.
Signed-off-by: Paul Mundt <lethal@linux-sh.org>
We have a few different ways to do the atomic operations, so split
them out in to different headers rather than bloating atomic.h.
Kernelspace gUSA will take this up to a third implementation.
Signed-off-by: Paul Mundt <lethal@linux-sh.org>
In the 64-bit PTE case there's no point in restricting the encoding
to the low bits of the PTE, we can instead bump all of this up to
the high 32 bits and extend PTE_FILE_MAX_BITS to 32, adopting the
same convention used by x86 PAE.
There's a minor discrepency between the number of bits used for the
swap type encoding between 32 and 64-bit PTEs, but this is unlikely
to cause any problem given the extended offset.
Signed-off-by: Paul Mundt <lethal@linux-sh.org>
Fix foobar in 15b1c0e822 and
e8cc49bb0f patch series.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
That accumulated over the last months hackaton, shame on me for not
using git-apply whitespace helping hand, will do that from now on.
Signed-off-by: Arnaldo Carvalho de Melo <acme@mandriva.com>
This patch
* resolves a bug where packets smaller than 32/64 bytes resulted in sending rates of 0
* supports all sending rates from 1/64 bytes/second up to 4Gbyte/second
* simplifies the present overflow problems in calculations
Current sending rate X and the cached value X_recv of the receiver-estimated
sending rate are both scaled by 64 (2^6) in order to
* cope with low sending rates (minimally 1 byte/second)
* allow upgrading to use a packets-per-second implementation of CCID 3
* avoid calculation errors due to integer arithmetic cut-off
The patch implements a revised strategy from
http://www.mail-archive.com/dccp@vger.kernel.org/msg01040.html
The only difference with regard to that strategy is that t_ipi is already
used in the calculation of the nofeedback timeout, which saves one division.
Signed-off-by: Gerrit Renker <gerrit@erg.abdn.ac.uk>
Acked-by: Ian McDonald <ian.mcdonald@jandi.co.nz>
Signed-off-by: Arnaldo Carvalho de Melo <acme@mandriva.com>
Make powerpc's __ilog2_u64() take a 64-bit argument.
Signed-off-by: David Howells <dhowells@redhat.com>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
We should not initialize rootfs before all the core initializers have
run. So do it as a separate stage just before starting the regular
driver initializers.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
As reported by Andy Whitcroft, at least the SLES9 initrd build process
depends on getting the kernel version from the kernel binary. It does
that by simply trawling the binary and looking for the signature of the
"linux_banner" string (the string "Linux version " to be exact. Which
is really broken in itself, but whatever..)
That got broken when the string was changed to allow /proc/version to
change the UTS release information dynamically, and "get_kernel_version"
thus returned "%s" (see commit a2ee8649ba6d71416712e798276bf7c40b64e6e5:
"[PATCH] Fix linux banner utsname information").
This just restores "linux_banner" as a static string, which should fix
the version finding. And /proc/version simply uses a different string.
To avoid wasting even that miniscule amount of memory, the early boot
string should really be marked __initdata, but that just causes the same
bug in SLES9 to re-appear, since it will then find other occurrences of
"Linux version " first.
Cc: Andy Whitcroft <apw@shadowen.org>
Acked-by: Herbert Poetzl <herbert@13thfloor.at>
Cc: Andi Kleen <ak@suse.de>
Cc: Andrew Morton <akpm@osdl.org>
Cc: Steve Fox <drfickle@us.ibm.com>
Acked-by: Olaf Hering <olaf@aepfle.de>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This makes powerpc use the generic BUG machinery. The biggest reports the
function name, since it is redundant with kallsyms, and not needed in general.
There is an overall reduction of code, since module_32/64 duplicated several
functions.
Unfortunately there's no way to tell gcc that BUG won't return, so the BUG
macro includes a goto loop. This will generate a real jmp instruction, which
is never used.
[akpm@osdl.org: build fix]
[paulus@samba.org: remove infinite loop in BUG_ON]
Signed-off-by: Jeremy Fitzhardinge <jeremy@goop.org>
Cc: Andi Kleen <ak@muc.de>
Cc: Hugh Dickens <hugh@veritas.com>
Cc: Michael Ellerman <michael@ellerman.id.au>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Paul Mackerras <paulus@samba.org>
Eric Dumazet