2005-04-17 06:20:36 +08:00
|
|
|
#ifndef _LINUX__INIT_TASK_H
|
|
|
|
|
#define _LINUX__INIT_TASK_H
|
|
|
|
|
|
2005-09-10 04:04:13 +08:00
|
|
|
#include <linux/rcupdate.h>
|
2006-07-03 15:24:42 +08:00
|
|
|
#include <linux/irqflags.h>
|
2006-10-02 17:18:14 +08:00
|
|
|
#include <linux/utsname.h>
|
[PATCH] lockdep: core
Do 'make oldconfig' and accept all the defaults for new config options -
reboot into the kernel and if everything goes well it should boot up fine and
you should have /proc/lockdep and /proc/lockdep_stats files.
Typically if the lock validator finds some problem it will print out
voluminous debug output that begins with "BUG: ..." and which syslog output
can be used by kernel developers to figure out the precise locking scenario.
What does the lock validator do? It "observes" and maps all locking rules as
they occur dynamically (as triggered by the kernel's natural use of spinlocks,
rwlocks, mutexes and rwsems). Whenever the lock validator subsystem detects a
new locking scenario, it validates this new rule against the existing set of
rules. If this new rule is consistent with the existing set of rules then the
new rule is added transparently and the kernel continues as normal. If the
new rule could create a deadlock scenario then this condition is printed out.
When determining validity of locking, all possible "deadlock scenarios" are
considered: assuming arbitrary number of CPUs, arbitrary irq context and task
context constellations, running arbitrary combinations of all the existing
locking scenarios. In a typical system this means millions of separate
scenarios. This is why we call it a "locking correctness" validator - for all
rules that are observed the lock validator proves it with mathematical
certainty that a deadlock could not occur (assuming that the lock validator
implementation itself is correct and its internal data structures are not
corrupted by some other kernel subsystem). [see more details and conditionals
of this statement in include/linux/lockdep.h and
Documentation/lockdep-design.txt]
Furthermore, this "all possible scenarios" property of the validator also
enables the finding of complex, highly unlikely multi-CPU multi-context races
via single single-context rules, increasing the likelyhood of finding bugs
drastically. In practical terms: the lock validator already found a bug in
the upstream kernel that could only occur on systems with 3 or more CPUs, and
which needed 3 very unlikely code sequences to occur at once on the 3 CPUs.
That bug was found and reported on a single-CPU system (!). So in essence a
race will be found "piecemail-wise", triggering all the necessary components
for the race, without having to reproduce the race scenario itself! In its
short existence the lock validator found and reported many bugs before they
actually caused a real deadlock.
To further increase the efficiency of the validator, the mapping is not per
"lock instance", but per "lock-class". For example, all struct inode objects
in the kernel have inode->inotify_mutex. If there are 10,000 inodes cached,
then there are 10,000 lock objects. But ->inotify_mutex is a single "lock
type", and all locking activities that occur against ->inotify_mutex are
"unified" into this single lock-class. The advantage of the lock-class
approach is that all historical ->inotify_mutex uses are mapped into a single
(and as narrow as possible) set of locking rules - regardless of how many
different tasks or inode structures it took to build this set of rules. The
set of rules persist during the lifetime of the kernel.
To see the rough magnitude of checking that the lock validator does, here's a
portion of /proc/lockdep_stats, fresh after bootup:
lock-classes: 694 [max: 2048]
direct dependencies: 1598 [max: 8192]
indirect dependencies: 17896
all direct dependencies: 16206
dependency chains: 1910 [max: 8192]
in-hardirq chains: 17
in-softirq chains: 105
in-process chains: 1065
stack-trace entries: 38761 [max: 131072]
combined max dependencies: 2033928
hardirq-safe locks: 24
hardirq-unsafe locks: 176
softirq-safe locks: 53
softirq-unsafe locks: 137
irq-safe locks: 59
irq-unsafe locks: 176
The lock validator has observed 1598 actual single-thread locking patterns,
and has validated all possible 2033928 distinct locking scenarios.
More details about the design of the lock validator can be found in
Documentation/lockdep-design.txt, which can also found at:
http://redhat.com/~mingo/lockdep-patches/lockdep-design.txt
[bunk@stusta.de: cleanups]
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Signed-off-by: Adrian Bunk <bunk@stusta.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-07-03 15:24:50 +08:00
|
|
|
#include <linux/lockdep.h>
|
2009-03-26 08:55:00 +08:00
|
|
|
#include <linux/ftrace.h>
|
2006-10-02 17:18:20 +08:00
|
|
|
#include <linux/ipc.h>
|
2006-12-08 18:37:59 +08:00
|
|
|
#include <linux/pid_namespace.h>
|
2007-07-16 14:40:59 +08:00
|
|
|
#include <linux/user_namespace.h>
|
capabilities: implement per-process securebits
Filesystem capability support makes it possible to do away with (set)uid-0
based privilege and use capabilities instead. That is, with filesystem
support for capabilities but without this present patch, it is (conceptually)
possible to manage a system with capabilities alone and never need to obtain
privilege via (set)uid-0.
Of course, conceptually isn't quite the same as currently possible since few
user applications, certainly not enough to run a viable system, are currently
prepared to leverage capabilities to exercise privilege. Further, many
applications exist that may never get upgraded in this way, and the kernel
will continue to want to support their setuid-0 base privilege needs.
Where pure-capability applications evolve and replace setuid-0 binaries, it is
desirable that there be a mechanisms by which they can contain their
privilege. In addition to leveraging the per-process bounding and inheritable
sets, this should include suppressing the privilege of the uid-0 superuser
from the process' tree of children.
The feature added by this patch can be leveraged to suppress the privilege
associated with (set)uid-0. This suppression requires CAP_SETPCAP to
initiate, and only immediately affects the 'current' process (it is inherited
through fork()/exec()). This reimplementation differs significantly from the
historical support for securebits which was system-wide, unwieldy and which
has ultimately withered to a dead relic in the source of the modern kernel.
With this patch applied a process, that is capable(CAP_SETPCAP), can now drop
all legacy privilege (through uid=0) for itself and all subsequently
fork()'d/exec()'d children with:
prctl(PR_SET_SECUREBITS, 0x2f);
This patch represents a no-op unless CONFIG_SECURITY_FILE_CAPABILITIES is
enabled at configure time.
[akpm@linux-foundation.org: fix uninitialised var warning]
[serue@us.ibm.com: capabilities: use cap_task_prctl when !CONFIG_SECURITY]
Signed-off-by: Andrew G. Morgan <morgan@kernel.org>
Acked-by: Serge Hallyn <serue@us.ibm.com>
Reviewed-by: James Morris <jmorris@namei.org>
Cc: Stephen Smalley <sds@tycho.nsa.gov>
Cc: Paul Moore <paul.moore@hp.com>
Signed-off-by: Serge E. Hallyn <serue@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-04-28 17:13:40 +08:00
|
|
|
#include <linux/securebits.h>
|
2007-09-12 17:55:17 +08:00
|
|
|
#include <net/net_namespace.h>
|
2005-04-17 06:20:36 +08:00
|
|
|
|
2008-05-09 06:19:16 +08:00
|
|
|
extern struct files_struct init_files;
|
2008-12-26 13:35:37 +08:00
|
|
|
extern struct fs_struct init_fs;
|
2005-04-17 06:20:36 +08:00
|
|
|
|
2006-12-08 18:37:55 +08:00
|
|
|
#define INIT_SIGNALS(sig) { \
|
|
|
|
|
.count = ATOMIC_INIT(1), \
|
2005-04-17 06:20:36 +08:00
|
|
|
.wait_chldexit = __WAIT_QUEUE_HEAD_INITIALIZER(sig.wait_chldexit),\
|
2006-12-08 18:37:55 +08:00
|
|
|
.shared_pending = { \
|
2005-04-17 06:20:36 +08:00
|
|
|
.list = LIST_HEAD_INIT(sig.shared_pending.list), \
|
2006-12-08 18:37:55 +08:00
|
|
|
.signal = {{0}}}, \
|
2005-04-17 06:20:36 +08:00
|
|
|
.posix_timers = LIST_HEAD_INIT(sig.posix_timers), \
|
|
|
|
|
.cpu_timers = INIT_CPU_TIMERS(sig.cpu_timers), \
|
|
|
|
|
.rlim = INIT_RLIMITS, \
|
2009-02-05 19:24:16 +08:00
|
|
|
.cputimer = { \
|
|
|
|
|
.cputime = INIT_CPUTIME, \
|
|
|
|
|
.running = 0, \
|
|
|
|
|
.lock = __SPIN_LOCK_UNLOCKED(sig.cputimer.lock), \
|
|
|
|
|
}, \
|
2005-04-17 06:20:36 +08:00
|
|
|
}
|
|
|
|
|
|
2006-10-02 17:18:06 +08:00
|
|
|
extern struct nsproxy init_nsproxy;
|
|
|
|
|
#define INIT_NSPROXY(nsproxy) { \
|
2006-12-08 18:37:59 +08:00
|
|
|
.pid_ns = &init_pid_ns, \
|
2006-10-02 17:18:06 +08:00
|
|
|
.count = ATOMIC_INIT(1), \
|
2006-10-02 17:18:14 +08:00
|
|
|
.uts_ns = &init_uts_ns, \
|
2006-12-08 18:37:56 +08:00
|
|
|
.mnt_ns = NULL, \
|
2007-09-13 15:16:29 +08:00
|
|
|
INIT_NET_NS(net_ns) \
|
2006-10-02 17:18:20 +08:00
|
|
|
INIT_IPC_NS(ipc_ns) \
|
2006-10-02 17:18:06 +08:00
|
|
|
}
|
|
|
|
|
|
2005-04-17 06:20:36 +08:00
|
|
|
#define INIT_SIGHAND(sighand) { \
|
|
|
|
|
.count = ATOMIC_INIT(1), \
|
|
|
|
|
.action = { { { .sa_handler = NULL, } }, }, \
|
2006-07-03 15:24:34 +08:00
|
|
|
.siglock = __SPIN_LOCK_UNLOCKED(sighand.siglock), \
|
2007-09-21 03:40:16 +08:00
|
|
|
.signalfd_wqh = __WAIT_QUEUE_HEAD_INITIALIZER(sighand.signalfd_wqh), \
|
2005-04-17 06:20:36 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
extern struct group_info init_groups;
|
|
|
|
|
|
2007-05-11 13:23:00 +08:00
|
|
|
#define INIT_STRUCT_PID { \
|
|
|
|
|
.count = ATOMIC_INIT(1), \
|
|
|
|
|
.tasks = { \
|
|
|
|
|
{ .first = &init_task.pids[PIDTYPE_PID].node }, \
|
|
|
|
|
{ .first = &init_task.pids[PIDTYPE_PGID].node }, \
|
|
|
|
|
{ .first = &init_task.pids[PIDTYPE_SID].node }, \
|
|
|
|
|
}, \
|
|
|
|
|
.rcu = RCU_HEAD_INIT, \
|
2007-10-19 14:40:03 +08:00
|
|
|
.level = 0, \
|
|
|
|
|
.numbers = { { \
|
|
|
|
|
.nr = 0, \
|
|
|
|
|
.ns = &init_pid_ns, \
|
|
|
|
|
.pid_chain = { .next = NULL, .pprev = NULL }, \
|
|
|
|
|
}, } \
|
2007-05-11 13:23:00 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#define INIT_PID_LINK(type) \
|
|
|
|
|
{ \
|
|
|
|
|
.node = { \
|
|
|
|
|
.next = NULL, \
|
|
|
|
|
.pprev = &init_struct_pid.tasks[type].first, \
|
|
|
|
|
}, \
|
|
|
|
|
.pid = &init_struct_pid, \
|
|
|
|
|
}
|
|
|
|
|
|
2008-01-10 17:53:18 +08:00
|
|
|
#ifdef CONFIG_AUDITSYSCALL
|
|
|
|
|
#define INIT_IDS \
|
2008-01-08 23:06:53 +08:00
|
|
|
.loginuid = -1, \
|
|
|
|
|
.sessionid = -1,
|
2008-01-10 17:53:18 +08:00
|
|
|
#else
|
|
|
|
|
#define INIT_IDS
|
|
|
|
|
#endif
|
capabilities: introduce per-process capability bounding set
The capability bounding set is a set beyond which capabilities cannot grow.
Currently cap_bset is per-system. It can be manipulated through sysctl,
but only init can add capabilities. Root can remove capabilities. By
default it includes all caps except CAP_SETPCAP.
This patch makes the bounding set per-process when file capabilities are
enabled. It is inherited at fork from parent. Noone can add elements,
CAP_SETPCAP is required to remove them.
One example use of this is to start a safer container. For instance, until
device namespaces or per-container device whitelists are introduced, it is
best to take CAP_MKNOD away from a container.
The bounding set will not affect pP and pE immediately. It will only
affect pP' and pE' after subsequent exec()s. It also does not affect pI,
and exec() does not constrain pI'. So to really start a shell with no way
of regain CAP_MKNOD, you would do
prctl(PR_CAPBSET_DROP, CAP_MKNOD);
cap_t cap = cap_get_proc();
cap_value_t caparray[1];
caparray[0] = CAP_MKNOD;
cap_set_flag(cap, CAP_INHERITABLE, 1, caparray, CAP_DROP);
cap_set_proc(cap);
cap_free(cap);
The following test program will get and set the bounding
set (but not pI). For instance
./bset get
(lists capabilities in bset)
./bset drop cap_net_raw
(starts shell with new bset)
(use capset, setuid binary, or binary with
file capabilities to try to increase caps)
************************************************************
cap_bound.c
************************************************************
#include <sys/prctl.h>
#include <linux/capability.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#ifndef PR_CAPBSET_READ
#define PR_CAPBSET_READ 23
#endif
#ifndef PR_CAPBSET_DROP
#define PR_CAPBSET_DROP 24
#endif
int usage(char *me)
{
printf("Usage: %s get\n", me);
printf(" %s drop <capability>\n", me);
return 1;
}
#define numcaps 32
char *captable[numcaps] = {
"cap_chown",
"cap_dac_override",
"cap_dac_read_search",
"cap_fowner",
"cap_fsetid",
"cap_kill",
"cap_setgid",
"cap_setuid",
"cap_setpcap",
"cap_linux_immutable",
"cap_net_bind_service",
"cap_net_broadcast",
"cap_net_admin",
"cap_net_raw",
"cap_ipc_lock",
"cap_ipc_owner",
"cap_sys_module",
"cap_sys_rawio",
"cap_sys_chroot",
"cap_sys_ptrace",
"cap_sys_pacct",
"cap_sys_admin",
"cap_sys_boot",
"cap_sys_nice",
"cap_sys_resource",
"cap_sys_time",
"cap_sys_tty_config",
"cap_mknod",
"cap_lease",
"cap_audit_write",
"cap_audit_control",
"cap_setfcap"
};
int getbcap(void)
{
int comma=0;
unsigned long i;
int ret;
printf("i know of %d capabilities\n", numcaps);
printf("capability bounding set:");
for (i=0; i<numcaps; i++) {
ret = prctl(PR_CAPBSET_READ, i);
if (ret < 0)
perror("prctl");
else if (ret==1)
printf("%s%s", (comma++) ? ", " : " ", captable[i]);
}
printf("\n");
return 0;
}
int capdrop(char *str)
{
unsigned long i;
int found=0;
for (i=0; i<numcaps; i++) {
if (strcmp(captable[i], str) == 0) {
found=1;
break;
}
}
if (!found)
return 1;
if (prctl(PR_CAPBSET_DROP, i)) {
perror("prctl");
return 1;
}
return 0;
}
int main(int argc, char *argv[])
{
if (argc<2)
return usage(argv[0]);
if (strcmp(argv[1], "get")==0)
return getbcap();
if (strcmp(argv[1], "drop")!=0 || argc<3)
return usage(argv[0]);
if (capdrop(argv[2])) {
printf("unknown capability\n");
return 1;
}
return execl("/bin/bash", "/bin/bash", NULL);
}
************************************************************
[serue@us.ibm.com: fix typo]
Signed-off-by: Serge E. Hallyn <serue@us.ibm.com>
Signed-off-by: Andrew G. Morgan <morgan@kernel.org>
Cc: Stephen Smalley <sds@tycho.nsa.gov>
Cc: James Morris <jmorris@namei.org>
Cc: Chris Wright <chrisw@sous-sol.org>
Cc: Casey Schaufler <casey@schaufler-ca.com>a
Signed-off-by: "Serge E. Hallyn" <serue@us.ibm.com>
Tested-by: Jiri Slaby <jirislaby@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-02-05 14:29:45 +08:00
|
|
|
|
|
|
|
|
#ifdef CONFIG_SECURITY_FILE_CAPABILITIES
|
|
|
|
|
/*
|
|
|
|
|
* Because of the reduced scope of CAP_SETPCAP when filesystem
|
|
|
|
|
* capabilities are in effect, it is safe to allow CAP_SETPCAP to
|
|
|
|
|
* be available in the default configuration.
|
|
|
|
|
*/
|
|
|
|
|
# define CAP_INIT_BSET CAP_FULL_SET
|
|
|
|
|
#else
|
|
|
|
|
# define CAP_INIT_BSET CAP_INIT_EFF_SET
|
|
|
|
|
#endif
|
|
|
|
|
|
rcu: Merge preemptable-RCU functionality into hierarchical RCU
Create a kernel/rcutree_plugin.h file that contains definitions
for preemptable RCU (or, under the #else branch of the #ifdef,
empty definitions for the classic non-preemptable semantics).
These definitions fit into plugins defined in kernel/rcutree.c
for this purpose.
This variant of preemptable RCU uses a new algorithm whose
read-side expense is roughly that of classic hierarchical RCU
under CONFIG_PREEMPT. This new algorithm's update-side expense
is similar to that of classic hierarchical RCU, and, in absence
of read-side preemption or blocking, is exactly that of classic
hierarchical RCU. Perhaps more important, this new algorithm
has a much simpler implementation, saving well over 1,000 lines
of code compared to mainline's implementation of preemptable
RCU, which will hopefully be retired in favor of this new
algorithm.
The simplifications are obtained by maintaining per-task
nesting state for running tasks, and using a simple
lock-protected algorithm to handle accounting when tasks block
within RCU read-side critical sections, making use of lessons
learned while creating numerous user-level RCU implementations
over the past 18 months.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: laijs@cn.fujitsu.com
Cc: dipankar@in.ibm.com
Cc: akpm@linux-foundation.org
Cc: mathieu.desnoyers@polymtl.ca
Cc: josht@linux.vnet.ibm.com
Cc: dvhltc@us.ibm.com
Cc: niv@us.ibm.com
Cc: peterz@infradead.org
Cc: rostedt@goodmis.org
LKML-Reference: <12509746134003-git-send-email->
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-08-23 04:56:52 +08:00
|
|
|
#ifdef CONFIG_PREEMPT_RCU
|
|
|
|
|
#define INIT_TASK_RCU_PREEMPT(tsk) \
|
|
|
|
|
.rcu_read_lock_nesting = 0, \
|
|
|
|
|
.rcu_flipctr_idx = 0,
|
|
|
|
|
#elif defined(CONFIG_TREE_PREEMPT_RCU)
|
|
|
|
|
#define INIT_TASK_RCU_PREEMPT(tsk) \
|
|
|
|
|
.rcu_read_lock_nesting = 0, \
|
|
|
|
|
.rcu_read_unlock_special = 0, \
|
|
|
|
|
.rcu_blocked_cpu = -1, \
|
|
|
|
|
.rcu_node_entry = LIST_HEAD_INIT(tsk.rcu_node_entry),
|
|
|
|
|
#else
|
|
|
|
|
#define INIT_TASK_RCU_PREEMPT(tsk)
|
|
|
|
|
#endif
|
|
|
|
|
|
2008-11-14 07:39:16 +08:00
|
|
|
extern struct cred init_cred;
|
|
|
|
|
|
2009-05-24 00:29:00 +08:00
|
|
|
#ifdef CONFIG_PERF_COUNTERS
|
|
|
|
|
# define INIT_PERF_COUNTERS(tsk) \
|
|
|
|
|
.perf_counter_mutex = \
|
|
|
|
|
__MUTEX_INITIALIZER(tsk.perf_counter_mutex), \
|
|
|
|
|
.perf_counter_list = LIST_HEAD_INIT(tsk.perf_counter_list),
|
|
|
|
|
#else
|
|
|
|
|
# define INIT_PERF_COUNTERS(tsk)
|
|
|
|
|
#endif
|
|
|
|
|
|
2005-04-17 06:20:36 +08:00
|
|
|
/*
|
|
|
|
|
* INIT_TASK is used to set up the first task table, touch at
|
|
|
|
|
* your own risk!. Base=0, limit=0x1fffff (=2MB)
|
|
|
|
|
*/
|
|
|
|
|
#define INIT_TASK(tsk) \
|
|
|
|
|
{ \
|
|
|
|
|
.state = 0, \
|
2007-05-09 17:35:17 +08:00
|
|
|
.stack = &init_thread_info, \
|
2005-04-17 06:20:36 +08:00
|
|
|
.usage = ATOMIC_INIT(2), \
|
2008-07-25 16:47:37 +08:00
|
|
|
.flags = PF_KTHREAD, \
|
2005-04-17 06:20:36 +08:00
|
|
|
.lock_depth = -1, \
|
|
|
|
|
.prio = MAX_PRIO-20, \
|
|
|
|
|
.static_prio = MAX_PRIO-20, \
|
2006-06-27 17:54:51 +08:00
|
|
|
.normal_prio = MAX_PRIO-20, \
|
2005-04-17 06:20:36 +08:00
|
|
|
.policy = SCHED_NORMAL, \
|
|
|
|
|
.cpus_allowed = CPU_MASK_ALL, \
|
|
|
|
|
.mm = NULL, \
|
|
|
|
|
.active_mm = &init_mm, \
|
2008-04-20 01:45:00 +08:00
|
|
|
.se = { \
|
|
|
|
|
.group_node = LIST_HEAD_INIT(tsk.se.group_node), \
|
|
|
|
|
}, \
|
2008-01-26 04:08:27 +08:00
|
|
|
.rt = { \
|
|
|
|
|
.run_list = LIST_HEAD_INIT(tsk.rt.run_list), \
|
2008-01-26 04:08:30 +08:00
|
|
|
.time_slice = HZ, \
|
|
|
|
|
.nr_cpus_allowed = NR_CPUS, \
|
|
|
|
|
}, \
|
2005-04-17 06:20:36 +08:00
|
|
|
.tasks = LIST_HEAD_INIT(tsk.tasks), \
|
sched: create "pushable_tasks" list to limit pushing to one attempt
The RT scheduler employs a "push/pull" design to actively balance tasks
within the system (on a per disjoint cpuset basis). When a task is
awoken, it is immediately determined if there are any lower priority
cpus which should be preempted. This is opposed to the way normal
SCHED_OTHER tasks behave, which will wait for a periodic rebalancing
operation to occur before spreading out load.
When a particular RQ has more than 1 active RT task, it is said to
be in an "overloaded" state. Once this occurs, the system enters
the active balancing mode, where it will try to push the task away,
or persuade a different cpu to pull it over. The system will stay
in this state until the system falls back below the <= 1 queued RT
task per RQ.
However, the current implementation suffers from a limitation in the
push logic. Once overloaded, all tasks (other than current) on the
RQ are analyzed on every push operation, even if it was previously
unpushable (due to affinity, etc). Whats more, the operation stops
at the first task that is unpushable and will not look at items
lower in the queue. This causes two problems:
1) We can have the same tasks analyzed over and over again during each
push, which extends out the fast path in the scheduler for no
gain. Consider a RQ that has dozens of tasks that are bound to a
core. Each one of those tasks will be encountered and skipped
for each push operation while they are queued.
2) There may be lower-priority tasks under the unpushable task that
could have been successfully pushed, but will never be considered
until either the unpushable task is cleared, or a pull operation
succeeds. The net result is a potential latency source for mid
priority tasks.
This patch aims to rectify these two conditions by introducing a new
priority sorted list: "pushable_tasks". A task is added to the list
each time a task is activated or preempted. It is removed from the
list any time it is deactivated, made current, or fails to push.
This works because a task only needs to be attempted to push once.
After an initial failure to push, the other cpus will eventually try to
pull the task when the conditions are proper. This also solves the
problem that we don't completely analyze all tasks due to encountering
an unpushable tasks. Now every task will have a push attempted (when
appropriate).
This reduces latency both by shorting the critical section of the
rq->lock for certain workloads, and by making sure the algorithm
considers all eligible tasks in the system.
[ rostedt: added a couple more BUG_ONs ]
Signed-off-by: Gregory Haskins <ghaskins@novell.com>
Acked-by: Steven Rostedt <srostedt@redhat.com>
2008-12-29 22:39:53 +08:00
|
|
|
.pushable_tasks = PLIST_NODE_INIT(tsk.pushable_tasks, MAX_PRIO), \
|
2008-03-25 09:36:23 +08:00
|
|
|
.ptraced = LIST_HEAD_INIT(tsk.ptraced), \
|
|
|
|
|
.ptrace_entry = LIST_HEAD_INIT(tsk.ptrace_entry), \
|
2005-04-17 06:20:36 +08:00
|
|
|
.real_parent = &tsk, \
|
|
|
|
|
.parent = &tsk, \
|
|
|
|
|
.children = LIST_HEAD_INIT(tsk.children), \
|
|
|
|
|
.sibling = LIST_HEAD_INIT(tsk.sibling), \
|
|
|
|
|
.group_leader = &tsk, \
|
2008-11-14 07:39:26 +08:00
|
|
|
.real_cred = &init_cred, \
|
2008-11-14 07:39:17 +08:00
|
|
|
.cred = &init_cred, \
|
2009-05-08 20:55:22 +08:00
|
|
|
.cred_guard_mutex = \
|
|
|
|
|
__MUTEX_INITIALIZER(tsk.cred_guard_mutex), \
|
2005-04-17 06:20:36 +08:00
|
|
|
.comm = "swapper", \
|
|
|
|
|
.thread = INIT_THREAD, \
|
|
|
|
|
.fs = &init_fs, \
|
|
|
|
|
.files = &init_files, \
|
|
|
|
|
.signal = &init_signals, \
|
|
|
|
|
.sighand = &init_sighand, \
|
2006-10-02 17:18:06 +08:00
|
|
|
.nsproxy = &init_nsproxy, \
|
2005-04-17 06:20:36 +08:00
|
|
|
.pending = { \
|
|
|
|
|
.list = LIST_HEAD_INIT(tsk.pending.list), \
|
|
|
|
|
.signal = {{0}}}, \
|
|
|
|
|
.blocked = {{0}}, \
|
2006-07-03 15:24:34 +08:00
|
|
|
.alloc_lock = __SPIN_LOCK_UNLOCKED(tsk.alloc_lock), \
|
2005-04-17 06:20:36 +08:00
|
|
|
.journal_info = NULL, \
|
|
|
|
|
.cpu_timers = INIT_CPU_TIMERS(tsk.cpu_timers), \
|
2005-06-27 16:55:12 +08:00
|
|
|
.fs_excl = ATOMIC_INIT(0), \
|
2007-05-08 15:30:08 +08:00
|
|
|
.pi_lock = __SPIN_LOCK_UNLOCKED(tsk.pi_lock), \
|
2008-09-02 06:52:40 +08:00
|
|
|
.timer_slack_ns = 50000, /* 50 usec default slack */ \
|
2007-05-11 13:23:00 +08:00
|
|
|
.pids = { \
|
|
|
|
|
[PIDTYPE_PID] = INIT_PID_LINK(PIDTYPE_PID), \
|
|
|
|
|
[PIDTYPE_PGID] = INIT_PID_LINK(PIDTYPE_PGID), \
|
|
|
|
|
[PIDTYPE_SID] = INIT_PID_LINK(PIDTYPE_SID), \
|
|
|
|
|
}, \
|
2007-10-17 14:25:50 +08:00
|
|
|
.dirties = INIT_PROP_LOCAL_SINGLE(dirties), \
|
2008-01-10 17:53:18 +08:00
|
|
|
INIT_IDS \
|
2009-05-24 00:29:00 +08:00
|
|
|
INIT_PERF_COUNTERS(tsk) \
|
2006-07-03 15:24:42 +08:00
|
|
|
INIT_TRACE_IRQFLAGS \
|
[PATCH] lockdep: core
Do 'make oldconfig' and accept all the defaults for new config options -
reboot into the kernel and if everything goes well it should boot up fine and
you should have /proc/lockdep and /proc/lockdep_stats files.
Typically if the lock validator finds some problem it will print out
voluminous debug output that begins with "BUG: ..." and which syslog output
can be used by kernel developers to figure out the precise locking scenario.
What does the lock validator do? It "observes" and maps all locking rules as
they occur dynamically (as triggered by the kernel's natural use of spinlocks,
rwlocks, mutexes and rwsems). Whenever the lock validator subsystem detects a
new locking scenario, it validates this new rule against the existing set of
rules. If this new rule is consistent with the existing set of rules then the
new rule is added transparently and the kernel continues as normal. If the
new rule could create a deadlock scenario then this condition is printed out.
When determining validity of locking, all possible "deadlock scenarios" are
considered: assuming arbitrary number of CPUs, arbitrary irq context and task
context constellations, running arbitrary combinations of all the existing
locking scenarios. In a typical system this means millions of separate
scenarios. This is why we call it a "locking correctness" validator - for all
rules that are observed the lock validator proves it with mathematical
certainty that a deadlock could not occur (assuming that the lock validator
implementation itself is correct and its internal data structures are not
corrupted by some other kernel subsystem). [see more details and conditionals
of this statement in include/linux/lockdep.h and
Documentation/lockdep-design.txt]
Furthermore, this "all possible scenarios" property of the validator also
enables the finding of complex, highly unlikely multi-CPU multi-context races
via single single-context rules, increasing the likelyhood of finding bugs
drastically. In practical terms: the lock validator already found a bug in
the upstream kernel that could only occur on systems with 3 or more CPUs, and
which needed 3 very unlikely code sequences to occur at once on the 3 CPUs.
That bug was found and reported on a single-CPU system (!). So in essence a
race will be found "piecemail-wise", triggering all the necessary components
for the race, without having to reproduce the race scenario itself! In its
short existence the lock validator found and reported many bugs before they
actually caused a real deadlock.
To further increase the efficiency of the validator, the mapping is not per
"lock instance", but per "lock-class". For example, all struct inode objects
in the kernel have inode->inotify_mutex. If there are 10,000 inodes cached,
then there are 10,000 lock objects. But ->inotify_mutex is a single "lock
type", and all locking activities that occur against ->inotify_mutex are
"unified" into this single lock-class. The advantage of the lock-class
approach is that all historical ->inotify_mutex uses are mapped into a single
(and as narrow as possible) set of locking rules - regardless of how many
different tasks or inode structures it took to build this set of rules. The
set of rules persist during the lifetime of the kernel.
To see the rough magnitude of checking that the lock validator does, here's a
portion of /proc/lockdep_stats, fresh after bootup:
lock-classes: 694 [max: 2048]
direct dependencies: 1598 [max: 8192]
indirect dependencies: 17896
all direct dependencies: 16206
dependency chains: 1910 [max: 8192]
in-hardirq chains: 17
in-softirq chains: 105
in-process chains: 1065
stack-trace entries: 38761 [max: 131072]
combined max dependencies: 2033928
hardirq-safe locks: 24
hardirq-unsafe locks: 176
softirq-safe locks: 53
softirq-unsafe locks: 137
irq-safe locks: 59
irq-unsafe locks: 176
The lock validator has observed 1598 actual single-thread locking patterns,
and has validated all possible 2033928 distinct locking scenarios.
More details about the design of the lock validator can be found in
Documentation/lockdep-design.txt, which can also found at:
http://redhat.com/~mingo/lockdep-patches/lockdep-design.txt
[bunk@stusta.de: cleanups]
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Signed-off-by: Adrian Bunk <bunk@stusta.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-07-03 15:24:50 +08:00
|
|
|
INIT_LOCKDEP \
|
2009-03-26 08:55:00 +08:00
|
|
|
INIT_FTRACE_GRAPH \
|
tracing: add same level recursion detection
The tracing infrastructure allows for recursion. That is, an interrupt
may interrupt the act of tracing an event, and that interrupt may very well
perform its own trace. This is a recursive trace, and is fine to do.
The problem arises when there is a bug, and the utility doing the trace
calls something that recurses back into the tracer. This recursion is not
caused by an external event like an interrupt, but by code that is not
expected to recurse. The result could be a lockup.
This patch adds a bitmask to the task structure that keeps track
of the trace recursion. To find the interrupt depth, the following
algorithm is used:
level = hardirq_count() + softirq_count() + in_nmi;
Here, level will be the depth of interrutps and softirqs, and even handles
the nmi. Then the corresponding bit is set in the recursion bitmask.
If the bit was already set, we know we had a recursion at the same level
and we warn about it and fail the writing to the buffer.
After the data has been committed to the buffer, we clear the bit.
No atomics are needed. The only races are with interrupts and they reset
the bitmask before returning anywy.
[ Impact: detect same irq level trace recursion ]
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2009-04-17 09:41:52 +08:00
|
|
|
INIT_TRACE_RECURSION \
|
rcu: Merge preemptable-RCU functionality into hierarchical RCU
Create a kernel/rcutree_plugin.h file that contains definitions
for preemptable RCU (or, under the #else branch of the #ifdef,
empty definitions for the classic non-preemptable semantics).
These definitions fit into plugins defined in kernel/rcutree.c
for this purpose.
This variant of preemptable RCU uses a new algorithm whose
read-side expense is roughly that of classic hierarchical RCU
under CONFIG_PREEMPT. This new algorithm's update-side expense
is similar to that of classic hierarchical RCU, and, in absence
of read-side preemption or blocking, is exactly that of classic
hierarchical RCU. Perhaps more important, this new algorithm
has a much simpler implementation, saving well over 1,000 lines
of code compared to mainline's implementation of preemptable
RCU, which will hopefully be retired in favor of this new
algorithm.
The simplifications are obtained by maintaining per-task
nesting state for running tasks, and using a simple
lock-protected algorithm to handle accounting when tasks block
within RCU read-side critical sections, making use of lessons
learned while creating numerous user-level RCU implementations
over the past 18 months.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: laijs@cn.fujitsu.com
Cc: dipankar@in.ibm.com
Cc: akpm@linux-foundation.org
Cc: mathieu.desnoyers@polymtl.ca
Cc: josht@linux.vnet.ibm.com
Cc: dvhltc@us.ibm.com
Cc: niv@us.ibm.com
Cc: peterz@infradead.org
Cc: rostedt@goodmis.org
LKML-Reference: <12509746134003-git-send-email->
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-08-23 04:56:52 +08:00
|
|
|
INIT_TASK_RCU_PREEMPT(tsk) \
|
2005-04-17 06:20:36 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#define INIT_CPU_TIMERS(cpu_timers) \
|
|
|
|
|
{ \
|
|
|
|
|
LIST_HEAD_INIT(cpu_timers[0]), \
|
|
|
|
|
LIST_HEAD_INIT(cpu_timers[1]), \
|
|
|
|
|
LIST_HEAD_INIT(cpu_timers[2]), \
|
|
|
|
|
}
|
|
|
|
|
|
2009-06-24 07:59:36 +08:00
|
|
|
/* Attach to the init_task data structure for proper alignment */
|
|
|
|
|
#define __init_task_data __attribute__((__section__(".data.init_task")))
|
|
|
|
|
|
2005-04-17 06:20:36 +08:00
|
|
|
|
|
|
|
|
#endif
|