From f108944ae0a88866a0d0b70bc42fb6724e5a3782 Mon Sep 17 00:00:00 2001 From: Osier Yang Date: Fri, 12 Oct 2012 17:50:43 +0800 Subject: [PATCH] doc: Sort out the relationship between , , and These 3 elements conflicts with each other in either the doc or the underlying codes. Current problems: Problem 1: The doc shouldn't simply say "These settings are superseded by CPU tuning. " for element . As except the tuning, allows to specify the current, maxmum vcpu number. Apart from that, also allows to specify the placement as "auto", which binds the domain process to the advisory nodeset from numad. Problem 2: Doc for says its "cpuset" specify the physical CPUs that the vcpus can be pinned. But it's not the truth, as actually it only pin domain process to the specified physical CPUs. So either it's a document bug, or code bug. Problem 3: Doc for says it supersed "cpuset" of , it's not quite correct, as each specify the pinning policy only for one vcpu. How about the ones which doesn't have specified? it says the vcpu will be pinned to all available physical CPUs, but what's the meaning of attribute "cpuset" of then? Problem 4: Doc for says it pin the emulator threads (domain process in other context, perhaps another follow up patch to cleanup the inconsistency is needed) to the physical CPUs specified its attribute "cpuset". Which conflicts with 's "cpuset". And actually in the underlying codes, it set the affinity for domain process twice if both "cpuset" for and are specified, and 's pinning will override 's. Problem 5: When "placement" of is "auto" (I.e. uses numad to get the advisory nodeset to which the domain process is pinned to), it will also be overridden by , This patch is trying to sort out the conflicts or bugs by: 1) Don't say is superseded by 2) Keep the semanteme for "cpuset" of (I.e. Still says it specify the physical CPUs the virtual CPUs). But modifying it to mention it also set the pinning policy for domain process, and the CPU placement of domain process specified by "cpuset" of will be ingored if specified, and similary, the CPU placement of vcpu thread will be ignored if it has specified, for vcpu which doesn't have specified, it inherits "cpuset" of . 3) Don't say is supersed by . If neither nor "cpuset" of is specified, the vcpu will be pinned to all available pCPUs. 4) If neither nor "cpuset" of is specified, the domain process (emulator threads in the context) will be pinned to all available pCPUs. 5) If "placement" of is "auto", is not allowed. 6) hotplugged vcpus will also inherit "cpuset" of Codes changes according to above document changes: 1) Inherit def->cpumask for each vcpu which doesn't have specified, during parsing. 2) ping the vcpu which doesn't have specified to def->cpumask either by cgroup for sched_setaffinity(2), which is actually done by 1). 3) Error out if "placement" == "auto", and is specified. Otherwise, is honored, and "cpuset" of is ignored. 4) Setup cgroup for each hotplugged vcpu, and setup the pinning policy by either cgroup or sched_setaffinity(2). 5) Remove cgroup and for each hot unplugged vcpu. Patches are following (6 in total except this patch) --- docs/formatdomain.html.in | 42 +++++++++++++++++++++++++-------------- 1 file changed, 27 insertions(+), 15 deletions(-) diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index d664e7e346..1ae8cf41b9 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -357,8 +357,18 @@ the maximum supported by the hypervisor. Since 0.4.4, this element can contain an optional cpuset attribute, which is a comma-separated - list of physical CPU numbers that virtual CPUs can be pinned - to. Each element in that list is either a single CPU number, + list of physical CPU numbers that domain process and virtual CPUs + can be pinned to by default. (NB: The pinning policy of domain + process and virtual CPUs can be specified separately by + cputune. If attribute emulatorpin + of cputune is specified, cpuset + specified by vcpu here will be ingored; Similarly, + For virtual CPUs which has vcpupin specified, + cpuset specified by cpuset here + will be ignored; For virtual CPUs which doesn't have + vcpupin specified, it will be pinned to the physical + CPUs specified by cpuset here). + Each element in that list is either a single CPU number, a range of CPU numbers, or a caret followed by a CPU number to be excluded from a previous range. Since 0.8.5, the optional attribute current can @@ -374,8 +384,7 @@ if it's specified. If both cpuset and placement are not specified, or if placement is "static", but no cpuset is specified, the domain process will be pinned to - all the available physical CPUs. These settings are superseded - by CPU tuning. + all the available physical CPUs. @@ -411,23 +420,26 @@
vcpupin
The optional vcpupin element specifies which of host's - physical CPUs the domain VCPU will be pinned to. This setting supersedes - previous VCPU placement specified in CPU - Allocation using vcpu element. If this is omitted, - each VCPU is pinned to all the physical CPUs by default. It contains two - required attributes, the attribute vcpu specifies vcpu id, - and the attribute cpuset is same as - attribute cpuset - of element vcpu. (NB: Only qemu driver support) + physical CPUs the domain VCPU will be pinned to. If this is omitted, + and attribute cpuset of element vcpu is + not specified, the vCPU is pinned to all the physical CPUs by default. + It contains two required attributes, the attribute vcpu + specifies vcpu id, and the attribute cpuset is same as + attribute cpuset of element vcpu. + (NB: Only qemu driver support) Since 0.9.0
emulatorpin
The optional emulatorpin element specifies which of host physical CPUs the "emulator", a subset of a domain not including vcpu, - will be pinned to. If this is omitted, "emulator" is pinned to all - the physical CPUs by default. It contains one required attribute - cpuset specifying which physical CPUs to pin to. + will be pinned to. If this is omitted, and attribute + cpuset of element vcpu is not specified, + "emulator" is pinned to all the physical CPUs by default. It contains + one required attribute cpuset specifying which physical + CPUs to pin to. NB, emulatorpin is not allowed if + attribute placement of element vcpu is + "auto".
shares