mirror of https://gitee.com/openkylin/libvirt.git
434 lines
18 KiB
HTML
434 lines
18 KiB
HTML
<html>
|
|
<head>
|
|
<title>Implementing a new API in Libvirt</title>
|
|
</head>
|
|
|
|
<body>
|
|
<h1>Implementing a new API in Libvirt</h1>
|
|
|
|
<ul id="toc"></ul>
|
|
|
|
<p>
|
|
This document walks you through the process of implementing a new
|
|
API in libvirt. It uses as an example the addition of an API for
|
|
separating maximum from current vcpu usage of a domain, over
|
|
the course of a fifteen-patch series.
|
|
Remember that new API consists of any new public functions, as
|
|
well as the addition of flags or extensions of XML used by
|
|
existing functions. The example in this document adds both new
|
|
functions and an XML extension. Not all libvirt API additions
|
|
require quite as many patches.
|
|
</p>
|
|
|
|
<p>
|
|
Before you begin coding, it is critical that you propose your
|
|
changes on the libvirt mailing list and get feedback on your ideas to
|
|
make sure what you're proposing fits with the general direction of the
|
|
project. Even before doing a proof of concept implementation, send an
|
|
email giving an overview of the functionality you think should be
|
|
added to libvirt. Someone may already be working on the feature you
|
|
want. Also, recognize that everything you write is likely to undergo
|
|
significant rework as you discuss it with the other developers, so
|
|
don't wait too long before getting feedback. In the vcpu example
|
|
below, list feedback was first requested
|
|
<a href="https://www.redhat.com/archives/libvir-list/2010-September/msg00423.html">here</a>
|
|
and resulted in several rounds of improvements before coding
|
|
began. In turn, this example is slightly rearranged from the actual
|
|
order of the commits.
|
|
</p>
|
|
|
|
<p>
|
|
Adding a new API to libvirt is not difficult, but there are quite a
|
|
few steps. This document assumes that you are familiar with C
|
|
programming and have checked out the libvirt code from the source code
|
|
repository and successfully built the existing tree. Instructions on
|
|
how to check out and build the code can be found at:
|
|
</p>
|
|
|
|
<p>
|
|
<a href="http://libvirt.org/downloads.html">http://libvirt.org/downloads.html</a>
|
|
</p>
|
|
|
|
<p>
|
|
Once you have a working development environment, the steps to create a
|
|
new API are:
|
|
</p>
|
|
<ol>
|
|
<li>define the public API</li>
|
|
<li>define the internal driver API</li>
|
|
<li>implement the public API</li>
|
|
<li>implement the remote protocol:
|
|
<ol>
|
|
<li>define the wire protocol format</li>
|
|
<li>implement the RPC client</li>
|
|
<li>implement the server side dispatcher</li>
|
|
</ol>
|
|
</li>
|
|
<li>use new API where appropriate in drivers</li>
|
|
<li>add virsh support</li>
|
|
<li>add common handling for new API</li>
|
|
<li>for each driver that can support the new API:
|
|
<ol>
|
|
<li>add prerequisite support</li>
|
|
<li>fully implement new API</li>
|
|
</ol>
|
|
</li>
|
|
</ol>
|
|
|
|
<p>
|
|
It is, of course, possible to implement the pieces in any order, but
|
|
if the development tasks are completed in the order listed, the code
|
|
will compile after each step. Given the number of changes required,
|
|
verification after each step is highly recommended.
|
|
</p>
|
|
|
|
<p>
|
|
Submit new code in the form shown in the example code: one patch
|
|
per step. That's not to say submit patches before you have working
|
|
functionality--get the whole thing working and make sure you're happy
|
|
with it. Then use git or some other version control system that lets
|
|
you rewrite your commit history and break patches into pieces so you
|
|
don't drop a big blob of code on the mailing list in one go.
|
|
Also, you should follow the upstream tree, and rebase your
|
|
series to adapt your patches to work with any other changes
|
|
that were accepted upstream during your development.
|
|
</p>
|
|
|
|
<p>
|
|
Don't mix anything else into the patches you submit. The patches
|
|
should be the minimal changes required to implement the functionality
|
|
you're adding. If you notice a bug in unrelated code (i.e., code you
|
|
don't have to touch to implement your API change) during development,
|
|
create a patch that just addresses that bug and submit it
|
|
separately.
|
|
</p>
|
|
|
|
<p>With that said, let's begin.</p>
|
|
|
|
<h2><a name='publicapi'>Defining the public API</a></h2>
|
|
|
|
<p>The first task is to define the public API. If the new API
|
|
involves an XML extension, you have to enhance the RelaxNG
|
|
schema and document the new elements or attributes:</p>
|
|
|
|
<p><code>
|
|
docs/schemas/domain.rng<br/>
|
|
docs/formatdomain.html.in
|
|
</code></p>
|
|
|
|
<p>If the API extension involves a new function, you have to add a
|
|
declaration in the public header, and arrange to export the
|
|
function name (symbol) so other programs can link against the
|
|
libvirt library and call the new function:</p>
|
|
|
|
<p><code>
|
|
include/libvirt/libvirt.h.in
|
|
src/libvirt_public.syms
|
|
</code></p>
|
|
|
|
<p>
|
|
This task is in many ways the most important to get right, since once
|
|
the API has been committed to the repository, it's libvirt's policy
|
|
never to change it. Mistakes in the implementation are bugs that you
|
|
can fix. Make a mistake in the API definition and you're stuck with
|
|
it, so think carefully about the interface and don't be afraid to
|
|
rework it as you go through the process of implementing it.
|
|
</p>
|
|
|
|
<p class="example">See <a href="api_extension/0001-add-to-xml.patch">0001-add-to-xml.patch</a>
|
|
and <a href="api_extension/0002-add-new-public-API.patch">0002-add-new-public-API.patch</a>
|
|
for example code.</p>
|
|
|
|
<h2><a name='internalapi'>Defining the internal API</a></h2>
|
|
|
|
<p>
|
|
Each public API call is associated with a driver, such as a host
|
|
virtualization driver, a network virtualization driver, a storage
|
|
virtualization driver, a state driver, or a device monitor. Adding
|
|
the internal API is ordinarily a matter of adding a new member to the
|
|
struct representing one of these drivers.
|
|
</p>
|
|
|
|
<p>
|
|
Of course, it's possible that the new API will involve the creation of
|
|
an entirely new driver type, in which case the changes will include the
|
|
creation of a new struct type to represent the new driver type.
|
|
</p>
|
|
|
|
<p>The driver structs are defined in:</p>
|
|
|
|
<p><code>src/driver.h</code></p>
|
|
|
|
<p>
|
|
To define the internal API, first typedef the driver function
|
|
prototype and then add a new field for it to the relevant driver
|
|
struct. Then, update all existing instances of the driver to
|
|
provide a <code>NULL</code> stub for the new function.
|
|
</p>
|
|
|
|
<p class="example">See <a href="api_extension/0003-define-internal-driver-API.patch">0003-define-internal-driver-API.patch</a></p>
|
|
|
|
<h2><a name='implpublic'>Implementing the public API</a></h2>
|
|
|
|
<p>
|
|
Implementing the public API is largely a formality in which we wire up
|
|
public API to the internal driver API. The public API implementation
|
|
takes care of some basic validity checks before passing control to the
|
|
driver implementation. In RFC 2119 vocabulary, this function:
|
|
</p>
|
|
|
|
<ol class="ordinarylist">
|
|
<li>SHOULD log a message with VIR_DEBUG() indicating that it is
|
|
being called and its parameters;</li>
|
|
<li>MUST call virResetLastError();</li>
|
|
<li>SHOULD confirm that the connection is valid with
|
|
VIR_IS_CONNECT(conn);</li>
|
|
<li><strong>SECURITY: If the API requires a connection with write
|
|
privileges, MUST confirm that the connection flags do not
|
|
indicate that the connection is read-only;</strong></li>
|
|
<li>SHOULD do basic validation of the parameters that are being
|
|
passed in;</li>
|
|
<li>MUST confirm that the driver for this connection exists and that
|
|
it implements this function;</li>
|
|
<li>MUST call the internal API;</li>
|
|
<li>SHOULD log a message with VIR_DEBUG() indicating that it is
|
|
returning, its return value, and status.</li>
|
|
<li>MUST return status to the caller.</li>
|
|
</ol>
|
|
|
|
<p>The public API calls are implemented in:</p>
|
|
|
|
<p><code>src/libvirt.c</code></p>
|
|
|
|
<p class="example">See <a href="api_extension/0004-implement-the-public-APIs.patch">0004-implement-the-public-APIs.patch</a></p>
|
|
|
|
<h2><a name='remoteproto'>Implementing the remote protocol</a></h2>
|
|
|
|
<p>
|
|
Implementing the remote protocol is essentially a
|
|
straightforward exercise which is probably most easily
|
|
understood by referring to the existing code and the example
|
|
patch. It involves several related changes, including the
|
|
regeneration of derived files, with further details below.
|
|
</p>
|
|
|
|
<p class="example">See <a href="api_extension/0005-implement-the-remote-protocol.patch">0005-implement-the-remote-protocol.patch</a></p>
|
|
|
|
<h3><a name='wireproto'>Defining the wire protocol format</a></h3>
|
|
|
|
<p>
|
|
Defining the wire protocol involves making additions to:
|
|
</p>
|
|
|
|
<p><code>src/remote/remote_protocol.x</code></p>
|
|
|
|
<p>
|
|
First, create two new structs for each new function that you're adding
|
|
to the API. One struct describes the parameters to be passed to the
|
|
remote function, and a second struct describes the value returned by
|
|
the remote function. The one exception to this rule is that functions
|
|
that return only 0 or -1 for status do not require a struct for returned
|
|
data.
|
|
</p>
|
|
|
|
<p>
|
|
Second, add values to the remote_procedure enum for each new function
|
|
added to the API.
|
|
</p>
|
|
|
|
<p>
|
|
Once these changes are in place, it's necessary to run 'make rpcgen'
|
|
in the src directory to create the .c and .h files required by the
|
|
remote protocol code. This must be done on a Linux host using the
|
|
GLibC rpcgen program. Other rpcgen versions may generate code which
|
|
results in bogus compile time warnings. This regenerates the
|
|
following files:
|
|
</p>
|
|
|
|
<p><code>
|
|
daemon/remote_dispatch_args.h
|
|
daemon/remote_dispatch_prototypes.h
|
|
daemon/remote_dispatch_table.h
|
|
src/remote/remote_protocol.c
|
|
src/remote/remote_protocol.h
|
|
</code></p>
|
|
|
|
<h3><a name='rpcclient'>Implement the RPC client</a></h3>
|
|
|
|
<p>
|
|
Implementing the uses the rpcgen generated .h files. The remote
|
|
method calls go in:
|
|
</p>
|
|
|
|
<p><code>src/remote/remote_internal.c</code></p>
|
|
|
|
<p>Each remote method invocation does the following:</p>
|
|
|
|
<ol class="ordinarylist">
|
|
<li>locks the remote driver;</li>
|
|
<li>sets up the method arguments;</li>
|
|
<li>invokes the remote function;</li>
|
|
<li>checks the return value, if necessary;</li>
|
|
<li>extracts any returned data;</li>
|
|
<li>frees any returned data;</li>
|
|
<li>unlocks the remote driver.</li>
|
|
</ol>
|
|
|
|
<h3><a name="serverdispatch">Implement the server side dispatcher</a></h3>
|
|
|
|
<p>
|
|
Implementing the server side of the remote function call is simply a
|
|
matter of deserializing the parameters passed in from the remote
|
|
caller and passing them to the corresponding internal API function.
|
|
The server side dispatchers are implemented in:
|
|
</p>
|
|
|
|
<p><code>daemon/remote.c</code></p>
|
|
|
|
<p>Again, this step uses the .h files generated by make rpcgen.</p>
|
|
|
|
<p>
|
|
After all three pieces of the remote protocol are complete, and
|
|
the generated files have been updated, it will be necessary to
|
|
update the file:</p>
|
|
|
|
<p><code>src/remote_protocol-structs</code></p>
|
|
|
|
<p>
|
|
This file should only have new lines added; modifications to
|
|
existing lines probably imply a backwards-incompatible API change.
|
|
</p>
|
|
|
|
<p class="example">See <a href="api_extension/0005-implement-the-remote-protocol.patch">0005-implement-the-remote-protocol.patch</a></p>
|
|
|
|
<h2><a name="internaluseapi">Use the new API internally</a></h2>
|
|
|
|
<p>
|
|
Sometimes, a new API serves as a superset of existing API, by
|
|
adding more granularity in what can be managed. When this is
|
|
the case, it makes sense to share a common implementation by
|
|
making the older API become a trivial wrapper around the new
|
|
API, rather than duplicating the common code. This step should
|
|
not introduce any semantic differences for the old API, and is
|
|
not necessary if the new API has no relation to existing API.
|
|
</p>
|
|
|
|
<p class="example">See <a href="api_extension/0006-make-old-API-trivially-wrap-to-new-API.patch">0006-make-old-API-trivially-wrap-to-new-API.patch</a></p>
|
|
|
|
<h2><a name="virshuseapi">Expose the new API in virsh</a></h2>
|
|
|
|
<p>
|
|
All new API should be manageable from the virsh command line
|
|
shell. This proves that the API is sufficient for the intended
|
|
purpose, and helps to identify whether the proposed API needs
|
|
slight changes for easier usage. However, remember that virsh
|
|
is used to connect to hosts running older versions of libvirtd,
|
|
so new commands should have fallbacks to an older API if
|
|
possible; implementing the virsh hooks at this point makes it
|
|
very easy to test these fallbacks. Also remember to document
|
|
virsh additions.
|
|
</p>
|
|
|
|
<p>
|
|
A virsh command is composed of a few pieces of code. You need to
|
|
define an array of vshCmdInfo structs for each new command that
|
|
contain the help text and the command description text. You also need
|
|
an array of vshCmdOptDef structs to describe the command options.
|
|
Once you have those pieces in place you can write the function
|
|
implementing the virsh command. Finally, you need to add the new
|
|
command to the commands[] array. The following files need changes:
|
|
</p>
|
|
|
|
<p><code>
|
|
tools/virsh.c<br/>
|
|
tools/virsh.pod
|
|
</code></p>
|
|
|
|
<p class="example">See <a href="api_extension/0007-add-virsh-support.patch">0007-add-virsh-support.patch</a></p>
|
|
|
|
<h2><a name="driverimpl">Implement the driver methods</a></h2>
|
|
|
|
<p>
|
|
So, after all that, we get to the fun part. All functionality in
|
|
libvirt is implemented inside a driver. Thus, here is where you
|
|
implement whatever functionality you're adding to libvirt. You'll
|
|
either need to add additional files to the src directory or extend
|
|
files that are already there, depending on what functionality you're
|
|
adding.
|
|
</p>
|
|
|
|
<h3><a name="commonimpl">Implement common handling</a></h3>
|
|
|
|
<p>
|
|
If the new API is applicable to more than one driver, it may
|
|
make sense to provide some utility routines, or to factor some
|
|
of the work into the dispatcher, to avoid reimplementing the
|
|
same code in every driver. In the example code, this involved
|
|
adding a member to the virDomainDefPtr struct for mapping
|
|
between the XML API addition and the in-memory representation of
|
|
a domain, along with updating all clients to use the new member.
|
|
Up to this point, there have been no changes to existing
|
|
semantics, and the new APIs will fail unless they are used in
|
|
the same way as the older API wrappers.
|
|
</p>
|
|
|
|
<p class="example">See <a href="api_extension/0008-support-new-xml.patch">0008-support-new-xml.patch</a></p>
|
|
|
|
<h3><a name="drivercode">Implement driver handling</a></h3>
|
|
|
|
<p>
|
|
The remaining patches should only touch one driver at a time.
|
|
It is possible to implement all changes for a driver in one
|
|
patch, but for review purposes it may still make sense to break
|
|
things into simpler steps. Here is where the new APIs finally
|
|
start working.
|
|
</p>
|
|
|
|
<p>
|
|
In the example patches, three separate drivers are supported:
|
|
test, qemu, and xen. It is always a good idea to patch the test
|
|
driver in addition to the target driver, to prove that the API
|
|
can be used for more than one driver. The example updates the
|
|
test driver in one patch:
|
|
</p>
|
|
|
|
<p class="example">See <a href="api_extension/0009-support-all-flags-in-test-driver.patch">0009-support-all-flags-in-test-driver.patch</a></p>
|
|
|
|
<p>
|
|
The qemu changes were easier to split into two phases, one for
|
|
updating the mapping between the new XML and the hypervisor
|
|
command line arguments, and one for supporting all possible
|
|
flags of the new API:
|
|
</p>
|
|
|
|
<p class="example">See <a href="api_extension/0010-improve-vcpu-support-in-qemu-command-line.patch">0010-improve-vcpu-support-in-qemu-command-line.patch</a>
|
|
and <a href="api_extension/0011-complete-vcpu-support-in-qemu-driver.patch">0011-complete-vcpu-support-in-qemu-driver.patch</a></p>
|
|
|
|
<p>
|
|
Finally, the example breaks the xen driver changes across four
|
|
patches. One maps the XML changes to the hypervisor command,
|
|
the next two are independently implementing the getter and
|
|
setter APIs, and the last one provides cleanup of code that was
|
|
rendered dead by the new API.
|
|
</p>
|
|
|
|
<p class="example">See <a href="api_extension/0012-improve-vcpu-support-in-xen-command-line.patch">0012-improve-vcpu-support-in-xen-command-line.patch</a>,
|
|
<a href="api_extension/0013-improve-getting-xen-vcpu-counts.patch">0013-improve-getting-xen-vcpu-counts.patch</a>,
|
|
<a href="api_extension/0014-improve-setting-xen-vcpu-counts.patch">0014-improve-setting-xen-vcpu-counts.patch</a>,
|
|
and <a href="api_extension/0015-remove-dead-xen-code.patch">0015-remove-dead-xen-code.patch</a></p>
|
|
|
|
<p>
|
|
The exact details of the example code are probably uninteresting
|
|
unless you're concerned with virtual cpu management.
|
|
</p>
|
|
|
|
<p>
|
|
Once you have working functionality, run make check and make
|
|
syntax-check on each patch of the series before submitting
|
|
patches. It may also be worth writing tests for the libvirt-TCK
|
|
testsuite to exercise your new API, although those patches are
|
|
not kept in the libvirt repository.
|
|
</p>
|
|
</body>
|
|
</html>
|