From b51afd97e56a6d9436f8706f952ebc6af647773c Mon Sep 17 00:00:00 2001 From: Peter Krempa Date: Mon, 4 Apr 2022 16:14:32 +0200 Subject: [PATCH] docs: Convert 'internals' to RST and move it to 'kbase/internal/overview.rst' Note that this document was not referenced from any top level page. This patch does a straight conversion and leaves it unreferenced. Next patch will then modify it to serve as an overview (hence the new name) of how an API call happens. Signed-off-by: Peter Krempa Reviewed-by: Michal Privoznik --- docs/internals.html.in | 119 ------------------------------ docs/kbase/internals/meson.build | 1 + docs/kbase/internals/overview.rst | 117 +++++++++++++++++++++++++++++ docs/meson.build | 1 - 4 files changed, 118 insertions(+), 120 deletions(-) delete mode 100644 docs/internals.html.in create mode 100644 docs/kbase/internals/overview.rst diff --git a/docs/internals.html.in b/docs/internals.html.in deleted file mode 100644 index e474f7ddd7..0000000000 --- a/docs/internals.html.in +++ /dev/null @@ -1,119 +0,0 @@ - - - - -

libvirt internals

- -

- This section provides documents useful to those working on the libvirt - internals, adding new public APIs, new hypervisor drivers or extending - the libvirtd daemon code. -

- - - -

Before adding new code it will be important to get a basic understanding - of the many elements involved with making any call or change to the libvirt - code. The architecture goals must be adhered to - when submitting new code. Understanding the many places that need to be - touched and the interactions between various subsystems within libvirt - will directly correlate to the ability to be successful in getting new - code accepted.

-

The following diagram depicts code flow from a client application, in - this case the libvirt provided virsh command through the - various layers to elicit a response from some chosen hypervisor. -

- -

- virConnectOpen calling sequence -

-
    -
  • "virsh -c qemu:///system list --all" -

    After the virsh code processes the input arguments, it eventually - will make a call to open the connection using a default set of - authentication credentials (virConnectAuthDefault).

    • -
  • virConnectOpenAuth() -

    Each of the virConnectOpen APIs will first call virInitialize() and - then revector through the local "do_open():" call.

    -
      -
    • virInitialize() -

      Calls the registration API for each of the drivers with - client-side only capabilities and then call the remoteRegister() - API last. This ensures the virDriverTab[] tries local drivers - first before using the remote driver.

      • -
    • Loop through virDriverTab[] entries trying to call their - respective "open" entry point (in our case remoteOpen())
      • -
    • After successful return from the virDriverTab[] open() - API, attempt to find and open other drivers (network, interface, - storage, etc.)
      • -
    -
    • -
  • remoteOpen() -

    After a couple of URI checks, a call to doRemoteOpen() is made

    -
      -
    • Determine network transport and host/port to use from URI -

      The transport will be either tls, unix, ssh, libssh2, ext, - or tcp with the default of tls. Decode the host/port if provided - or default to "localhost".

      • -
    • virNetClientRegisterAsyncIO() -

      Register an I/O callback mechanism to get returned data via - virNetClientIncomingEvent()

      • -
    • "call(...REMOTE_PROC_OPEN...)" -

      Eventually routes into virNetClientProgramCall() which will - call virNetClientSendWithReply() and eventually uses - virNetClientIO()to send the message to libvirtd and - then waits for a response using virNetClientIOEventLoop()

      • -
    • virNetClientIncomingEvent() -

      Receives the returned packet and processes through - virNetClientIOUpdateCallback()

      • -
    -
    • -
  • libvirtd Daemon -

    -
      -
    • Daemon Startup -

      The daemon initialization processing will declare itself - as a daemon via a virNetDaemonNew() call, then creates new server - using virNetServerNew() and adds that server to the main daemon - struct with virNetDaemonAddServer() call. It will then use - virDriverLoadModule() to find/load all known drivers, - set up an RPC server program using the remoteProcs[] - table via a virNetServerProgramNew() call. The table is the - corollary to the remote_procedure enum list in - the client. It lists all the functions to be called in - the same order. Once RPC is set up, networking server sockets - are opened, the various driver state initialization routines - are run from the virStateDriverTab[], the network - links are enabled, and the daemon waits for work.

      • -
    • RPC -

      When a message is received, the remoteProcs[] - table is referenced for the 'REMOTE_PROC_OPEN' call entry. - This results in remoteDispatchOpen() being called via the - virNetServerProgramDispatchCall().

      • -
    • remoteDispatchOpen() -

      The API will read the argument passed picking out the - name of the driver to be opened. The code - will then call virConnectOpen() or virConnectOpenReadOnly() - depending on the argument flags.

      • -
    • virConnectOpen() or virConnectOpenReadOnly() -

      Just like the client except that upon entry the URI - is what was passed from the client and will be found - and opened to process the data.

      -

      The returned structure data is returned via the - virNetServer interfaces to the remote driver which then - returns it to the client application.

      • -
    -
    • -
- - diff --git a/docs/kbase/internals/meson.build b/docs/kbase/internals/meson.build index 879c4b2de8..8b5daad1f9 100644 --- a/docs/kbase/internals/meson.build +++ b/docs/kbase/internals/meson.build @@ -4,6 +4,7 @@ docs_kbase_internals_files = [ 'incremental-backup', 'locking', 'migration', + 'overview', 'rpc', ] diff --git a/docs/kbase/internals/overview.rst b/docs/kbase/internals/overview.rst new file mode 100644 index 0000000000..84ea7858db --- /dev/null +++ b/docs/kbase/internals/overview.rst @@ -0,0 +1,117 @@ +========================== +libvirt internals overview +========================== + +This section provides documents useful to those working on the libvirt +internals, adding new public APIs, new hypervisor drivers or extending the +libvirtd daemon code. + +- Introduction to basic rules and guidelines for `hacking <../../hacking.html>`__ on + libvirt code +- Guide to adding `public APIs <../../api_extension.html>`__ +- Insight into libvirt `event loop and worker + pool `__ +- Approach for `spawning commands `__ from libvirt + driver code +- The libvirt `RPC infrastructure `__ +- The `Resource Lock Manager `__ + +Before adding new code it will be important to get a basic understanding of the +many elements involved with making any call or change to the libvirt code. The +architecture `goals <../../goals.html>`__ must be adhered to when submitting new code. +Understanding the many places that need to be touched and the interactions +between various subsystems within libvirt will directly correlate to the ability +to be successful in getting new code accepted. + +The following diagram depicts code flow from a client application, in this case +the libvirt provided ``virsh`` command through the various layers to elicit a +response from some chosen hypervisor. + +.. image:: ../../images/libvirt-virConnect-example.png + :alt: virConnectOpen calling sequence + +- "virsh -c qemu:///system list --all" + + After the virsh code processes the input arguments, it eventually will make a + call to open the connection using a default set of authentication credentials + (virConnectAuthDefault). + +- virConnectOpenAuth() + + Each of the virConnectOpen APIs will first call virInitialize() and then + revector through the local "do_open():" call. + + - virInitialize() + + Calls the registration API for each of the drivers with client-side only + capabilities and then call the remoteRegister() API last. This ensures the + virDriverTab[] tries local drivers first before using the remote driver. + + - Loop through virDriverTab[] entries trying to call their respective "open" + entry point (in our case remoteOpen()) + + - After successful return from the virDriverTab[] open() API, attempt to + find and open other drivers (network, interface, storage, etc.) + +- remoteOpen() + + After a couple of URI checks, a call to doRemoteOpen() is made + + - Determine network transport and host/port to use from URI + + The transport will be either tls, unix, ssh, libssh2, ext, or tcp with the + default of tls. Decode the host/port if provided or default to + "localhost". + + - virNetClientRegisterAsyncIO() + + Register an I/O callback mechanism to get returned data via + virNetClientIncomingEvent() + + - "call(...REMOTE_PROC_OPEN...)" + + Eventually routes into virNetClientProgramCall() which will call + virNetClientSendWithReply() and eventually uses virNetClientIO()to send + the message to libvirtd and then waits for a response using + virNetClientIOEventLoop() + + - virNetClientIncomingEvent() + + Receives the returned packet and processes through + virNetClientIOUpdateCallback() + +- libvirtd Daemon + + - Daemon Startup + + The daemon initialization processing will declare itself as a daemon via a + virNetDaemonNew() call, then creates new server using virNetServerNew() + and adds that server to the main daemon struct with + virNetDaemonAddServer() call. It will then use virDriverLoadModule() to + find/load all known drivers, set up an RPC server program using the + ``remoteProcs[]`` table via a virNetServerProgramNew() call. The table is + the corollary to the ``remote_procedure`` enum list in the client. It + lists all the functions to be called in the same order. Once RPC is set + up, networking server sockets are opened, the various driver state + initialization routines are run from the ``virStateDriverTab[]``, the + network links are enabled, and the daemon waits for work. + + - RPC + + When a message is received, the ``remoteProcs[]`` table is referenced for + the 'REMOTE_PROC_OPEN' call entry. This results in remoteDispatchOpen() + being called via the virNetServerProgramDispatchCall(). + + - remoteDispatchOpen() + + The API will read the argument passed picking out the ``name`` of the + driver to be opened. The code will then call virConnectOpen() or + virConnectOpenReadOnly() depending on the argument ``flags``. + + - virConnectOpen() or virConnectOpenReadOnly() + + Just like the client except that upon entry the URI is what was passed + from the client and will be found and opened to process the data. + + The returned structure data is returned via the virNetServer interfaces to + the remote driver which then returns it to the client application. diff --git a/docs/meson.build b/docs/meson.build index f03d9b0816..a0e69f3119 100644 --- a/docs/meson.build +++ b/docs/meson.build @@ -23,7 +23,6 @@ docs_html_in_files = [ 'formatnode', 'formatnwfilter', 'index', - 'internals', 'remote', 'storage', 'uri',