<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
  <body>
    <h1>Project Strategy</h1>

    <p>
      This document attempts to outline the libvirt project strategy for
      the near future. Think of this as a high level vision or to-do list
      setting the direction for the project and its developers to take.
    </p>

    <h2>Language consolidation</h2>

    <p>
      At time of writing libvirt uses the following languages:
    </p>

    <dl>
      <dt>C</dt>
      <dd>The core libvirt library, daemons, and helper tools are all written
        in the C language.</dd>
      <dt>Python</dt>
      <dd>Various supporting build/test scripts are written in Python, with
        compatibility for Python 2 and 3.</dd>
      <dt>Perl</dt>
      <dd>Various supporting build/test scripts are written in Perl. It is
        also used for many syntax-check inline rules</dd>
      <dt>Shell</dt>
      <dd><code>configure</code>, generated by autoconf, is a shell script.
        Shell is also used for some simple build/test scripts. At runtime
        libvirt avoids shell except when using SSH tunnels to a remote
        host</dd>
      <dt>XSLT</dt>
      <dd>The website uses XSLT for its templating system. The API
        documentation is also autogenerated from an XML description
        using XSLT</dd>
      <dt>HTML</dt>
      <dd>The website documentation is all written in plain HTML. Some HTML
        is also auto-generated for API documentation</dd>
      <dt>M4</dt>
      <dd>The autoconf <code>configure</code> script uses a large number of
        M4 macros to generate its content</dd>
      <dt>make</dt>
      <dd>The core build system uses the traditional GNU make recipes</dd>
      <dt>automake</dt>
      <dd>The make recipes use automake's language extensions which are
        then turned into regular make rules</dd>
      <dt>awk/sed</dt>
      <dd>A number of the syntax-check inline rules involve use of awk/sed
        scripts</dd>
      <dt>POD</dt>
      <dd>The command line manual pages are typically written in Perl's POD
        format, and converted to troff</dd>
    </dl>

    <p>
      The wide range of languages used present a knowledge burden for
      developers involved in libvirt, especially when there are multiple
      languages all used in the same problem spaces. This is most notable
      in the build system which uses a combination of shell, M4, make,
      automake, awk, sed, Perl and Python, with debugging requiring
      understanding of the interactions between many languages. The
      popularity of Perl has declined, while Python has become
      more popular. This directly influences the amount and quality of
      contributions that can be expected for programs written in the
      respective languages.
    </p>
    <p>
      The C language has served libvirt well over the years, but its age shows
      giving rise to limitations which negatively impact the project in terms
      of code quality, reliability, and efficiency of development. Most notably
      its lack of memory safety means that many code bugs become trivially
      exploitable security flaws or denial of service. The lack of a high
      level portable runtime results in a lot of effort being spent to
      ensure cross platform portability. The modern languages Rust and Go
      provide viable options for low level systems programming, in a way that
      is not practical with other common languages such as Python and Java.
      There is thus a desire to make use of either Rust or Go, or a combination
      of both, to incrementally replace existing use of C, and also for
      greenfield development.
    </p>
    <p>
      With this in mind the libvirt project has set a vision for language
      usage in the future:
    </p>

    <dl>
      <dt>C</dt>
      <dd>Large parts of the core libvirt library, daemons, and helper tools
        will continue to make use in the C language. Integration of other
        languages will be an incremental, targetted process where they can
        bring the greatest benefit.</dd>
      <dt>Rust / Go</dt>
      <dd>Parts of the core libvirt library, daemons and helper tools are to
        leverage Rust or Go or both to replace C.</dd>
      <dt>Meson</dt>
      <dd>The core build system is to be written in Meson.</dd>
      <dt>Python</dt>
      <dd>Various supporting build/test scripts are written in Python 3
        compatible mode only.</dd>
      <dt>reStructuredText</dt>
      <dd>The website and command man pages are to be written in RST, using
        Sphinx as the engine to convert to end user formats like HTML, troff,
        etc</dd>
    </dl>

    <p>
      Some notable points from the above. Whether the core library / daemons
      will use Rust or Go internally is still to be decided based on more
      detailed evaluation to identify the best fit. The need to link and embed
      this functionality in other processes has complex interactions both at a
      technical and non-technical level. For standalone helper tools, either
      language is viable, but there are fewer concerns around interactions with
      other in-process code from 3rd parties. Thus a different decision may be
      made for daemons/libraries vs tools. Any rewrite proposed for existing
      functionality will have to weigh up the benefits of the new code,
      against the risk of introducing regressions with respect to the previous
      code.
    </p>

    <p>
      The Meson build system is written in Python 3. This directly informs the
      choice of Python 3 as the language for all supporting build scripts,
      re-inforcing the other benefits of Python over Perl, Shell, M4,
      automake, etc. There is no intention to support Python 2 given Meson's
      requirement for Python 3.
    </p>

    <p>
      Using the RST format for documentation allows for the use of XSLT to be
      eliminated from the build process. RST and the Sphinx toolkit are widely
      used, as seen by the huge repository of content on
      <a href="https://readthedocs.org/">Read The Docs</a>.
      The ability to embed raw HTML in the RST docs will greatly facilitate its
      adoption, avoiding the need for a big bang conversion of existing content.
      Given the desire to eliminate Perl usage, replacing the use of POD
      documentation for manual pages is an obvious followup task. RST is the
      obvious choice to achieve alignment with the website, allowing the man
      pages to be easily published online with other docs. It is further
      anticipated that the current API docs generator which uses XSLT to
      convert the XML API description would be converted to something which
      generates RST using Python instead of XSLT.
    </p>
  </body>
</html>
