doxygen/testing

Doxygen regression test suite
=============================

This directory contains a set of regression tests. Each test consists of a
file starting with a 3 digit number and a corresponding directory whose name
has the same 3 digit number. The directory contains one or more reference
files that are compared against the XML output produced by doxygen. If the
result is the same, there is no regression and the test passes. If there is a
difference the test fails and the difference (in diff -u format) will be shown.
It is also possible to see whether or not the test can be built to a xhtml set
of files (and tested against a DTD), it is also possible to create a pdf file
for each test to see if the LaTeX / pdf generation is possible, similar for
rtf and docbook output.

The runtests.py script responsible for running the tests takes a number of
optional parameters:
  --updateref           update the reference files. Should be used in
                        combination with --id to update the reference file(s)
                        for the given test
  --doxygen [DOXYGEN]   path/name of the doxygen executable
  --xmllint [XMLLINT]   path/name of the xmllint executable
  --id IDS [IDS ...]    run test with number n only (the option can be
                        specified to run test with number n only (the option
                        can be specified multiple times
  --start_id START_ID   run tests starting with number n
  --end_id END_ID       run tests ending with number n
  --all                 can be used in combination with -updateref to update
                        the reference files for all tests.
  --inputdir [INPUTDIR]
                        input directory containing the tests
  --outputdir [OUTPUTDIR]
                        output directory to write the doxygen output to
  --noredir             disable redirection of doxygen warnings
  --pool [POOL]         pool size of multiprocess tests
  --xml                 create xml output and check
  --rtf                 create rtf output
  --docbook             create docbook output and check with xmllint
  --xhtml               create xhtml output and check with xmllint
  --xmlxsd              create xml output and check with xmllint against xsd
  --pdf                 create LaTeX output and create pdf from it
  --subdirs             use the configuration parameter CREATE_SUBDIRS=YES
  --clang               use CLANG_ASSISTED_PARSING, works only when doxygen
                        has been compiled with "use_libclang"
  --keep                keep result directories
  --cfg CFGS [CFGS ...]
                        run test with extra doxygen configuration settings
                        (the option may be specified multiple times
In case neither --xml, --xmlxsd, --pdf, --rtf, --docbook or --xhtml is used the
default is set to --xml.

The runtests.py has the following dependencies on 3rd party tools:
- python  to run the script
- xmllint to normalize the XML output
- diff    to show the differences in case a test fails

Each test file can have a number of special comment lines that are extracted by
the runtests.py script and take the form:
// <identifier>: 'argument'
Where <identifier> can be one of:
- objective: 'argument' provides the objective for the test (i.e. its purpose)
- check:     'argument' names a file that is generated by doxygen, which should
             be compared against the reference.
- config:    'argument' is a line that is added to the default Doxyfile used to
             run doxygen on the test file.

Example to run all tests:
    python runtests.py

Example to run a test
    python runtests.py --id 10

Example to update the reference files for a test
    python runtests.py --updateref --id 10

There is also a CMakeLists.txt, which can be used from the build directory
to run all tests by simply invoking 'make tests', to use the specific options use
the flag TEST_FLAGS with make
  e.g. make tests TEST_FLAGS="--id=5 --id=10 --pdf --xhtml"