Commit Graph

6871 Commits

Author SHA1 Message Date
Peter Maydell e75de8354a * qtest fixes (e.g. memory leaks)
* Fix for Xen dummy cpu loop (which happened due to qtest accel rework)
 * Introduction of the generic device fuzzer
 * Run more check-acceptance tests in the gitlab-CI
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAl+WmAwRHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbUGDA/+Ntog6jDcudxZwrPjB5GUiAyYOHo6MClc
 SdBw/abcs7G/EeGvIEexFLkuHTzroynbiE1l2RIwb5s8nB97mpXZYhT+cQ3qGUwg
 Uh4TAI0YM9GiEvhIOr9DdDj2XPF5yZ5YIzDSBk5lb5KHcgjrwj5P6EJs0k4RvX36
 5yJJeHPi0dDnfcfdowQ0zh7G/4drVLTS78gR+UKIYlbrZnMY2KK2tXwIi0ZE/QUK
 e4vnZAbAHFg6p7D/nw+DjDYUGr1ihFtiLLKueBFkpgGkMwFUYLkL548mwiL/ZFdT
 0ECiSVJUOFrT0nAbhUzrzBXEKXl6VpP66Dz62wRf8O+RRHihlWotij50ExtHUT+m
 6J1G3ycky4WQn4ORaKXG+MhkneiXiovoDEeFxBL7kX8VcTligdZzzjU6nOcE0LC3
 HOyCkocKy5u9sXi4rHIS+ui4FLpxZAf+pvYGdYVpekkFZvefeWYm+xUEIFWbTjfb
 vH7SeycAWG8KDhwm+DBhY/CC15sC4QGLndS7Kx4l7u9frZhacN48KnC2rfooku/q
 9+N/dSpo6Vj0jcq//ZQ9BIV/TkKtptpSKTm2Bd6kaidUpXrcst8CscYkBz7UUnjY
 7BcLk37i2yZPnwkGzAhqwgRlfFatDYW7jaWG7R0A7hEp2U3ujvOuxCCls6jcjW2I
 PDpsAlt6VnM=
 =ZpMy
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2020-10-26' into staging

* qtest fixes (e.g. memory leaks)
* Fix for Xen dummy cpu loop (which happened due to qtest accel rework)
* Introduction of the generic device fuzzer
* Run more check-acceptance tests in the gitlab-CI

# gpg: Signature made Mon 26 Oct 2020 09:34:04 GMT
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/huth-gitlab/tags/pull-request-2020-10-26: (31 commits)
  tests/acceptance: Use .ppm extention for Portable PixMap files
  tests/acceptance: Remove unused import
  test/docker/dockerfiles: Add missing packages for acceptance tests
  tests/acceptance: Enable AVOCADO_ALLOW_UNTRUSTED_CODE in the gitlab-CI
  test/acceptance: Remove the CONTINUOUS_INTEGRATION tags
  tests/acceptance/ppc_prep_40p: Fix the URL to the NetBSD-4.0 archive
  scripts/oss-fuzz: ignore the generic-fuzz target
  scripts/oss-fuzz: use hardlinks instead of copying
  fuzz: register predefined generic-fuzz configs
  fuzz: add generic-fuzz configs for oss-fuzz
  fuzz: add an "opaque" to the FuzzTarget struct
  fuzz: Add instructions for using generic-fuzz
  scripts/oss-fuzz: Add crash trace minimization script
  scripts/oss-fuzz: Add script to reorder a generic-fuzzer trace
  fuzz: add a crossover function to generic-fuzzer
  fuzz: add a DISABLE_PCI op to generic-fuzzer
  fuzz: Add support for custom crossover functions
  fuzz: Add fuzzer callbacks to DMA-read functions
  fuzz: Declare DMA Read callback function
  fuzz: Add DMA support to the generic-fuzzer
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-26 13:16:29 +00:00
Philippe Mathieu-Daudé 28bbe20ce2 tests/acceptance: Use .ppm extention for Portable PixMap files
The HMP 'screendump' command generates Portable PixMap files.
Make it obvious by using the .ppm file extention.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201021105035.2477784-3-f4bug@amsat.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:54:16 +01:00
Philippe Mathieu-Daudé 239f0d455b tests/acceptance: Remove unused import
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201021105035.2477784-2-f4bug@amsat.org>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:54:16 +01:00
Thomas Huth c4cb1c9f2e test/docker/dockerfiles: Add missing packages for acceptance tests
Some of the "check-acceptance" tests are still skipped in the CI
since the docker images do not provide the necessary packages, e.g.
the netcat binary. Add them to get more test coverage.

Message-Id: <20201023073351.251332-5-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:54:16 +01:00
Thomas Huth 1d60f46fc6 test/acceptance: Remove the CONTINUOUS_INTEGRATION tags
We are not running the acceptance tests on Travis anymore, so these
checks can be removed now.

Message-Id: <20201023073351.251332-3-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:54:16 +01:00
Thomas Huth a60f755c9c tests/acceptance/ppc_prep_40p: Fix the URL to the NetBSD-4.0 archive
The current URL on cdn.netbsd.org is failing - using archive.netbsd.org
instead seems to be fine.

Message-Id: <20201023073351.251332-2-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:54:16 +01:00
Alexander Bulekov 7fdb505384 fuzz: register predefined generic-fuzz configs
We call get_generic_fuzz_configs, which fills an array with
predefined {name, args, objects} triples. For each of these, we add a
new FuzzTarget, that uses a small wrapper to set
QEMU_FUZZ_{ARGS,OBJECTS} to the corresponding predefined values.

Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20201023150746.107063-16-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:53:54 +01:00
Alexander Bulekov 61fc27e0df fuzz: add generic-fuzz configs for oss-fuzz
Predefine some generic-fuzz configs. For each of these, we will create a
separate FuzzTarget that can be selected through argv0 and, therefore,
fuzzed on oss-fuzz.

Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20201023150746.107063-15-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:53:53 +01:00
Alexander Bulekov 82849bcf30 fuzz: add an "opaque" to the FuzzTarget struct
It can be useful to register FuzzTargets that have nearly-identical
initialization handlers (e.g. for using the same fuzzing code, with
different configuration options). Add an opaque pointer to the
FuzzTarget struct, so that FuzzTargets can hold some data, useful for
storing target-specific configuration options, that can be read by the
get_init_cmdline function.

Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20201023150746.107063-14-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:53:53 +01:00
Alexander Bulekov a253932227 fuzz: add a crossover function to generic-fuzzer
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20201023150746.107063-10-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:53:53 +01:00
Alexander Bulekov ccbd4bc8af fuzz: add a DISABLE_PCI op to generic-fuzzer
This new operation is used in the next commit, which concatenates two
fuzzer-generated inputs. With this operation, we can prevent the second
input from clobbering the PCI configuration performed by the first.

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Message-Id: <20201023150746.107063-9-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:53:53 +01:00
Alexander Bulekov f81cb729be fuzz: Add support for custom crossover functions
libfuzzer supports a "custom crossover function". Libfuzzer often tries
to blend two inputs to create a new interesting input. Sometimes, we
have a better idea about how to blend inputs together. This change
allows fuzzers to specify a custom function for blending two inputs
together.

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Message-Id: <20201023150746.107063-8-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:53:53 +01:00
Alexander Bulekov 20f5a30293 fuzz: Add DMA support to the generic-fuzzer
When a virtual-device tries to access some buffer in memory over DMA, we
add call-backs into the fuzzer(next commit). The fuzzer checks verifies
that the DMA request maps to a physical RAM address and fills the memory
with fuzzer-provided data. The patterns that we use to fill this memory
are specified using add_dma_pattern and clear_dma_patterns operations.

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Message-Id: <20201023150746.107063-5-alxndr@bu.edu>
[thuth: Reformatted one comment according to the QEMU coding style]
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:53:34 +01:00
Alexander Bulekov 05efbf2497 fuzz: Add PCI features to the generic fuzzer
This patch compares TYPE_PCI_DEVICE objects against the user-provided
matching pattern. If there is a match, we use some hacks and leverage
QOS to map each possible BAR for that device. Now fuzzed inputs might be
converted to pci_read/write commands which target specific. This means
that we can fuzz a particular device's PCI configuration space,

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Message-Id: <20201023150746.107063-4-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-24 07:43:48 +02:00
Alexander Bulekov da9bf53198 fuzz: Add generic virtual-device fuzzer
This is a generic fuzzer designed to fuzz a virtual device's
MemoryRegions, as long as they exist within the Memory or Port IO (if it
exists) AddressSpaces. The fuzzer's input is interpreted into a sequence
of qtest commands (outb, readw, etc). The interpreted commands are
separated by a magic seaparator, which should be easy for the fuzzer to
guess. Without ASan, the separator can be specified as a "dictionary
value" using the -dict argument (see libFuzzer documentation).

Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20201023150746.107063-3-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-24 07:43:48 +02:00
Maxim Levitsky d232b87ec6 libqtest: fix memory leak in the qtest_qmp_event_ref
The g_list_remove_link doesn't free the link element,
opposed to what I thought.
Switch to g_list_delete_link that does free it.

Also refactor the code a bit.
Thanks for Max Reitz for helping me with this.

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-Id: <20201019163702.471239-4-mlevitsk@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-24 07:36:53 +02:00
Maxim Levitsky 7f9d519c0d libqtest: fix the order of buffered events
By a mistake I added the pending events in a wrong order.
Fix this by using g_list_append.

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-Id: <20201019163702.471239-3-mlevitsk@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-24 07:36:53 +02:00
Havard Skinnemoen 288c31e30d tests/qtest: Make npcm7xx_timer-test conditional on CONFIG_NPCM7XX
This test won't work if qemu was compiled without CONFIG_NPCM7XX, as
pointed out by Thomas Huth on a different patch.

Signed-off-by: Havard Skinnemoen <hskinnemoen@google.com>
Message-Id: <20201023210637.351238-2-hskinnemoen@google.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-24 07:23:19 +02:00
Thomas Huth b7f47e82e2 tests/qtest/libqtest: Fix detection of architecture for binaries without path
The qtests can be run directly by specifying the QEMU binary with the
QTEST_QEMU_BINARY environment variable, for example:

 $ QTEST_QEMU_BINARY=x86_64-softmmu/qemu-system-x86_64 tests/qtest/test-hmp

However, if you specify a binary without a path, for example with
QTEST_QEMU_BINARY=qemu-system-x86_64 if the QEMU binary is in your
$PATH, then the test currently simply crashes.

Let's try a little bit smarter here by looking for the final '-'
instead of the slash.

Message-Id: <20201012114816.43546-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-24 07:23:19 +02:00
Chen Qun 9ab4fb21f5 tests/migration: fix memleak in wait_command/wait_command_fd
Properly free each command resp to avoid memory leak.
ASAN shows memory leak stack:

Indirect leak of 2352520 byte(s) in 571 object(s) allocated from:
    #0 0x7f6ca3308d4e in __interceptor_calloc (/lib64/libasan.so.5+0x112d4e)
    #1 0x7f6ca3127a50 in g_malloc0 (/lib64/libglib-2.0.so.0+0x55a50)
    #2 0x557bf3c71d2b in qdict_new ../qobject/qdict.c:29
    #3 0x557bf3c9caba in parse_object ../qobject/json-parser.c:318
    #4 0x557bf3c9ce75 in json_parser_parse ../qobject/json-parser.c:580
    #5 0x557bf3c8c8cf in json_message_process_token ../qobject/json-streamer.c:92
    #6 0x557bf3c9ea59 in json_lexer_feed_char ../qobject/json-lexer.c:313
    #7 0x557bf3c9eeb5 in json_lexer_feed ../qobject/json-lexer.c:350
    #8 0x557bf3c4793a in qmp_fd_receive ../tests/qtest/libqtest.c:608
    #9 0x557bf3c47b58 in qtest_qmp_receive ../tests/qtest/libqtest.c:618
    #10 0x557bf3c44245 in wait_command ../tests/qtest/migration-helpers.c:59
    #11 0x557bf3c445cb in migrate_query_status ../tests/qtest/migration-helpers.c:108
    #12 0x557bf3c44642 in check_migration_status ../tests/qtest/migration-helpers.c:124
    #13 0x557bf3c447e7 in wait_for_migration_status ../tests/qtest/migration-helpers.c:148
    #14 0x557bf3c43b8f in test_migrate_auto_converge ../tests/qtest/migration-test.c:1243
    ......

Fix: 5e34005571

Reported-by: Euler Robot <euler.robot@huawei.com>
Signed-off-by: Chen Qun <kuhn.chenqun@huawei.com>
Message-Id: <20201023061218.2080844-2-kuhn.chenqun@huawei.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-24 07:23:19 +02:00
Vladimir Sementsov-Ogievskiy 4f193168af iotests: add commit top->base cases to 274
These cases are fixed by previous patches around block_status and
is_allocated.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-id: 20200924194003.22080-6-vsementsov@virtuozzo.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-10-23 13:42:16 +01:00
Coiby Xu 049f55502a libvhost-user: Allow vu_message_read to be replaced
Allow vu_message_read to be replaced by one which will make use of the
QIOChannel functions. Thus reading vhost-user message won't stall the
guest. For slave channel, we still use the default vu_message_read.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Coiby Xu <coiby.xu@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20200918080912.321299-2-coiby.xu@gmail.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-10-23 13:42:16 +01:00
Peter Maydell 02aa56c4bc microvm: fix PCIe IRQs in APIC table.
microvm: add usb support.
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.22 (GNU/Linux)
 
 iQIcBAABCgAGBQJfkEogAAoJEEy22O7T6HE4yBMP/1csWFbfJ6JWbqiLoH/ltMn5
 ojgGfcP6ErGNtNP6kBO/FupCzc26ejRDJuvYLmlJ6+Jkga/zCAXjdLyXrSMRbtKK
 Nk8KwYSo82ApK45yJuVKZonUoGV6l9sIMyipkctlUXFMrvVaPh0MtnLIFckxtkCJ
 npAUnakkke5fP5Bsluw/N+fApAKQlPyqSBlPoPgsjNQ3ODisCitaH6Fx5sGuNJnV
 RSCAynkVkJ+vRT5iEojR6nNEn3vei8KHzL7vGxSdVd6ypjUc16ARpaPyXs2+opdP
 jqtGArrLn8V94rOPnGBwpV5P5soUch/tllCZtzbBZDaFr3bCHo0kNytxufoZ7yzM
 3z+wLgEHm3ZVRhraKE/Ed0Fz99tHm13zBWM3HctNtaY1tP7/4MnjMXRYArLQFFIt
 SoG7hSOPQJ/9JgtMPLRwjne2kHqYWwQ7wrROmnTWvx7RlfsD5JhUqMPocBnEb4pq
 kl5r6ajtZ9D67IZYKtv1JxKBdOXy7VmoEQMsZPwIxuMb5EN4N+RBqdMXs9lzzNEG
 mNCDzW5jDUlVG3YWu1Zxbs8j0+Mvlk7yD5zj368DvZZIgAMR0OxlcNbLVEFMpu8u
 pndpELjb/A/zHZloJyMXfCz19ahKxQeKjF1n4H5Ja6PvC8olALzhIy0ikOA66At0
 2I7Qs8bfGahjOdz4HSnJ
 =LkZR
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kraxel/tags/microvm-20201021-pull-request' into staging

microvm: fix PCIe IRQs in APIC table.
microvm: add usb support.

# gpg: Signature made Wed 21 Oct 2020 15:48:00 BST
# gpg:                using RSA key 4CB6D8EED3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>" [full]
# gpg:                 aka "Gerd Hoffmann <gerd@kraxel.org>" [full]
# gpg:                 aka "Gerd Hoffmann (private) <kraxel@gmail.com>" [full]
# Primary key fingerprint: A032 8CFF B93A 17A7 9901  FE7D 4CB6 D8EE D3E8 7138

* remotes/kraxel/tags/microvm-20201021-pull-request:
  tests/acpi: update expected data files
  tests/acpi: add microvm rtc test
  tests/acpi: add microvm usb test
  tests/acpi: add empty tests/data/acpi/microvm/DSDT.{usb, rtc} files
  tests/acpi: allow updates for expected data files
  microvm: add usb support
  usb/xhci: fixup xhci kconfig deps
  usb/xhci: add xhci_sysbus_build_aml() helper
  usb/xhci: add include/hw/usb/xhci.h header file
  acpi: add aml builder stubs
  tests/acpi: disallow changes for microvm/APIC.pcie
  tests/acpi: update expected data files
  apci: drop has_pci arg for acpi_build_madt
  microvm: set pci_irq_mask
  x86: make pci irqs runtime configurable
  tests/acpi: add empty microvm/APIC.pcie
  tests/acpi: allow changes for microvm/APIC.pcie

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-21 21:45:46 +01:00
Peter Maydell 67e8498937 Pull request
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+ber27ys35W+dsvQfe+BBqr8OQ4FAl+PNNYACgkQfe+BBqr8
 OQ5WHA/8CxKN5vHw1Ob4rAc1vjp58plwgpFOnwOUlw11FQ5+qJhSFvWjIaqFMzJ5
 tCs6Qus+k2arpcFY/RcTHEkNfJR7t8o4u62cZh+CzcpxpaTleM7Ru32xaiiw1dKH
 3ujfjnT4U9dTDGg6vqQeaZj9RBSKd3cFCzzupO6iho9md3ZCJXsO05w0SNyAe1Ct
 o7y0jHAxJMRQJ4kDtp8e2F9w+jNP2zLQZxPB3Af3SF0CTkXymuEv+5Jv+aoJ5Z9d
 tZxGF8imi9Gq+JSMxp7Oatw5RMBqCKEqr81cddbiPafisNBQJ4fR/HiPxzGjNTIp
 TW61duK/hLMbQk7my6bmGr2FOAo4Xw2t3e0pGORi20W19Su/us07KwFx81Ls6Wok
 3n/biFi6pWeFwDXO8nsgs4eJEcD0blrEqDbjLrMhcKgnFUGJTIbnZ+AIjpNy8OLZ
 edT5DZRymIROiPopBSsvUM9KYjo9kvoFodPl9RtiWFvjrkw1gyqnUAaa10xKpZ0s
 WKFrktFoPXOyNjlmZO936X2jKuGGv7+yYBLGKNpUOSQu8P4Jan8GEGmOtHZAWKMH
 xRnDSEfkBQPRmSriI0je7P3WVqlwqBzVH+a4P0d4b00LKVLiKtYAqO/x3UcQ20CQ
 AlBglcRL96hxm+capuGOmm79Nl5+u3dxwpQZEBVa2M65kbbXdZ4=
 =iStV
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/jsnow-gitlab/tags/python-pull-request' into staging

Pull request

# gpg: Signature made Tue 20 Oct 2020 20:04:54 BST
# gpg:                using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full]
# Primary key fingerprint: FAEB 9711 A12C F475 812F  18F2 88A9 064D 1835 61EB
#      Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76  CBD0 7DEF 8106 AAFC 390E

* remotes/jsnow-gitlab/tags/python-pull-request: (21 commits)
  python/qemu/qmp.py: Fix settimeout operation
  python/qemu/qmp.py: re-raise OSError when encountered
  python: add mypy config
  python/qemu/qmp.py: Preserve error context on re-raise
  python/qemu/console_socket.py: avoid encoding to/from string
  python/qemu/console_socket.py: Add type hint annotations
  python/qemu/console_socket.py: Clarify type of drain_thread
  python/qemu/console_socket.py: fix typing of settimeout
  python/qemu/console_socket.py: Correct type of recv()
  python/qemu: Add mypy type annotations
  iotests.py: Adjust HMP kwargs typing
  python/qemu: make 'args' style arguments immutable
  python/machine.py: fix _popen access
  python/machine.py: Add _qmp access shim
  python/machine.py: use qmp.command
  python/machine.py: Handle None events in events_wait
  python/machine.py: Don't modify state in _base_args()
  python/machine.py: reorder __init__
  python/machine.py: Fix monitor address typing
  python/qemu: use isort to lay out imports
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-21 11:09:13 +01:00
Gerd Hoffmann 66907f3d3b tests/acpi: update expected data files
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-id: 20201020074844.5304-11-kraxel@redhat.com
2020-10-21 11:36:19 +02:00
Gerd Hoffmann 0c491c8485 tests/acpi: add microvm rtc test
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-id: 20201020074844.5304-10-kraxel@redhat.com
2020-10-21 11:36:19 +02:00
Gerd Hoffmann f072fd2f85 tests/acpi: add microvm usb test
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-id: 20201020074844.5304-9-kraxel@redhat.com
2020-10-21 11:36:19 +02:00
Gerd Hoffmann d9b68f1add tests/acpi: add empty tests/data/acpi/microvm/DSDT.{usb, rtc} files
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-id: 20201020074844.5304-8-kraxel@redhat.com
2020-10-21 11:36:19 +02:00
Gerd Hoffmann 702ef9b6ae tests/acpi: allow updates for expected data files
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-id: 20201020074844.5304-7-kraxel@redhat.com
2020-10-21 11:36:19 +02:00
Gerd Hoffmann 0a10b8f718 tests/acpi: disallow changes for microvm/APIC.pcie
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-id: 20201016113835.17465-8-kraxel@redhat.com
2020-10-21 11:36:05 +02:00
Gerd Hoffmann d3b5a15f8d tests/acpi: update expected data files
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-id: 20201016113835.17465-7-kraxel@redhat.com
2020-10-21 11:36:05 +02:00
Gerd Hoffmann e6b5a0718a tests/acpi: add empty microvm/APIC.pcie
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-id: 20201016113835.17465-3-kraxel@redhat.com
2020-10-21 11:36:05 +02:00
Gerd Hoffmann 129927fdae tests/acpi: allow changes for microvm/APIC.pcie
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-id: 20201016113835.17465-2-kraxel@redhat.com
2020-10-21 11:36:05 +02:00
Havard Skinnemoen 19d50149c8 tests/qtest: Add npcm7xx timer test
This test exercises the various modes of the npcm7xx timer. In
particular, it triggers the bug found by the fuzzer, as reported here:

https://lists.gnu.org/archive/html/qemu-devel/2020-09/msg02992.html

It also found several other bugs, especially related to interrupt
handling.

The test exercises all the timers in all the timer modules, which
expands to 180 test cases in total.

Reviewed-by: Tyrone Ting <kfting@nuvoton.com>
Signed-off-by: Havard Skinnemoen <hskinnemoen@google.com>
Message-id: 20201008232154.94221-2-hskinnemoen@google.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-20 16:12:00 +01:00
John Snow 090744d552 iotests.py: Adjust HMP kwargs typing
mypy wants to ensure there's consistency between the kwargs arguments
types and any unspecified keyword arguments. In this case, conv_keys is
a bool, but the remaining keys are Any type. Mypy (correctly) infers the
**kwargs type to be **Dict[str, str], which is not compatible with
conv_keys: bool.

Because QMP typing is a little fraught right now anyway, re-type kwargs
to Dict[str, Any] which has the benefit of silencing this check right
now.

A future re-design might type these more aggressively, but this will
give us a baseline to work from with minimal disruption.

(Thanks Kevin Wolf for the debugging assist here)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20201006235817.3280413-11-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20 09:37:57 -04:00
Christian Schoenebeck 653daf3897 tests/9pfs: add local Tmkdir test
This test case uses the 9pfs 'local' driver to create a directory
and then checks if the expected directory was actually created
(as real directory) on host side.

This patch introduces a custom split() implementation, because
the test code requires non empty array elements as result. For
that reason g_strsplit() would not be a good alternative, as
it would require additional filter code for reshuffling the
array, and the resulting code would be even more complex than
this split() function.

Signed-off-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
Message-Id: <be342f236842272275f65dbe05587f0a5409ad77.1602182956.git.qemu_oss@crudebyte.com>
Signed-off-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
2020-10-19 14:25:40 +02:00
Christian Schoenebeck e55b178b5e tests/9pfs: add virtio_9p_test_path()
This new public function virtio_9p_test_path() allows 9pfs
'local' tests to translate a path from guest scope to host
scope. For instance by passing an empty string it would
return the root path on host of the exported 9pfs tree.

Signed-off-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
Message-Id: <b563d3c73c6391ec927a2622c9f65c09ca56bd83.1602182956.git.qemu_oss@crudebyte.com>
Signed-off-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
2020-10-19 14:25:40 +02:00
Christian Schoenebeck 051f0e5be1 tests/9pfs: wipe local 9pfs test directory
Before running the first 9pfs test case, make sure the test directory
for running the 9pfs 'local' tests on is entirely empty. For that
reason simply delete the test directory (if any) before (re)creating
it on test suite startup.

Note: The preferable precise behaviour would be the test directory
only being wiped once *before* a test suite run. Right now the test
directory is also wiped at the *end* of a test suite run because
libqos is calling the virtio_9p_register_nodes() callback for some
reason also when a test suite completed. This is suboptimal as
developers cannot immediately see what files and directories the
9pfs local tests created precisely after the test suite completed.
But fortunately the test directory is not wiped if some test failed.
So it is probably not worth it drilling another hole into libqos
for this issue.

Signed-off-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
Message-Id: <b30776ea3289dc40dabc7d0063d825d21d9a65bf.1602182956.git.qemu_oss@crudebyte.com>
Signed-off-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
2020-10-19 14:25:40 +02:00
Christian Schoenebeck 3a565c641a tests/9pfs: introduce local tests
This patch introduces 9pfs test cases using the 9pfs 'local'
filesystem driver which reads/writes/creates/deletes real files
and directories.

In this initial version, there is only one local test which actually
only checks if the 9pfs 'local' device was created successfully.

Before the 9pfs 'local' tests are run, a test directory 'qtest-9p-local'
is created (with world rwx permissions) under the current working
directory. At this point that test directory is not auto deleted yet.

Signed-off-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
Message-Id: <81fc4b3b6b6c9bf7999e79f5e7cbc364a5f09ddb.1602182956.git.qemu_oss@crudebyte.com>
Signed-off-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
2020-10-19 14:25:40 +02:00
Christian Schoenebeck eefd2394ef tests/9pfs: change qtest name prefix to synth
All existing 9pfs test cases are using the 'synth' fs driver so far, which
means they are not accessing real files, but a purely simulated (in RAM
only) file system.

Let's make this clear by changing the prefix of the individual qtest case
names from 'fs/' to 'synth/'. That way they'll be easily distinguishable
from upcoming new 9pfs test cases supposed to be using a different fs
driver.

Signed-off-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
Message-Id: <e04e75acb849b085c6d6320b2433a15fa935bcff.1602182956.git.qemu_oss@crudebyte.com>
Signed-off-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
2020-10-19 14:25:40 +02:00
Peter Maydell 22d30b340a MIPS patches queue
. Fix some comment spelling errors
 . Demacro some TCG helpers
 . Add loongson-ext lswc2/lsdc2 group of instructions
 . Log unimplemented cache opcode
 . Increase number of TLB entries on the 34Kf core
 . Allow the CPU to use dynamic frequencies
 . Calculate the CP0 timer period using the CPU frequency
 . Set CPU frequency for each machine
 . Fix Malta FPGA I/O region size
 . Allow running qtests when ROM is missing
 . Add record/replay acceptance tests
 . Update MIPS CPU documentation
 . MAINTAINERS updates
 
 CI jobs results:
   https://gitlab.com/philmd/qemu/-/pipelines/203931842
   https://travis-ci.org/github/philmd/qemu/builds/736491461
   https://cirrus-ci.com/build/6272264062631936
   https://app.shippable.com/github/philmd/qemu/runs/886/summary/console
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+qvnXhKRciHc/Wuy4+MsLN6twN4FAl+K+NkACgkQ4+MsLN6t
 wN4oPRAArZ5v0fjGrylt9g4xCAygLoSMkH3sZxltB77UVN/dCawSTLK2seKKO5g/
 UtGt/j4/OAt8Ms+nF8FT+UZbknkgq+h7coOorHvz6gDEAx9UIg6/S2TRZJEx28+l
 LbzkqdxvNSoHRrQDpGo43xoaxjzCxSTSOKpPfje6p2YDxWjkxdr/ahcsbKHSKc+x
 uGdVdEAlLiAs/fBhkaJD3yy1VfqJKu8V5JJo1g4gSQOD1worRbZ4Us9QfuYr79Q7
 Kce1Z1MQSf/TceZuDubhzZBep5lF1uW4lTywcaDby0LvGNK4K+RnH+i+t7CNhtKs
 LH5j6iFQY1ecjb1Vh0IgKNAFaM2sTtO7A6fbBSOkVTO60wEp7i9fpbI5TRIjv7z/
 EBkzP3n00hhbFFDci6Lnh/Ko0Xy0ODe3Um5l410sTnJe9+LK0HR5V6WH8PD/wKV2
 nnKzSgb1U51KS6+FzLGLbQzDEvCgRKAJ9mwiQ+dlRfFHj+rEM6a9rlQmtsADBhKi
 sEx62BKe6mM/+qQL9AOwZ5xBmFAn6wquuLYoA2Bwfg0wPIiAiFTwrz/eVSm9qYsw
 O9Fer+1IMmd06T1REUtSDAh8+D2ekknKmFA3AG0818WvluD0Qm3KZp8uLLHJ/XkO
 jiRtmeW+hApeh8hP4E0bzmrfJPKseBCYYP1By7XavIOoCxlqhew=
 =BOxw
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/mips-next-20201017' into staging

MIPS patches queue

. Fix some comment spelling errors
. Demacro some TCG helpers
. Add loongson-ext lswc2/lsdc2 group of instructions
. Log unimplemented cache opcode
. Increase number of TLB entries on the 34Kf core
. Allow the CPU to use dynamic frequencies
. Calculate the CP0 timer period using the CPU frequency
. Set CPU frequency for each machine
. Fix Malta FPGA I/O region size
. Allow running qtests when ROM is missing
. Add record/replay acceptance tests
. Update MIPS CPU documentation
. MAINTAINERS updates

CI jobs results:
  https://gitlab.com/philmd/qemu/-/pipelines/203931842
  https://travis-ci.org/github/philmd/qemu/builds/736491461
  https://cirrus-ci.com/build/6272264062631936
  https://app.shippable.com/github/philmd/qemu/runs/886/summary/console

# gpg: Signature made Sat 17 Oct 2020 14:59:53 BST
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD  6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/mips-next-20201017: (44 commits)
  target/mips: Increase number of TLB entries on the 34Kf core (16 -> 64)
  MAINTAINERS: Remove duplicated Malta test entries
  MAINTAINERS: Downgrade MIPS Boston to 'Odd Fixes', fix Paul Burton mail
  MAINTAINERS: Put myself forward for MIPS target
  MAINTAINERS: Remove myself
  docs/system: Update MIPS CPU documentation
  tests/acceptance: Add MIPS record/replay tests
  hw/mips: Remove exit(1) in case of missing ROM
  hw/mips: Rename TYPE_MIPS_BOSTON to TYPE_BOSTON
  hw/mips: Simplify code using ROUND_UP(INITRD_PAGE_SIZE)
  hw/mips: Simplify loading 64-bit ELF kernels
  hw/mips/malta: Use clearer qdev style
  hw/mips/malta: Move gt64120 related code together
  hw/mips/malta: Fix FPGA I/O region size
  target/mips/cpu: Display warning when CPU is used without input clock
  hw/mips/cps: Do not allow use without input clock
  hw/mips/malta: Set CPU frequency to 320 MHz
  hw/mips/boston: Set CPU frequency to 1 GHz
  hw/mips/cps: Expose input clock and connect it to CPU cores
  hw/mips/jazz: Correct CPU frequencies
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-19 10:52:57 +01:00
Peter Maydell 782d7b30dd * Drop ninjatool and just require ninja (Paolo)
* Fix docs build under msys2 (Yonggang)
 * HAX snafu fix (Claudio)
 * Disable signal handlers during fuzzing (Alex)
 * Miscellaneous fixes (Bruce, Greg)
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAl+LA6QUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroPa0wgAk6DqUR/eSv2tmwd6EiSLi+ngnA2q
 WcqbsEe8DQRZeNWB8WQHTp3g6DdwMjhCdn2KBZmT51g1pdyVoqu0wiW1lfb/plRR
 2aCcpMt++X4s8KdrILQ+gR7ZOR88GT6Mbm+9tt8MlYJu8aC8KW4wLMr/wAorNe+i
 iEMweEK0cw+TFYdb/cQV3vRgfp/DyiRGGG9MHrYa6eygkDqA1YBffTjlybivibjg
 WxRFtSd8pqFGyVsPsnRk0MX3+MY1U4Ah2Film/PukW6c2K0IOHR2zVRJOOR+Zd2E
 cCD3w+26/akivI01yH1V2kOzfsV2NipOtD7YW1DTMl5FFTSAKvXLzFibpA==
 =uWTa
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging

* Drop ninjatool and just require ninja (Paolo)
* Fix docs build under msys2 (Yonggang)
* HAX snafu fix (Claudio)
* Disable signal handlers during fuzzing (Alex)
* Miscellaneous fixes (Bruce, Greg)

# gpg: Signature made Sat 17 Oct 2020 15:45:56 BST
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini-gitlab/tags/for-upstream: (22 commits)
  ci: include configure and meson logs in all jobs if configure fails
  hax: unbreak accelerator cpu code after cpus.c split
  fuzz: Disable QEMU's SIG{INT,HUP,TERM} handlers
  cirrus: Enable doc build on msys2/mingw
  meson: Move the detection logic for sphinx to meson
  meson: move SPHINX_ARGS references within "if build_docs"
  docs: Fix Sphinx configuration for msys2/mingw
  meson: Only install icons and qemu.desktop if have_system
  configure: fix handling of --docdir parameter
  meson: cleanup curses/iconv test
  meson.build: don't condition iconv detection on library detection
  build: add --enable/--disable-libudev
  build: replace ninjatool with ninja
  build: cleanups to Makefile
  add ninja to dockerfiles, CI configurations and test VMs
  dockerfiles: enable Centos 8 PowerTools
  configure: move QEMU_INCLUDES to meson
  tests: add missing generated sources to testqapi
  make: run shell with pipefail
  tests/Makefile.include: unbreak non-tcg builds
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-17 20:52:55 +01:00
Alexander Bulekov fc69fa216c fuzz: Disable QEMU's SIG{INT,HUP,TERM} handlers
Prior to this patch, the only way I found to terminate the fuzzer was
either to:
 1. Explicitly specify the number of fuzzer runs with the -runs= flag
 2. SIGKILL the process with "pkill -9 qemu-fuzz-*" or similar

In addition to being annoying to deal with, SIGKILLing the process skips
over any exit handlers(e.g. registered with atexit()). This is bad,
since some fuzzers might create temporary files that should ideally be
removed on exit using an exit handler. The only way to achieve a clean
exit now is to specify -runs=N , but the desired "N" is tricky to
identify prior to fuzzing.

Why doesn't the process exit with standard SIGINT,SIGHUP,SIGTERM
signals? QEMU installs its own handlers for these signals in
os-posix.c:os_setup_signal_handling, which notify the main loop that an
exit was requested. The fuzzer, however, does not run qemu_main_loop,
which performs the main_loop_should_exit() check.  This means that the
fuzzer effectively ignores these signals. As we don't really care about
cleanly stopping the disposable fuzzer "VM", this patch uninstalls
QEMU's signal handlers. Thus, we can stop the fuzzer with
SIG{INT,HUP,TERM} and the fuzzing code can optionally use atexit() to
clean up temporary files/resources.

Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20201014142157.46028-1-alxndr@bu.edu>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-17 10:45:53 -04:00
Paolo Bonzini 9dc6ee3fd7 meson: move SPHINX_ARGS references within "if build_docs"
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-17 10:45:52 -04:00
Paolo Bonzini 09e93326e4 build: replace ninjatool with ninja
Now that the build is done entirely by Meson, there is no need
to keep the Makefile conversion.  Instead, we can ask Ninja about
the targets it exposes and forward them.

The main advantages are, from smallest to largest:

- reducing the possible namespace pollution within the Makefile

- removal of a relatively large Python program

- faster build because parsing Makefile.ninja is slower than
parsing build.ninja; and faster build after Meson runs because
we do not have to generate Makefile.ninja.

- tracking of command lines, which provides more accurate rebuilds

In addition the change removes the requirement for GNU make 3.82, which
was annoying on Mac, and avoids bugs on Windows due to ninjatool not
knowing how to convert Windows escapes to POSIX escapes.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-17 10:45:51 -04:00
Paolo Bonzini 345d7053ca add ninja to dockerfiles, CI configurations and test VMs
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-17 10:45:50 -04:00
Paolo Bonzini f2f984a3b3 dockerfiles: enable Centos 8 PowerTools
ninja is included in the CentOS PowerTools repository.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-17 10:45:50 -04:00
Paolo Bonzini 97d6efd0a3 tests: add missing generated sources to testqapi
Ninja notices them due to a different order in visiting the graph.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-17 10:45:50 -04:00
Paolo Bonzini 88da4b043b tests/Makefile.include: unbreak non-tcg builds
Remove from check-block the requirement that all TARGET_DIRS are built.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-17 10:45:49 -04:00
Pavel Dovgalyuk 1d2ff14b72 tests/acceptance: Add MIPS record/replay tests
This patch adds MIPS-targeted acceptance tests for
record/replay functions.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <160276110297.2705.10918105269658307206.stgit@pasha-ThinkPad-X280>
[PMD: Moved 'override timeout' comment from instance to class,
      moved nanomips tests to ReplayKernelSlow class,
      tagged ReplayKernelSlow class with AVOCADO_TIMEOUT_EXPECTED]
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2020-10-17 13:59:40 +02:00