Commit Graph

189 Commits

Author SHA1 Message Date
Peter Krempa aea559fa74 virstoragetest: Drop testing of NBD backends via parsing real images
We now have specific tests for the backing store parser and previous
tests cover the extraction of the backing store string so there's no
need for these particular tests.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2021-09-09 15:29:00 +02:00
Peter Krempa 97d7177a11 virstoragetest: Drop testing of RBD backends via parsing real images
We now have specific tests for the backing store parser and previous
tests cover the extraction of the backing store string so there's no
need for these particular tests.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2021-09-09 15:29:00 +02:00
Ján Tomko 979d1ba3ae tests: virstoragetest: remove tests without backing type
As of qemu commit:

  commit 497a30dbb065937d67f6c43af6dd78492e1d6f6d
    qemu-img: Require -F with -b backing image

creating images with backing images requires specifying the format.

Remove tests which do not pass the backing format on the command
line.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2021-08-31 16:49:03 +02:00
Michal Privoznik c8238579fb lib: Drop internal virXXXPtr typedefs
Historically, we declared pointer type to our types:

  typedef struct _virXXX virXXX;
  typedef virXXX *virXXXPtr;

But usefulness of such declaration is questionable, at best.
Unfortunately, we can't drop every such declaration - we have to
carry some over, because they are part of public API (e.g.
virDomainPtr). But for internal types - we can do drop them and
use what every other C project uses 'virXXX *'.

This change was generated by a very ugly shell script that
generated sed script which was then called over each file in the
repository. For the shell script refer to the cover letter:

https://listman.redhat.com/archives/libvir-list/2021-March/msg00537.html

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2021-04-13 17:00:38 +02:00
Peter Krempa 887d747dbe tests: Remove testing of virStorageFileCanonicalizePath
Remove the last code using the function.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2021-04-12 15:55:09 +02:00
Peter Krempa a43c8763bf virStorageSourceGetMetadata: Use depth limit instead of unique path checking
Prevent unbounded chains by limiting the recursion depth of
virStorageSourceGetMetadataRecurse to the maximum number of image layers
we limit anyways.

This removes the last use of virStorageSourceGetUniqueIdentifier which
will allow us to delete some crusty old infrastructure which isn't
really needed.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2021-04-12 15:55:09 +02:00
Michal Privoznik 7f482a67e4 lib: Replace virFileMakePath() with g_mkdir_with_parents()
Generated using the following spatch:

  @@
  expression path;
  @@
  - virFileMakePath(path)
  + g_mkdir_with_parents(path, 0777)

However, 14 occurrences were not replaced, e.g. in
virHostdevManagerNew(). I don't really understand why.
Fixed by hand afterwards.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2021-03-04 20:52:23 +01:00
Peter Krempa ab82d41f41 tests: storage: Replace index testing in testStorageLookup
Test the actual index in the returned virStorageSource rather than the
parsed one. Some tests need to be adapted as they were on failed lookup.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
2021-01-27 07:49:58 +01:00
Peter Krempa 8fd72501c8 virStorageSourceChainLookup: Handle names like 'vda[4]' internally
All callers of this function called virStorageFileParseChainIndex
before. Internalize the logic of that function to prevent multiple calls
and passing around unnecessary temporary variables.

This is achieved by calling virStorageFileParseBackingStoreStr and using
it to fill the values internally.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
2021-01-27 07:49:57 +01:00
Peter Krempa 49c89fa70e test: storage: Remove double testing in testStorageLookup
The function attempts two calls to virStorageSourceChainLookup to see
whether the function handles NULL correctly. This isn't very useful and
additionally upcoming patch will remove the 'idx' parameter thus the
test becomes obsolete. Remove it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
2021-01-27 07:49:57 +01:00
Pavel Hrdina 836e0a960b storage_source: use virStorageSource prefix for all functions
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2021-01-22 11:10:27 +01:00
Pavel Hrdina 01f7ade912 util: extract virStorageFile code into storage_source
Up until now we had a runtime code and XML related code in the same
source file inside util directory.

This patch takes the runtime part and extracts it into the new
storage_file directory.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2021-01-22 11:10:27 +01:00
Ryan Gahagan 0f1f3f1228 util: virstoragefile: Add 'json:' pseudo-protocol parser for 'nfs' protocol
Enable parsing of backing store strings containing the native 'nfs'
protocol specification.

Signed-off-by: Ryan Gahagan <rgahagan@cs.utexas.edu>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2021-01-08 15:09:26 +01:00
Pavel Hrdina ba9b419910 virstoragefile: remove unused virStorageFileChainCheckBroken
The last usage outside of tests was removed by commit
<780f8c94ca8b3dee7eb59c1bfbc32f672f965df8>.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2021-01-06 13:15:17 +01:00
Peter Krempa bc3a78f61a virStorageSourceNew: Abort on failure
Add an abort() on the class/object allocation failures so that
virStorageSourceNew() always returns a virStorageSource and remove
checks from all callers.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-09-23 22:37:56 +02:00
Michal Privoznik 95b9db4ee2 lib: Prefer WITH_* prefix for #if conditionals
Currently, we are mixing: #if HAVE_BLAH with #if WITH_BLAH.
Things got way better with Pavel's work on meson, but apparently,
mixing these two lead to confusing and easy to miss bugs (see
31fb929eca for instance). While we were forced to use HAVE_
prefix with autotools, we are free to chose our own prefix with
meson and since WITH_ prefix appears to be more popular let's use
it everywhere.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-09-02 10:28:10 +02:00
Ján Tomko fd7644cba9 tests: use VIR_WARNINGS_NO_DECLARATION_AFTER_STATEMENT
Some test rely too much on declaring variables in the middle
of the function. Use the macro to locally suppress the warning

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-08-25 19:03:13 +02:00
Peter Krempa 6bde2a1e20 conf: Sanitize handling of <auth> and <encryption> placement for disks
Modern way to store <auth> and <encryption> of a <disk> is under
<source>. This was added to mirror how <backingStore> handles these and
in fact they are relevant to the source rather than to any other part of
the disk. Historically we allowed them to be directly under <disk> and
we need to keep compatibility.

This wasn't a problem until introduction of -blockdev in qemu using of
<auth> or <encryption> plainly wouldn't work with backing chains.

Now that it works in backing chains and can be moved back and forth
using snapshots/block-commit we need to ensure that the original
placement is properly kept even if the source changes.

To achieve the above semantics we need to store the preferred placement
with the disk definition rather than the storage source definitions and
also ensure that the modern way is chosen when the VM started with
<source/encryption> only in the backing store.

https://bugzilla.redhat.com/show_bug.cgi?id=1822878

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-05-12 06:55:00 +02:00
Peter Krempa f8d6b319a6 virStorageSourceParseNBDColonString: Rewrite to match what qemu does
Our implementation wasn't quite able to parse everything that qemu does.
This patch rewrites the parser to a code that semantically resembles the
combination of 'nbd_parse_filename' and 'inet_parse' methods in qemu to
be able to parse the strings in an equivalent manner.

The only thing that libvirt doesn't do is to check the lengths of
various components in the nbd string in places where qemu uses constant
size buffers.

The test cases validate that some of the corner cases involving colons
are parsed properly.

https://bugzilla.redhat.com/show_bug.cgi?id=1826652

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-04-27 08:05:49 +02:00
Peter Krempa 524de6cc35 virstoragetest: testBackingParse: Use VIR_DOMAIN_DEF_FORMAT_SECURE when formatting xml
We want to format even the secure information in tests.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2020-04-14 18:47:01 +02:00
Peter Krempa 544ef82d05 virStorageSourceParseBackingURI: Preserve query string of URI for http(s)
For http/https URIs we need to preserve the query part as it may be
important to refer to the image.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-30 16:30:34 +02:00
Peter Krempa 1b84dd190c storage: Parse 'nvme' disk source properties from json:{} pseudo-uri
Our code allows snapshots of NVMe based disks which means we create
overlay file with a 'json:{}' pseudo-uri refering to the NVME device.
Our parser code doesn't handle them though. Add the parser and test it
via the XML->json->XML round-trip and reference data.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-24 14:17:48 +01:00
Peter Krempa 5a70f1048f storage: Implement backing store support for "fat:" prefix
qemublocktest showed that we don't add the "fat:" prefix for directory
storage when formatting the backing store string. While it's unlikely to
be used it's simple enough to actually implement the support rather than
trying to forbid it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-24 14:17:47 +01:00
Peter Krempa 096e94cef6 tests: virstoragetest: validate that array deflattening works for gluster
Validate that we are able to parse back the dotted syntax arrays we were
generating in the pre-blockdev era.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-20 09:47:16 +01:00
Peter Krempa 5179cc6b08 virStorageSourceParseBackingJSONUri: Handle undocumented value 'off' for sslverify
libguestfs abuses a quirk of qemu's parser to accept also other variants
of the 'sslverify' field which would be valid on the command line but
are not documented in the QMP schema.

If we encounter the 'off' string instead of an boolean handle it rather
than erroring out to continue support of pre-blockdev configurations.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa 77194db01c virstoragefile: Add JSON parser for 'sslverify', 'readahead', 'cookies' and 'timeout'
Add support for parsing the recently added fields from backing file
pseudo-protocol strings.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Ján Tomko abd68b2bdb Include unistd.h where used
Include unistd.h in all files that use it, instead
of relying on it being pulled in via virutil.h

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2020-02-24 23:15:50 +01:00
Peter Krempa e5c8f6e080 tests: virstorage: Fix backing file format of created image
We create some images for testing our code. We've recorded wrong format
of the backing file for one of the images though.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-24 15:12:32 +01:00
Peter Krempa e8a819e87f virStorageSourceParseBackingJSONRaw: Parse 'offset' and 'size' attributes
If the parsed 'raw' format JSON string has 'offset' or 'size' attributes
parse them as the format slice.

https://bugzilla.redhat.com/show_bug.cgi?id=1791788

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa 554ae62637 tests: virstorage: Add test data for json specified raw image with offset/size
QEMU allows specifying the offset and size into a raw file to expose a
sub-slice of the image to the guest with the raw driver. Libvirt
currently doesn't support it but we can add test case for future
reference.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa 0d0d60ddc5 tests: virstorage: Add test cases for "json:" pseudo-URI without 'file' wrapper
Add few cases that prove the second format of "json:" pseudo-URIs.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:27 +01:00
Peter Krempa 2775aada16 tests: virstorage: Add tests for NBD URI style syntax over UNIX
Add few test cases for nbd+unix style URIs with few corner cases.

The NBD URI syntax is documented at
https://github.com/NetworkBlockDevice/nbd/blob/master/doc/uri.md

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-01-17 12:34:10 +01:00
Peter Krempa 5f2fa393f7 util: storagefile: Properly set transport type when parsing NBD strings
When parsing legacy NBD backing file strings such as
'nbd:unix:/tmp/sock:exportname=/' we'd fail to set the transport to
VIR_STORAGE_NET_HOST_TRANS_UNIX. This started to be a problem once we
actually started to generate config of the backing store on the command
line with -blockdev as the JSON code would try to format it as TCP and
fail with:

 internal error: argument key 'host' must not have null value

Set the type properly and add a test.

This bug was found by the libguestfs test suite in:

https://bugzilla.redhat.com/show_bug.cgi?id=1791614

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reported-by: Ming Xie <mxie@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
2020-01-17 12:34:10 +01:00
Peter Krempa 3f2d167d9c conf: Always format storage source auth and encryption under <source> for backing files
Historically there are two places where we format authentication and
encryption for a disk. The logich which formats it for backing files was
flawed though and didn't format it at all. This worked if the image
became a backing file through the means of a snapshot but not directly.

Force formatting of the source and encryption for any non-disk case to
fix the issue.

This caused problems in many places as we use the formatter to copy the
definition. Effectively any copy lost the secret definition.

https://bugzilla.redhat.com/show_bug.cgi?id=1789310
https://bugzilla.redhat.com/show_bug.cgi?id=1788898

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
2020-01-13 12:53:58 +01:00
Daniel P. Berrangé 8812163124 src: remove unused imports of dirname.h
A few places were importing dirname.h without actually using it.

Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-03 15:42:13 +00:00
Peter Krempa 3615e8b39b util: storage: Don't treat files with missing backing store format as 'raw'
Assuming that the backing image format is raw is wrong when doing image
detection:

1) In -drive mode qemu will still probe the image format of the backing
   image. This means it will try to open a backing file of the image
   which will fail if a more advanced security model is in use.

2) In blockdev mode the image will be opened as raw actually which is
   wrong since it might be qcow. Not opening the backing images will
   also end up in the guest seeing corrupted data.

Rather than attempt to solve various corner cases when us assuming the
storage file being raw and actually being right forbid startup when the
guest image doesn't have the format specified in the metadata.

https://bugzilla.redhat.com/show_bug.cgi?id=1588373

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-18 09:36:48 +01:00
Peter Krempa a649369480 tests: storage: Remove unused test modes
EXP_WARN and ALLOW_PROBE flags for the testStorageChain cases are no
longer used so we can remove them.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-18 09:36:48 +01:00
Peter Krempa 7e582fe995 tests: storage: Use strict version of virStorageFileGetMetadata
Pass in 'true' as '@report_broken' of virStorageFileGetMetadata to make
it fail in the tests. The most important code paths (when starting the
VM) expect this function to fail rather than silently return partial
data. Switch the test to exercise this more important code path.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-18 09:36:48 +01:00
Michal Privoznik 4fa804c0c7 tests: Use g_strdup_printf() instead of virAsprintf()
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2019-11-12 16:15:59 +01:00
Michal Privoznik 3b4df5d350 Drop needless ret variable
In few places we have the following code pattern:

  int ret;
  ... /* @ret is not accessed here */
  ret = f(...);
  return ret;

This pattern can be written less verbose:

  ...
  return f(...);

This patch was generated with following coccinelle spatch:

  @@
  type T;
  constant C;
  expression f;
  identifier ret;
  @@
  -T ret = C;
   ... when != ret
  -ret = f;
  -return ret;
  +return f;

Afterwards I needed to fix a few places, e.g. comment in
virDomainNetIPParseXML() was removed too because coccinelle
thinks it refers to @ret while in fact it doesn't. Also in few
places it replaced @ret declaration with a few spaces instead of
removing the line. But nothing terribly wrong.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2019-10-24 08:10:37 +02:00
Ján Tomko 29b1e859e3 tests: use g_strdup instead of VIR_STRDUP
Replace all occurrences of
  if (VIR_STRDUP(a, b) < 0)
     /* effectively dead code */
with:
  a = g_strdup(b);

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-21 12:51:59 +02:00
Ján Tomko b6108a04ea Use g_steal_pointer instead of VIR_STEAL_PTR everywhere
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-16 15:59:42 +02:00
Ján Tomko 2b390b97b4 Use g_autoptr instead of VIR_AUTOUNREF
Now that all the types using VIR_AUTOUNREF have a cleanup func defined
to virObjectUnref, use g_autoptr instead of VIR_AUTOUNREF.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-16 12:06:44 +02:00
Ján Tomko 45678bd70a Use g_autoptr instead of VIR_AUTOPTR
Since commit 44e7f02915
    util: rewrite auto cleanup macros to use glib's equivalent

VIR_AUTOPTR aliases to g_autoptr. Replace all of its use by the GLib
macro version.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-16 12:06:43 +02:00
Ján Tomko 1e2ae2e311 Use g_autofree instead of VIR_AUTOFREE
Since commit 44e7f02915
    util: rewrite auto cleanup macros to use glib's equivalent

VIR_AUTOFREE is just an alias for g_autofree. Use the GLib macros
directly instead of our custom aliases.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-16 12:06:43 +02:00
Ján Tomko 2b2c67b401 virbuffer: use g_auto directly for virBuffer
Since commit 44e7f02915
    util: rewrite auto cleanup macros to use glib's equivalent

VIR_AUTOCLEAN is just an alias for g_auto. Use the GLib macros
directly instead of our custom aliases.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-16 12:06:43 +02:00
Ján Tomko 67e72053c1 Use G_N_ELEMENTS instead of ARRAY_CARDINALITY
Prefer the GLib version of the macro.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-15 16:14:19 +02:00
Ján Tomko 0d94f02455 tests: use G_GNUC_UNUSED
Use G_GNUC_UNUSED from GLib instead of ATTRIBUTE_UNUSED.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-15 11:25:25 +02:00
Peter Krempa 6ff9241058 util: storagefile: Flag backing store strings with authentication
Using inline authentication for storage volumes will not work properly
as libvirt requires use of the secret driver for the auth data and
thus would not be able to represent the passwords stored in the backing
store string.

Make sure that the backing store parsers return 1 which is a sign for
the caller to not use the file in certain cases.

The test data include iscsi via a json pseudo-protocol string and URIs
with the userinfo part being present.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-08-26 13:49:17 +02:00
Peter Krempa e1189ae5f3 tests: virstorage: Allow testing return value of virStorageSourceNewFromBackingAbsolute
Modify testBackingParse to allow testing other return values of the
backing store string parser.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-08-26 13:49:17 +02:00