Make it much more understandable, add a missing
iov_cnt argument (number of iovs in the iov), and
add comments to it.
The new implementation has been extensively tested
by splitting a large buffer into many small
randomly-sized chunks, sending it over socket to
another, slow process and verifying the receiving
data is the same.
Also add a unit test for iov_send_recv(), sending/
receiving data between two processes over a socketpair
using random vectors and random sizes.
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Rename do_sendv_recvv() to iov_send_recv(), change its last arg
(do_send) from int to bool, export it in iov.h, and made the two
callers of it (iov_send() and iov_recv()) to be trivial #defines
just adding 5th arg.
iov_send_recv() will be used later.
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Rename arguments and use size_t for sizes instead of int,
from
int
qemu_sendv(int sockfd, struct iovec *iov,
int len, int iov_offset)
to
ssize_t
iov_send(int sockfd, struct iovec *iov,
size_t offset, size_t bytes)
The main motivation was to make it clear that length
and offset are in _bytes_, not in iov elements: it was
very confusing before, because all standard functions
which deals with iovecs expects number of iovs, not
bytes, even the fact that struct iovec has iov_len and
iov_ prefix does not help. With "bytes" and "offset",
especially since they're now size_t, it is much more
explicit. Also change the return type to be ssize_t
instead of int.
This also changes it to match other iov-related functons,
but not _quite_: there's still no argument indicating
where iovec ends, ie, no iov_cnt parameter as used
in iov_size() and friends. If will be added in subsequent
patch/rewrite.
All callers of qemu_sendv() and qemu_recvv() and
related, like qemu_co_sendv() and qemu_co_recvv(),
were checked to verify that it is safe to use unsigned
datatype instead of int.
Note that the order of arguments is changed to: offset
and bytes (len and iov_offset) are swapped with each
other. This is to make them consistent with very similar
functions from qemu_iovec family, where offset always
follows qiov, to mean the place in it to start from.
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
It now allows specifying offset within qiov to start from and
amount of bytes to copy. Actual implementation is just a call
to iov_to_buf().
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
qemu_iovec_concat() is currently a wrapper for
qemu_iovec_copy(), use the former (with extra
"0" arg) in a few places where it is used.
Change skip argument of qemu_iovec_copy() from
uint64_t to size_t, since size of qiov itself
is size_t, so there's no way to skip larger
sizes. Rename it to soffset, to make it clear
that the offset is applied to src.
Also change the only usage of uint64_t in
hw/9pfs/virtio-9p.c, in v9fs_init_qiov_from_pdu() -
all callers of it actually uses size_t too,
not uint64_t.
One added restriction: as for all other iovec-related
functions, soffset must point inside src.
Order of argumens is already good:
qemu_iovec_memset(QEMUIOVector *qiov, size_t offset,
int c, size_t bytes)
vs:
qemu_iovec_concat(QEMUIOVector *dst,
QEMUIOVector *src,
size_t soffset, size_t sbytes)
(note soffset is after _src_ not dst, since it applies to src;
for memset it applies to qiov).
Note that in many places where this function is used,
the previous call is qemu_iovec_reset(), which means
many callers actually want copy (replacing dst content),
not concat. So we may want to add a wrapper like
qemu_iovec_copy() with the same arguments but which
calls qemu_iovec_reset() before _concat().
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Similar to
qemu_iovec_memset(QEMUIOVector *qiov, size_t offset,
int c, size_t bytes);
the new prototype is:
qemu_iovec_from_buf(QEMUIOVector *qiov, size_t offset,
const void *buf, size_t bytes);
The processing starts at offset bytes within qiov.
This way, we may copy a bounce buffer directly to
a middle of qiov.
This is exactly the same function as iov_from_buf() from
iov.c, so use the existing implementation and rename it
to qemu_iovec_from_buf() to be shorter and to match the
utility function.
As with utility implementation, we now assert that the
offset is inside actual iovec. Nothing changed for
current callers, because `offset' parameter is new.
While at it, stop using "bounce-qiov" in block/qcow2.c
and copy decrypted data directly from cluster_data
instead of recreating a temp qiov for doing that.
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
This patch combines two functions into one, and replaces
the implementation with already existing iov_memset() from
iov.c.
The new prototype of qemu_iovec_memset():
size_t qemu_iovec_memset(qiov, size_t offset, int fillc, size_t bytes)
It is different from former qemu_iovec_memset_skip(), and
I want to make other functions to be consistent with it
too: first how much to skip, second what, and 3rd how many
of it. It also returns actual number of bytes filled in,
which may be less than the requested `bytes' if qiov is
smaller than offset+bytes, in the same way iov_memset()
does.
While at it, use utility function iov_memset() from
iov.h in posix-aio-compat.c, where qiov was used.
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
The qemu-img.c:is_not_zero() function checks if a buffer contains all
zeroes. This function will come in handy for zero-detection in the
block layer, so clean it up and move it to cutils.c.
Note that the function now returns true if the buffer is all zeroes.
This avoids the double-negatives (i.e. !is_not_zero()) that the old
function can cause in callers.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Outside coroutines, avoid busy waiting on EAGAIN by temporarily
making the socket blocking.
The API of qemu_recvv/qemu_sendv is slightly different from
do_readv/do_writev because they do not handle coroutines. It
returns the number of bytes written before encountering an
EAGAIN. The specificity of yielding on EAGAIN is entirely in
qemu-coroutine.c.
Reviewed-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
qemu_iovec_destroy does not clear the QEMUIOVector fully, and the data
could thus be used after free or freed again. While I do not know any
example in the tree, I observed this using virtio-scsi (and SCSI
scatter/gather) when canceling DMA requests.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
strtosz() & friends require the size to be at the end of the string,
or be followed by whitespace or ','. I find this surprising, because
the name suggests it works like strtol().
The check simplifies callers that accept exactly that follow set
slightly. No such callers exist.
The check is redundant for callers that accept a smaller follow set,
and thus need to check themselves anyway. Right now, this is the case
for all but one caller. All of them neglected to check, or checked
incorrectly, but the previous few commits fixed them up.
Finally, the check is problematic for callers that accept a larger
follow set. This is the case in monitor_parse_command().
Fortunately, the problems there are relatively harmless.
monitor_parse_command() uses strtosz() for argument type 'o'. When
the last argument is of type 'o', a trailing ',' is diagnosed
differently than other trailing junk:
(qemu) migrate_set_speed 1x
invalid size
(qemu) migrate_set_speed 1,
migrate_set_speed: extraneous characters at the end of line
A related inconsistency exists with non-last arguments. No such
command exists, but let's use memsave to explore the inconsistency.
The monitor permits, but does not require whitespace between
arguments. For instance, "memsave (1-1)1024foo" is parsed as command
memsave with three arguments 0, 1024 and "foo". Yes, this is daft,
but at least it's consistently daft.
If I change memsave's second argument from 'i' to 'o', then "memsave
(1-1)1foo" is rejected, because the size is followed by an 'f'. But
"memsave (1-1)1," is still accepted, and duly saves to file ",".
We don't have any users of strtosz that profit from the check. In the
users we have, it appears to encourage sloppy error checking, or gets
in the way. Drop the bothersome check.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
Commit 9f9b17a4's strtosz() defaults a missing suffix to 'M', except
it rejects fractions then (switch case 0).
When commit d8427002 introduced strtosz_suffix(), that changed:
fractions are no longer rejected, because we go to switch case 'M' on
missing suffix now. Not mentioned in commit message, probably
unintentional. Not worth changing back now.
Because case 0 is still around, you can get the old behavior by
passing a zero default_suffix to strtosz_suffix() or
strtosz_suffix_unit(). Undocumented and not used. Drop.
Commit d8427002 also neglected to update the function comment. Fix it
up.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
Move the parsing of a filedescriptor into a common function
qemu_parse_fd() so others can use it as well. Have net.c call this
function.
v2:
- moving qemu_parse_fd into cutils.c
Signed-off-by: Stefan Berger <stefanb@linux.vnet.ibm.com>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
This function does the same as the strtosz_suffix function
except that it allows to specify the unit to which the
k/M/B/T suffixes apply. This function will be used later to
parse the tsc-frequency from the command-line.
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
This patch fixes the following bug in QCOW2. For a QCOW2 image that is larger
than its base image, when handling a read request straddling over the end of the
base image, the QCOW2 driver attempts to read beyond the end of the base image
and the request would fail.
This bug was found by Fast Virtual Disk (FVD)'s fully automated testing tool.
The following test triggered the bug.
dd if=/dev/zero of=/var/ramdisk/truth.raw count=0 bs=1 seek=1098561536
dd if=/dev/zero of=/var/ramdisk/zero-500M.raw count=0 bs=1 seek=593099264
./qemu-img create -f qcow2 -ocluster_size=65536,backing_fmt=blksim -b /var/ramdisk/zero-500M.raw /var/ramdisk/test.qcow2 1098561536
./qemu-io --auto --seed=30477694 --truth=/var/ramdisk/truth.raw --format=qcow2 --test=blksim:/var/ramdisk/test.qcow2 --verify_write=true --compare_before=false --compare_after=true --round=100000 --parallel=100 --io_size=10485760 --fail_prob=0 --cancel_prob=0 --instant_qemubh=true
Signed-off-by: Chunqiang Tang <ctang@us.ibm.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
isspace() behavior is undefined for signed char.
Bug pointed out by Eric Blake, thanks!
Signed-off-by: Jes Sorensen <Jes.Sorensen@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
strtosz() needs to return a 64 bit type even on 32 bit
architectures. Otherwise qemu-img will fail to create disk
images >= 2GB
Signed-off-by: Jes Sorensen <Jes.Sorensen@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This introduces strtosz_suffix() which allows the caller to specify a
default suffix in case the non default of MB is wanted.
strtosz() is kept as a wrapper for strtosz_suffix() which keeps it's
current default of MB.
Signed-off-by: Jes Sorensen <Jes.Sorensen@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
strtosz() returns -1 on error. It now supports human unit formats in
eg. 1.0G, with better error handling.
The following suffixes are supported:
B/b = bytes
K/k = KB
M/m = MB
G/g = GB
T/t = TB
This patch changes -numa and -m input to use strtosz().
Signed-off-by: Jes Sorensen <Jes.Sorensen@redhat.com>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
If we are flushing the caches for our image files we only care about the
data (including the metadata required for accessing it) but not things
like timestamp updates. So try to use fdatasync instead of fsync to
implement the flush operations.
Unfortunately many operating systems still do not support fdatasync,
so we add a qemu_fdatasync wrapper that uses fdatasync if available
as per the _POSIX_SYNCHRONIZED_IO feature macro or fsync otherwise.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
One performance problem of qcow2 during the initial image growth are
sequential writes that are not cluster aligned. In this case, when a first
requests requires to allocate a new cluster but writes only to the first
couple of sectors in that cluster, the rest of the cluster is zeroed - just
to be overwritten by the following second request that fills up the cluster.
Let's try to merge sequential write requests to the same cluster, so we can
avoid to write the zero padding to the disk in the first place.
As a nice side effect, also other formats take advantage of dealing with less
and larger requests.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
Fix missing strnlen (a GNU extension) problems by using qemu_strnlen
used for user emulators also for system emulators.
Signed-off-by: Blue Swirl <blauwirbel@gmail.com>
Problem: It is impossible to feed filenames with the character colon because
qemu interprets such names as a protocol. For example filename scsi:0, is
interpreted as a protocol by name "scsi".
This patch allows user to espace colon characters. For example the above
filename can now be expressed either as 'scsi\:0' or as file:scsi:0
anything following the "file:" tag is interpreted verbatin. However if "file:"
tag is omitted then any colon characters in the string must be escaped using
backslash.
Here are couple of examples:
scsi\:0\:abc is a local file scsi:0:abc
http\://myweb is a local file by name http://myweb
file:scsi:0:abc is a local file scsi:0:abc
file:http://myweb is a local file by name http://myweb
Signed-off-by: Ram Pai <linuxram@us.ibm.com>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
Include assert.h from qemu-common.h and remove other direct uses.
cpu-all.h still need to include it because of the dyngen-exec.h hacks
Signed-off-by: Paul Brook <paul@codesourcery.com>
Allow to initialize a QEMUIOVector from an externally allocated iovec.
qiov->nalloc is initialized to -1 to indicate external storage for qiov->iov
and all functions dealing with memory management assert on the iovec beeing
an internally managed first.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6902 c046a42c-6fe2-441c-8c8c-71466251a162
Add a helper to zero out an existing iovec. Removes the need to deallocate
and reallocate it.
Signed-off-by: Avi Kivity <avi@redhat.com>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6523 c046a42c-6fe2-441c-8c8c-71466251a162
In general, it is not possible to predict the size of of an I/O vector since
a contiguous guest region may map to a disconiguous host region. Add some
helpers to manage I/O vector growth.
Signed-off-by: Avi Kivity <avi@redhat.com>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@6396 c046a42c-6fe2-441c-8c8c-71466251a162
Fix compiler warning on OSX, reported by Andreas Faerber.
git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@5982 c046a42c-6fe2-441c-8c8c-71466251a162
As suggested by Laurent Desnogues.
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@5872 c046a42c-6fe2-441c-8c8c-71466251a162
This is needed for virtio. The implementation is originally from
Marcelo Tosatti.
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@5868 c046a42c-6fe2-441c-8c8c-71466251a162
when compiling on NetBSD:
warning: array subscript has type 'char'
Signed-off-by: Christoph Egger <Christoph.Egger@amd.com>
git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@5727 c046a42c-6fe2-441c-8c8c-71466251a162
The use of strncat and strndup was correct, pstrcpy and pstrdup wasn't.
I'll try to restore building on non-gnu OSes in a later commit.
git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@5651 c046a42c-6fe2-441c-8c8c-71466251a162
Remove QEMU_TOOL. Replace with QEMU_IMG and NEED_CPU_H.
Avoid linking qemu-img against whole system emulatior.
git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@3578 c046a42c-6fe2-441c-8c8c-71466251a162
Michael Tokarev