257 lines
12 KiB
ReStructuredText
257 lines
12 KiB
ReStructuredText
.. _faq:
|
||
|
||
iperf3 FAQ
|
||
==========
|
||
|
||
What is the history of iperf3, and what is the difference between iperf2 and iperf3?
|
||
iperf2 was orphaned in the late 2000s at version 2.0.5, despite some
|
||
known bugs and issues. After spending some time trying to fix
|
||
iperf2's problems, ESnet decided by 2010 that a new, simpler tool
|
||
was needed, and began development of iperf3. The goal was make the
|
||
tool as simple as possible, so others could contribute to the code
|
||
base. For this reason, it was decided to make the tool single
|
||
threaded, and not worry about backwards compatibility with
|
||
iperf2. Many of the feature requests for iperf3 came from the
|
||
perfSONAR project (http://www.perfsonar.net).
|
||
|
||
Then in 2014, Bob (Robert) McMahon from Broadcom restarted
|
||
development of iperf2 (See
|
||
https://sourceforge.net/projects/iperf2/). He fixed many of the
|
||
problems with iperf2, and added a number of new features similar to
|
||
iperf3. iperf2.0.8, released in 2015, made iperf2 a useful tool. iperf2's
|
||
current development is focused is on using UDP for latency testing, as well
|
||
as broad platform support.
|
||
|
||
As of this writing (2017), both iperf2 and iperf3 are being actively
|
||
(although independently) developed. We recommend being familiar with
|
||
both tools, and use whichever tool’s features best match your needs.
|
||
|
||
A feature comparison of iperf2, iperf3, and nuttcp is available at:
|
||
https://fasterdata.es.net/performance-testing/network-troubleshooting-tools/throughput-tool-comparision/
|
||
|
||
iperf3 parallel stream performance is much less than iperf2. Why?
|
||
iperf3 is single threaded, and iperf2 is multi-threaded. We
|
||
recommend using iperf2 for parallel streams.
|
||
If you want to use multiple iperf3 streams use the method described `here <https://fasterdata.es.net/performance-testing/network-troubleshooting-tools/iperf/multi-stream-iperf3/>`_.
|
||
|
||
I’m trying to use iperf3 on Windows, but having trouble. What should I do?
|
||
iperf3 is not officially supported on Windows, but iperf2 is. We
|
||
recommend you use iperf2.
|
||
|
||
Some people are using Cygwin to run iperf3 in Windows, but not all
|
||
options will work. Some community-provided binaries of iperf3 for
|
||
Windows exist.
|
||
|
||
How can I build a statically-linked executable of iperf3?
|
||
There are a number of reasons for building an iperf3 executable with
|
||
no dependencies on any shared libraries. Unfortunately this isn't
|
||
quite a straight-forward process.
|
||
|
||
The steps below have nominally been tested on CentOS 7.4, but
|
||
can probably be adapted for use with other Linux distributions:
|
||
|
||
#. If necessary, install the static C libraries; for CentOS this is
|
||
the ``glibc-static`` package.
|
||
|
||
#. If OpenSSL is installed, be sure that its static libraries are
|
||
also installed, from the ``openssl-static`` package.
|
||
|
||
#. Be sure that ``lksctp-*`` packages are not installed, because
|
||
as of this writing, there do not appear to be any static
|
||
libraries available for SCTP.
|
||
|
||
#. Configure iperf3 thusly: ``configure "LDFLAGS=--static"
|
||
--disable-shared`` These options are necessary to disable the
|
||
generation of shared libraries and link the executable
|
||
statically. For iperf-3.8 or later, configuring as ``configure
|
||
--enable-static-bin`` is another, shorter way to accomplish
|
||
this. If SCTP is installed on the system it might also be
|
||
necessary to pass the ``--without-sctp`` flag at configure
|
||
time.
|
||
|
||
#. Compile as normal.
|
||
|
||
It appears that for FreeBSD (tested on FreeBSD 11.1-RELEASE), only
|
||
the last two steps are needed to produce a static executable.
|
||
|
||
How can I build on a system that doesn't support profiled executables?
|
||
This problem has been noted by users attempting to build iperf3 for
|
||
Android systems, as well as some recent versions of macOS.
|
||
There are several workarounds. In order from least
|
||
effort to most effort:
|
||
|
||
#. Beginning with iperf-3.8, profiled executables are actually not
|
||
built by default, so this question becomes somewhat moot. Pass
|
||
the ``--enable-profiling`` flag to ``configure`` to build
|
||
profiled executables.
|
||
|
||
#. In iperf-3.6 and iperf-3.7, the ``--disable-profiling`` flag can be
|
||
passed to ``configure`` to disable the building of profiled
|
||
object files and the profiled executable.
|
||
|
||
#. At the time the linking of the iperf3 profiled executable fails,
|
||
the "normal" iperf3 executable is probably already created. So if
|
||
you are willing to accept the error exit from the make process
|
||
(and a little bit of wasted work on the build host), you might
|
||
not need to do anything.
|
||
|
||
#. After the configure step, there will be a definition in
|
||
``src/Makefile`` that looks like this::
|
||
|
||
noinst_PROGRAMS = t_timer$(EXEEXT) t_units$(EXEEXT) t_uuid$(EXEEXT) \
|
||
iperf3_profile$(EXEEXT)
|
||
|
||
If you edit it to look like this, it will disable the build of the profiled iperf3::
|
||
|
||
noinst_PROGRAMS = t_timer$(EXEEXT) t_units$(EXEEXT) t_uuid$(EXEEXT)
|
||
|
||
#. Similar to item 2 above, but more permanent...if you edit
|
||
``src/Makefile.am`` and change the line reading like this::
|
||
|
||
noinst_PROGRAMS = t_timer t_units t_uuid iperf3_profile
|
||
|
||
To look like this::
|
||
|
||
noinst_PROGRAMS = t_timer t_units t_uuid
|
||
|
||
And then run ``./bootstrap.sh``, that will regenerate the project
|
||
Makefiles to make the exclusion of the profiled iperf3 executable
|
||
permanant (within that source tree).
|
||
|
||
I'm seeing quite a bit of unexpected UDP loss. Why?
|
||
First, confirm you are using iperf 3.1.5 or higher. There was an
|
||
issue with the default UDP send size that was fixed in
|
||
3.1.5. Second, try adding the flag ``-w2M`` to increase the socket
|
||
buffer sizes. That seems to make a big difference on some hosts.
|
||
|
||
iperf3 UDP does not seem to work at bandwidths less than 100Kbps. Why?
|
||
You'll need to reduce the default packet length to get UDP rates of less that 100Kbps. Try ``-l100``.
|
||
|
||
TCP throughput drops to (almost) zero during a test, what's going on?
|
||
A drop in throughput to almost zero, except maybe for the first
|
||
reported interval(s), may be related to problems in NIC TCP Offload,
|
||
which is used to offload TCP functionality to the NIC (see
|
||
https://en.wikipedia.org/wiki/TCP_offload_engine). The goal of TCP
|
||
Offload is to save main CPU performance, mainly in the areas of
|
||
segmentation and reassembly of large packets and checksum
|
||
computation.
|
||
|
||
When TCP packets are sent with the "Don't Fragment" flag set, which
|
||
is the recommended setting, segmentation is done by the TCP stack
|
||
based on the reported next hop MSS in the ICMP Fragmentation Needed
|
||
message. With TCP Offload, active segmentation is done by the NIC on
|
||
the sending side, which is known as TCP Segmentation offload (TSO)
|
||
or in Windows as Large Send Offload (LSO). It seems that there are
|
||
TSO/LSO implementations which for some reason ignore the reported
|
||
MSS and therefore don’t perform segmentation. In these cases, when
|
||
large packets are sent, e.g. the default iperf3 128KB (131,072
|
||
bytes), iperf3 will show that data was sent in the first interval,
|
||
but since the packets don’t get to the server, no ack is received
|
||
and therefore no data is sent in the following intervals. It may
|
||
happen that after certain timeout the main CPU will re-send the
|
||
packet by re-segmenting it, and in these cases data will get to the
|
||
server after a while. However, it seems that segmentation is not
|
||
automatically continued with the next packet, so the data transfer
|
||
rate be very low.
|
||
|
||
The recommended solution in such a case is to disable TSO/LSO, at
|
||
least on the relevant port. See for example:
|
||
https://atomicit.ca/kb/articles/slow-network-speed-windows-10/. If
|
||
that doesn’t help then "Don't Fragment" TCP flag may be
|
||
disabled. See for example:
|
||
https://support.microsoft.com/en-us/help/900926/recommended-tcp-ip-settings-for-wan-links-with-a-mtu-size-of-less-than. However,
|
||
note that disabling the “Don’t Fragment” flag may cause other
|
||
issues.
|
||
|
||
To test whether TSO/LSO may be the problem, do the following:
|
||
|
||
* If different machine configurations are used for the client and
|
||
server, try the iperf3 reverse mode (``-R``). If TSO/LSO is only
|
||
enabled on the client machine, this test should succeed.
|
||
* Reduce the sending length to a small value that should not require
|
||
segmentation, using the iperf3 ``-l`` option, e.g. ``-l 512``. It
|
||
may also help to reduce the MTU by using the iperf3 ``-M`` option,
|
||
e.g. ``-M 1460``.
|
||
* Using tools like Wireshark, identify the required MSS in the ICMP
|
||
Fragmentation Needed messages (if reported). Run tests with the
|
||
``-l`` value set to 2 times the MSS and then 4 times, 6 times,
|
||
etc. With TSO/LSO issue in each test the throughput should be
|
||
reduced more. It may help to increase the testing time beyond the
|
||
default 10 seconds to better see the behavior (iperf3 ``-t``
|
||
option).
|
||
|
||
What congestion control algorithms are supported?
|
||
On Linux, run this command to see the available congestion control
|
||
algorithms (note that some algorithms are packaged as kernel
|
||
modules, which must be loaded before they can be used)::
|
||
|
||
/sbin/sysctl net.ipv4.tcp_available_congestion_control
|
||
|
||
On FreeBSD, the equivalent command is::
|
||
|
||
/sbin/sysctl net.inet.tcp.cc.available
|
||
|
||
I’m using the ``--logfile`` option. How do I see file output in real time?
|
||
Use the ``--forceflush`` flag.
|
||
|
||
I'm using the --fq-rate flag, but it does not seem to be working. Why?
|
||
You need to add 'net.core.default_qdisc = fq' to /etc/sysctl.conf for that option to work.
|
||
|
||
I'm having trouble getting iperf3 to work on Windows, Android, etc. Where can I get help?
|
||
iperf3 only supports Linux, FreeBSD, and OSX. For other platforms we recommend using iperf2.
|
||
|
||
I managed to get a Windows executable built, but why do I get a BSOD on Windows 7?
|
||
There seems to be a bug in Windows 7 where running iperf3 from a
|
||
network filesystem can cause a system crash (in other words Blue
|
||
Screen of Death, or BSOD). This is a Windows bug addressed in kb2839149:
|
||
|
||
https://support.microsoft.com/en-us/help/2839149/stop-error-0x00000027-in-the-rdbss-sys-process-in-windows-7-or-windows
|
||
|
||
A hotfix is available under kb2732673:
|
||
|
||
https://support.microsoft.com/en-us/help/2732673/-delayed-write-failed-error-message-when--pst-files-are-stored-on-a-ne
|
||
|
||
Why can’t I run a UDP client with no server?
|
||
This is potentially dangerous, and an attacker could use this for a
|
||
denial of service attack. We don't want iperf3 to be an attack tool.
|
||
|
||
I'm trying to use iperf3 to test a 40G/100G link...What do I need to know?
|
||
See the following pages on fasterdata.es.net:
|
||
|
||
- https://fasterdata.es.net/host-tuning/100g-tuning/
|
||
- https://fasterdata.es.net/performance-testing/network-troubleshooting-tools/iperf/multi-stream-iperf3/
|
||
|
||
My receiver didn't get all the bytes that got sent but there was no loss. Huh?
|
||
iperf3 uses a control connection between the client and server to
|
||
manage the start and end of each test. Sometimes the commands on
|
||
the control connection can be received and acted upon before all of
|
||
the test data has been processed. Thus the test ends with data
|
||
still in flight. This effect can be significant for short (a few
|
||
seconds) tests, but is probably negligible for longer tests.
|
||
|
||
A file sent using the ``-F`` option got corrupted...what happened?
|
||
The ``-F`` option to iperf3 is not a file transfer utility. It's a
|
||
way of testing the end-to-end performance of a file transfer,
|
||
including filesystem and disk overheads. So while the test will
|
||
mimic an actual file transfer, the data stored to disk may not be
|
||
the same as what was sent. In particular, the file size will be
|
||
rounded up to the next larger multiple of the transfer block size,
|
||
and for UDP tests, iperf's metadata (containing timestamps and
|
||
sequence numbers) will overwrite the start of every UDP packet
|
||
payload.
|
||
|
||
I have a question regarding iperf3...what's the best way to get help?
|
||
Searching on the Internet is a good first step.
|
||
http://stackoverflow.com/ has a number of iperf3-related questions
|
||
and answers, but a simple query into your favorite search engine can
|
||
also yield some results.
|
||
|
||
There is a mailing list nominally used for iperf3 development,
|
||
iperf-dev@googlegroups.com.
|
||
|
||
We discourage the use of the iperf3 issue tracker on GitHub for
|
||
support questions. Actual bug reports, enhancement requests, or
|
||
pull requests are encouraged, however.
|
||
|
||
|