Commit Graph

35 Commits

Author SHA1 Message Date
Jon Cooper daf37b556e sfc: PIO:Restrict to 64bit arch and use 64-bit writes.
Fixes:ee45fd92c739
("sfc: Use TX PIO for sufficiently small packets")

The linux net driver uses memcpy_toio() in order to copy into
the PIO buffers.
Even on a 64bit machine this causes 32bit accesses to a write-
combined memory region.
There are hardware limitations that mean that only 64bit
naturally aligned accesses are safe in all cases.
Due to being write-combined memory region two 32bit accesses
may be coalesced to form a 64bit non 64bit aligned access.
Solution was to open-code the memory copy routines using pointers
and to only enable PIO for x86_64 machines.

Not tested on platforms other than x86_64 because this patch
disables the PIO feature on other platforms.
Compile-tested on x86 to ensure that works.

The WARN_ON_ONCE() code in the previous version of this patch
has been moved into the internal sfc debug driver as the
assertion was unnecessary in the upstream kernel code.

This bug fix applies to v3.13 and v3.14 stable branches.

Signed-off-by: Shradha Shah <sshah@solarflare.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2014-06-11 15:36:21 -07:00
Ben Hutchings 93413f5058 sfc: Rename 'use_options' variable in tso_start() to clearer 'use_opt_desc'
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
Signed-off-by: Shradha Shah <sshah@solarflare.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2014-02-12 17:53:35 -05:00
Ben Hutchings 0bdadad166 sfc: Replace TSOH_OFFSET with the equivalent NET_IP_ALIGN
If CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS is defined then NET_IP_ALIGN
will be defined as 0, so this macro is redundant.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
Signed-off-by: Shradha Shah <sshah@solarflare.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2014-02-12 17:53:34 -05:00
Paul Gortmaker 440b87eac4 drivers/net: fix build warning in ethernet/sfc/tx.c
Commit ee45fd92c7 ("sfc: Use TX PIO
for sufficiently small packets") introduced the following warning:

drivers/net/ethernet/sfc/tx.c: In function 'efx_enqueue_skb':
drivers/net/ethernet/sfc/tx.c:432:1: warning: label 'finish_packet' defined but not used

Stick the label inside the same #ifdef that the code which calls
it uses.  Note that this is only seen for arch that do not set
ARCH_HAS_IOREMAP_WC, such as arm, mips, sparc, ..., as the others
enable the write combining code and hence use the label.

Cc: Jon Cooper <jcooper@solarflare.com>
Cc: Ben Hutchings <bhutchings@solarflare.com>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Acked-by: Shradha Shah <sshah@solarflare.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2014-02-06 21:28:05 -08:00
Alexandre Rames 2acdb92e85 sfc: Fix DMA unmapping issue with firmware assisted TSO
When using firmware assisted TSO, we use a single DMA mapping for
the linear area of a TSO skb.

We still have to segment the super-packet and insert a descriptor
containing the original headers before each segment of payload, so we
can unmap the linear area only after the last segment is completed.
The unmapping information for the linear area is therefore associated
with the last header descriptor.

We calculate the DMA address to unmap from using the map length and
the invariant that the end of the DMA mapping matches the end of
the data referenced by the last descriptor.  But this invariant is
broken when there is TCP payload in the linear area.

Fix this by adding and using an explicit dma_offset field.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-10-31 20:58:14 +00:00
Jon Cooper ee45fd92c7 sfc: Use TX PIO for sufficiently small packets
Sufficiently small linear packets can be copied into the PIO buffer
with a single call to memcpy_toio().  Non-linear packets require an
intermediate cache-line-sized buffer.

[bwh: I wrote the first version of this, but Jon did the hard work to
 handle non-linear packets.]
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-09-20 19:31:52 +01:00
Ben Hutchings 0fe5565b64 sfc: Introduce inline functions to simplify TX insertion
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-09-20 19:31:51 +01:00
Ben Hutchings 183233bec8 sfc: Allocate and link PIO buffers; map them with write-combining
Try to allocate a segment of PIO buffer to each TX channel.  If
allocation fails, log an error but continue.

PIO buffers must be mapped separately from the NIC registers, with
write-combining enabled.  Where the host page size is 4K, we could
potentially map each VI's registers and PIO buffer separately.
However, this would add significant complexity, and we also need to
support architectures such as POWER which have a greater page size.
So make a single contiguous write-combining mapping after the
uncacheable mapping, aligned to the host page size, and link PIO
buffers there.  Where necessary, allocate additional VIs within
the write-combining mapping purely for access to PIO buffers.

Link all TX buffers to TX queues and the additional VIs in
efx_ef10_dimension_resources() and in efx_ef10_init_nic() after
an MC reboot.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-09-20 19:31:49 +01:00
Ben Hutchings dfa50be95c sfc: Implement firmware-assisted TSO for EF10
Segmentation remains in the driver, but we generate option descriptors
describing the required packet editing rather than making our own
copies.

Reduce tso_state::ipv4_id to 16 bits, so it doesn't overflow into the
TCP_FLAGS field of the option descriptor.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-09-20 19:31:48 +01:00
Ben Hutchings c78c39e67c sfc: Fold tso_get_head_fragment() into tso_start()
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-09-20 19:31:46 +01:00
Ben Hutchings f7a6d2c442 sfc: Update copyright banners
Update the dates for files that have been added to in 2012-2013.
Drop the 'Solarstorm' brand name that's still lingering here.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-08-29 23:34:51 +01:00
Ben Hutchings ba8977bdb2 sfc: Extend struct efx_tx_buffer to allow pushing option descriptors
The TX path firmware for EF10 supports 'option descriptors' to control
offloads and various other features.  Add a flag and field for these
in struct efx_tx_buffer, and don't treat them as DMA descriptors on
completion.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-08-29 18:12:14 +01:00
Ben Hutchings 02e121650b sfc: Add TX merged completion counter
Add a counter for TX merged completion events.

This is implemented in the common TX path, because the NIC event
handlers only know how many descriptors were completed, not how many
packets.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-08-27 22:28:47 +01:00
Ben Hutchings e42c3d85af sfc: Refactor queue teardown sequence to allow for EF10 flush behaviour
Currently efx_stop_datapath() will try to flush our DMA queues (if DMA
is enabled), then finalise software and hardware state for each queue.
However, for EF10 we must ask the MC to finalise each queue, which
implicitly starts flushing it, and then wait for the flush events.
We therefore need to delegate more of this to the NIC type.

Combine all the hardware operations into a new NIC-type operation
efx_nic_type::fini_dmaq, and call this before tearing down the
software state and buffers for all the DMA queues.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-08-21 19:49:23 +01:00
Ben Hutchings 0d19a540be sfc: Add GFP flags to efx_nic_alloc_buffer() and make most callers allow blocking
Most call sites for efx_nic_alloc_buffer() are part of the probe or
reconfiguration paths and can allocate with GFP_KERNEL.  A few others
should use GFP_NOIO (I think).  Only one is in atomic context and
must use the current GFP_ATOMIC.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-08-21 19:43:47 +01:00
Stuart Hodgson 7c236c43b8 sfc: Add support for IEEE-1588 PTP
Add PTP IEEE-1588 support and make accesible via the PHC subsystem.

This work is based on prior code by Andrew Jackson

Signed-off-by: Stuart Hodgson <smhodgson@solarflare.com>
[bwh:
 - Add byte order conversion in efx_ptp_send_times()
 - Simplify conversion of PPS event times
 - Add the built-in vs module check to CONFIG_SFC_PTP dependencies]
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2012-09-19 02:54:12 +01:00
Ben Hutchings 9714284f83 sfc: Stash header offsets for TSO in struct tso_state
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2012-08-24 20:10:17 +01:00
Ben Hutchings 53cb13c680 sfc: Replace tso_state::full_packet_space with ip_base_len
We only use tso_state::full_packet_space to calculate the IPv4 tot_len
or IPv6 payload_len, not to set tso_state::packet_space.  Replace it
with an ip_base_len field holding the value of tot_len or payload_len
before including the TCP payload, which is much more useful when
constructing the new headers.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2012-08-24 20:10:17 +01:00
Ben Hutchings f7251a9ce9 sfc: Simplify TSO header buffer allocation
TSO header buffers contain a control structure immediately followed by
the packet headers, and are kept on a free list when not in use.  This
complicates buffer management and tends to result in cache read misses
when we recycle such buffers (particularly if DMA-coherent memory
requires caches to be disabled).

Replace the free list with a simple mapping by descriptor index.  We
know that there is always a payload descriptor between any two
descriptors with TSO header buffers, so we can allocate only one
such buffer for each two descriptors.

While we're at it, use a standard error code for allocation failure,
not -1.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2012-08-24 20:10:11 +01:00
Ben Hutchings 14bf718fb9 sfc: Stop TX queues before they fill up
We now have a definite upper bound on the number of descriptors per
skb; use that to stop the queue when the next packet might not fit.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2012-08-24 19:00:27 +01:00
Ben Hutchings 7668ff9c2a sfc: Refactor struct efx_tx_buffer to use a flags field
Add a flags field to struct efx_tx_buffer, replacing the
continuation and map_single booleans.

Since a single descriptor cannot be both a TSO header and the last
descriptor for an skb, unionise efx_tx_buffer::{skb,tsoh} and add
flags for validity of these fields.

Clear all flags in free buffers (whereas previously the continuation
flag would be set).

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2012-08-24 19:00:26 +01:00
Ben Hutchings 7e6d06f0de sfc: Fix maximum number of TSO segments and minimum TX queue size
Currently an skb requiring TSO may not fit within a minimum-size TX
queue.  The TX queue selected for the skb may stall and trigger the TX
watchdog repeatedly (since the problem skb will be retried after the
TX reset).  This issue is designated as CVE-2012-3412.

Set the maximum number of TSO segments for our devices to 100.  This
should make no difference to behaviour unless the actual MSS is less
than about 700.  Increase the minimum TX queue size accordingly to
allow for 2 worst-case skbs, so that there will definitely be space
to add an skb after we wake a queue.

To avoid invalidating existing configurations, change
efx_ethtool_set_ringparam() to fix up values that are too small rather
than returning -EINVAL.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2012-08-02 00:19:17 -07:00
Ben Hutchings f7cbb163d9 sfc: Stop changing header offsets on TX
There is nothing in the VLAN driver or core VLAN support that
invalidates the TCP and IP header offsets.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2012-07-17 16:12:31 +01:00
Ben Hutchings e718905c4b sfc: Remove dead write to tso_state::packet_space
tso_state::packet_space is always set in tso_start_packet(); the
value set in tso_start() is not used, and is also incorrect.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2012-07-17 16:12:30 +01:00
Ben Hutchings 0e33d87033 sfc: Use generic DMA API, not PCI-DMA API
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2012-07-17 16:12:30 +01:00
Ben Hutchings 2d0cc56da3 sfc: Minor formatting cleanup
Fix some indentation and line continuations.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2012-02-22 20:48:38 +00:00
Ben Hutchings 5b6262d0cc sfc: Replace some literal constants with EFX_PAGE_SIZE/EFX_BUF_SIZE
The 'page size' for PCIe DMA, i.e. the alignment of boundaries at
which DMA must be broken, is 4KB.  Name this value as EFX_PAGE_SIZE
and use it in efx_max_tx_len().  Redefine EFX_BUF_SIZE as
EFX_PAGE_SIZE since its value is also a result of that requirement,
and use it in efx_init_special_buffer().

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2012-02-13 23:40:49 +00:00
Ben Hutchings 73ba7b68e9 sfc: Remove remnants of on-load self-test
The out-of-tree version of the sfc driver used to run a self-test on
each device before registering it.  Although this was never included
in-tree, some functions have checks for this special case which is not
really possible.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2012-01-27 00:10:55 +00:00
Thomas Meyer c2e4e25afc sfc: Use kcalloc instead of kzalloc to allocate array
The advantage of kcalloc is, that will prevent integer overflows which could
result from the multiplication of number of elements and size and it is also
a bit nicer to read.

The semantic patch that makes this change is available
in https://lkml.org/lkml/2011/11/25/107

Signed-off-by: Thomas Meyer <thomas@m3y3r.de>
Signed-off-by: David S. Miller <davem@davemloft.net>
2011-12-04 01:32:06 -05:00
Eric Dumazet 449fa023bc sfc: fix race in efx_enqueue_skb_tso()
As soon as skb is pushed to hardware, it can be completed and freed, so
we should not dereference skb anymore.

Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2011-11-30 17:12:27 -05:00
Tom Herbert c3940999b2 sfc: Support for byte queue limits
Changes to sfc to use byte queue limits.

Signed-off-by: Tom Herbert <therbert@google.com>
Acked-by: Eric Dumazet <eric.dumazet@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2011-11-29 12:46:20 -05:00
Eric Dumazet 9e903e0852 net: add skb frag size accessors
To ease skb->truesize sanitization, its better to be able to localize
all references to skb frags size.

Define accessors : skb_frag_size() to fetch frag size, and
skb_frag_size_{set|add|sub}() to manipulate it.

Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2011-10-19 03:10:46 -04:00
Ian Campbell 5d6bcdfe38 net: use DMA_x_DEVICE and dma_mapping_error with skb_frag_dma_map
When I converted some drivers from pci_map_page to skb_frag_dma_map I
neglected to convert PCI_DMA_xDEVICE into DMA_x_DEVICE and
pci_dma_mapping_error into dma_mapping_error.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2011-10-06 16:17:20 -04:00
Ian Campbell 4a22c4c919 sfc: convert to SKB paged frag API.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Solarflare linux maintainers <linux-net-drivers@solarflare.com>
Cc: Steve Hodgson <shodgson@solarflare.com>
Cc: Ben Hutchings <bhutchings@solarflare.com>
Cc: netdev@vger.kernel.org
Signed-off-by: David S. Miller <davem@davemloft.net>
2011-09-22 16:04:15 -04:00
Jeff Kirsher 874aeea5d0 sfc: Move the Solarflare drivers
Moves the Solarflare drivers into drivers/net/ethernet/sfc/ and
make the necessary Kconfig and Makefile changes.

CC: Steve Hodgson <shodgson@solarflare.com>
CC: Ben Hutchings <bhutchings@solarflare.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
2011-08-11 02:33:50 -07:00