The current logic suffers from a slow response time to disable user DB
usage, and also fails to avoid DB FIFO drops under heavy load. This commit
fixes these deficiencies and makes the avoidance logic more optimal.
This is done by more efficiently notifying the ULDs of potential DB
problems, and implements a smoother flow control algorithm in iw_cxgb4,
which is the ULD that puts the most load on the DB fifo.
Design:
cxgb4:
Direct ULD callback from the DB FULL/DROP interrupt handler. This allows
the ULD to stop doing user DB writes as quickly as possible.
While user DB usage is disabled, the LLD will accumulate DB write events
for its queues. Then once DB usage is reenabled, a single DB write is
done for each queue with its accumulated write count. This reduces the
load put on the DB fifo when reenabling.
iw_cxgb4:
Instead of marking each qp to indicate DB writes are disabled, we create
a device-global status page that each user process maps. This allows
iw_cxgb4 to only set this single bit to disable all DB writes for all
user QPs vs traversing the idr of all the active QPs. If the libcxgb4
doesn't support this, then we fall back to the old approach of marking
each QP. Thus we allow the new driver to work with an older libcxgb4.
When the LLD upcalls iw_cxgb4 indicating DB FULL, we disable all DB writes
via the status page and transition the DB state to STOPPED. As user
processes see that DB writes are disabled, they call into iw_cxgb4
to submit their DB write events. Since the DB state is in STOPPED,
the QP trying to write gets enqueued on a new DB "flow control" list.
As subsequent DB writes are submitted for this flow controlled QP, the
amount of writes are accumulated for each QP on the flow control list.
So all the user QPs that are actively ringing the DB get put on this
list and the number of writes they request are accumulated.
When the LLD upcalls iw_cxgb4 indicating DB EMPTY, which is in a workq
context, we change the DB state to FLOW_CONTROL, and begin resuming all
the QPs that are on the flow control list. This logic runs on until
the flow control list is empty or we exit FLOW_CONTROL mode (due to
a DB DROP upcall, for example). QPs are removed from this list, and
their accumulated DB write counts written to the DB FIFO. Sets of QPs,
called chunks in the code, are removed at one time. The chunk size is 64.
So 64 QPs are resumed at a time, and before the next chunk is resumed, the
logic waits (blocks) for the DB FIFO to drain. This prevents resuming to
quickly and overflowing the FIFO. Once the flow control list is empty,
the db state transitions back to NORMAL and user QPs are again allowed
to write directly to the user DB register.
The algorithm is designed such that if the DB write load is high enough,
then all the DB writes get submitted by the kernel using this flow
controlled approach to avoid DB drops. As the load lightens though, we
resume to normal DB writes directly by user applications.
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Based on original work by Casey Leedom <leedom@chelsio.com>
Signed-off-by: Kumar Sanghvi <kumaras@chelsio.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Based on original work by Casey Leedom <leedom@chelsio.com>
Signed-off-by: Kumar Sanghvi <kumaras@chelsio.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Query firmware to see whether we're allowed to use T5 ULPTX MEMWRITE DSGL
capabilities. Also pass that information to Upper Layer Drivers via the
new (struct cxgb4_lld_info).ulptx_memwrite_dsgl boolean.
Based on original work by Casey Leedom <leedom@chelsio.com>
Signed-off-by: Kumar Sanghvi <kumaras@chelsio.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
When we attach to adapter, print VPD Part Number instead of Engineering Change field.
Based on original work by Casey Leedom <leedom@chelsio.com>
Signed-off-by: Kumar Sanghvi <kumaras@chelsio.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Also, create a new Common Code interface to translate Firmware Port Technology
Type values (enum fw_port_type) to string descriptions. This will allow us
to maintain the description translation table in one place rather than in
every driver.
Based on original work by Scott Bardone and Casey Leedom <leedom@chelsio.com>
Signed-off-by: Kumar Sanghvi <kumaras@chelsio.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
If we have EEH error happens to the adapter and we have to remove
it from the system for some reasons (e.g. more than 5 EEH errors
detected from the device in last hour), the adapter will be disabled
for towice separately by eeh_err_detected() and remove_one(), which
will incur following unexpected backtrace. The patch tries to avoid
it.
WARNING: at drivers/pci/pci.c:1431
CPU: 12 PID: 121 Comm: eehd Not tainted 3.13.0-rc7+ #1
task: c0000001823a3780 ti: c00000018240c000 task.ti: c00000018240c000
NIP: c0000000003c1e40 LR: c0000000003c1e3c CTR: 0000000001764c5c
REGS: c00000018240f470 TRAP: 0700 Not tainted (3.13.0-rc7+)
MSR: 8000000000029032 <SF,EE,ME,IR,DR,RI> CR: 28000024 XER: 00000004
CFAR: c000000000706528 SOFTE: 1
GPR00: c0000000003c1e3c c00000018240f6f0 c0000000010fe1f8 0000000000000035
GPR04: 0000000000000000 0000000000000000 00000000003ae509 0000000000000000
GPR08: 000000000000346f 0000000000000000 0000000000000000 0000000000003fef
GPR12: 0000000028000022 c00000000ec93000 c0000000000c11b0 c000000184ac3e40
GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
GPR24: 0000000000000000 c0000000009398d8 c00000000101f9c0 c0000001860ae000
GPR28: c000000182ba0000 00000000000001f0 c0000001860ae6f8 c0000001860ae000
NIP [c0000000003c1e40] .pci_disable_device+0xd0/0xf0
LR [c0000000003c1e3c] .pci_disable_device+0xcc/0xf0
Call Trace:
[c0000000003c1e3c] .pci_disable_device+0xcc/0xf0 (unreliable)
[d0000000073881c4] .remove_one+0x174/0x320 [cxgb4]
[c0000000003c57e0] .pci_device_remove+0x60/0x100
[c00000000046396c] .__device_release_driver+0x9c/0x120
[c000000000463a20] .device_release_driver+0x30/0x60
[c0000000003bcdb4] .pci_stop_bus_device+0x94/0xd0
[c0000000003bcf48] .pci_stop_and_remove_bus_device+0x18/0x30
[c00000000003f548] .pcibios_remove_pci_devices+0xa8/0x140
[c000000000035c00] .eeh_handle_normal_event+0xa0/0x3c0
[c000000000035f50] .eeh_handle_event+0x30/0x2b0
[c0000000000362c4] .eeh_event_handler+0xf4/0x1b0
[c0000000000c12b8] .kthread+0x108/0x130
[c00000000000a168] .ret_from_kernel_thread+0x5c/0x74
Signed-off-by: Gavin Shan <shangw@linux.vnet.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Conflicts:
drivers/net/ethernet/qlogic/qlcnic/qlcnic_sriov_pf.c
net/ipv6/ip6_tunnel.c
net/ipv6/ip6_vti.c
ipv6 tunnel statistic bug fixes conflicting with consolidation into
generic sw per-cpu net stats.
qlogic conflict between queue counting bug fix and the addition
of multiple MAC address support.
Signed-off-by: David S. Miller <davem@davemloft.net>
Cleanup by making local functions static.
The code to load config file is unreachable in net-next, probably came
from some out of tree driver.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Adds API cxgb4_select_ntuple so as to enable Upper Level Drivers to correctly
calculate the tuple fields.
Adds constant definitions for TP_VLAN_PRI_MAP for the Compressed
Filter Tuple field widths and structures and uses them.
Also, the CPL Parameters field for T5 is 40 bits so we need to prototype
cxgb4_select_ntuple() to calculate and return u64 values.
Based on original work by Casey Leedom <leedom@chelsio.com>
Signed-off-by: Kumar Sanghvi <kumaras@chelsio.com>
Signed-off-by: Hariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Reported by "make includecheck"
Tested that C sources including this file still compile well on x86
Signed-off-by: Michael Opdenacker <michael.opdenacker@free-electrons.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The Compressed LIP region is used to hold a limited number of Local
IPv6 addresses. This region is primarily used to reduce the TCAM
space consumed for an IPv6 offloaded connection. A 128-bit LIP will
be reduced to 13-bit and stored in the TCAM if there is a match
between the IPv6 tuple's LIP and the one stored in the CLIP region.
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: Roland Dreier <roland@purestorage.com>
Forcing uninitialized state allows us to upgrade and reinitialize the adapter.
FW_VERSION_T4 = 1.4.0.0
FW_VERSION_T5 = 0.0.0.0
At this point driver supports above and greater than above version of firmware.
If it doesn't find the required firmware version than it forces the adapter to
be reinitialized as shown below.
1) If FW_ON_ADAPTER < FW_VERSION and we're the MASTER_PF force uninitialized
state and a FW upgrade if available.
- If FW_ON_ADAPTER < /lib/firmware/cxgb4/t*fw.bin we will update the
adapters FW.
- If FW_ON_ADAPTER >= /lib/firmware/cxgb4/t*fw.bin don't upgrade FW.
- If upgrade_fw() fails force reinitialization of the adapter anyways,
it might still work.
Either way forcing the uninitialized state allows cxgb4 reinitialize FW.
2) If FW_ON_ADAPTER >= FW_VERSION driver follows normal path.
Signed-off-by: Jay Hernandez <jay@chelsio.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
All T5 adapters will only support VFs on PF0-3 despite the ability of the
hardware to support them on PF4-7. This keeps our T4 and T5 adapters more
similar which simplifies host driver software.
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Santosh Rastapur <santosh@chelsio.com>
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch implements a low latency Write Combining (aka Write Coalescing) work
request path. PCIE maps User Space Doorbell BAR2 region writes to the new
interface to SGE. SGE pulls a new message from PCIE new interface and if its a
coalesced write work request then pushes it for processing. This patch copies
coalesced work request to memory mapped BAR2 space.
Signed-off-by: Santosh Rastapur <santosh@chelsio.com>
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Santosh Rastapur <santosh@chelsio.com>
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Retries active opens for INUSE errors.
Logs any active ofld_connect_wr error replies.
Sends ofld_connect_wr on same ctrlq. It needs to go on the same control txq as
regular CPL active/passive messages.
Retries on active open replies with EADDRINUSE.
Uses active open fw wr only if active filter region is set.
Adds stat for ofld_connect_wr failures.
This patch also adds debugfs file to show endpoints.
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: Roland Dreier <roland@purestorage.com>
It supports establishing passive open connection through firmware filter work
request. Passive open connection will go through this path as now instead of
listening server we create a server filter which will redirect the incoming SYN
packet to the offload queue.
It divides filter region into regular filters and server filter portion. It
introduces new server filter region which will be exclusively used for creating
server filters. This region will not overlap with regular filter region.
It provides new API cxgb4_alloc_sftid in LLD for getting stid in case of LE
hash collision path. This new stid will be used to open server filter in the
filter region.
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: Roland Dreier <roland@purestorage.com>
The T4 architecture is capable of filtering ingress packets at line rate
using the rule in TCAM. If packet hits a rule in the TCAM then it can be either
dropped or passed to the receive queues based on a rule settings.
This patch adds framework for managing filters and to use T4's filter
capabilities. It constructs a Firmware Filter Work Request which writes the
filter at a specified index to get the work done. It hosts shadow copy of
ingress filter entry to check field size limitations and save memory in the
case where the filter table is large.
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: Roland Dreier <roland@purestorage.com>
This patch also fixes the build failure caused due to removal of #ifdef
CONFIG_CHELSIO_T4_OFFLOAD condition
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch fixes type assignment issues, function definition and symbol
shadowing which triggered sparse warnings.
Signed-off-by: Jay Hernandez <jay@chelsio.com>
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch adds a new common code routine to upgrade an adapter's
firmware. This routine handles all of the complexities of working with the
the existing adapter firmware in order to quiesce the adapter and uP, etc.
For an automatic upgrade it will send a HELLO command to check if cxgb4
want/can upgrade firmware, i.e. if cxgb4 is MASTER and has newer firmware
that it wants to load and call the new common code routine t4_fw_upgrade.
Note that it should not issue a RESET command after a successful firmware
upgrade.
Signed-off-by: Jay Hernandez <jay@chelsio.com>
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
In case if user defined configuration file at /lib/firmware/cxgb4/t4-config.txt
location and also factory default configuration file written to FLASH are not
present then driver will use hardwired configuration settings.
Signed-off-by: Jay Hernandez <jay@chelsio.com>
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Starting with T4 firmware version 1.3.11.0 the firmware now supports device
configuration via a Firmware Configuration File. The Firmware Configuration
File was primarily developed in order to centralize all of the configuration,
resource allocation, etc. for Unified Wire operation where multiple
Physical / Virtual Function Drivers would be using a T4 adapter simultaneously.
The Firmware Configuration file can live in three locations as shown below
in order of precedence.
1) User defined configuration file: /lib/firmware/cxgb4/t4-config.txt
2) Factory Default configuration file written to FLASH within
the manufacturing process.
3) Hardwired driver configuration.
Signed-off-by: Jay Hernandez <jay@chelsio.com>
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch adds new enums and macros to enable T4 configuration file support. It
also removes duplicate macro definitions.
It fixes the build failure in cxgb4vf driver introduced because of old macro
definition removal.
It also performs SGE initialization based on T4 configuration file is provided
or not. If it is provided then it uses the parameters provided in it otherwise
it uses hard coded values.
Signed-off-by: Jay Hernandez <jay@chelsio.com>
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch implements two new functions t4_mem_win_read and t4_memory_read.
These new functions can be used to read memory via the PCIE memory window.
Please note, for proper execution of these functions PCIE_MEM_ACCESS_BASE_WIN
registers must be setup correctly like how setup_memwin in the cxgb4 driver
does it.
Signed-off-by: Jay Hernandez <jay@chelsio.com>
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
recover LLD EQs for DB drop interrupts. This includes adding a new
db_lock, a spin lock disabling BH too, used by the recovery thread and
the ring_tx_db() paths to allow db drop recovery.
Clean up initial DB avoidance code.
Add read_eq_indices() - this allows the LLD to use the PCIe mw to
efficiently read hw eq contexts.
Add cxgb4_sync_txq_pidx() - called by iw_cxgb4 to sync up the sw/hw
pidx value.
Add flush_eq_cache() and cxgb4_flush_eq_cache(). This allows iw_cxgb4
to flush the sge eq context cache before beginning db drop recovery.
Add module parameter, dbfoifo_int_thresh, to allow tuning the db
interrupt threshold value.
Add dbfifo_int_thresh to cxgb4_lld_info so iw_cxgb4 knows the threshold.
Add module parameter, dbfoifo_drain_delay, to allow tuning the amount
of time delay between DB FULL and EMPTY upcalls to iw_cxgb4.
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Roland Dreier <roland@purestorage.com>
Add platform-specific callback functions for interrupts. This is
needed to do a single read-clear of the CAUSE register and then call
out to platform specific functions for DB threshold interrupts and DB
drop interrupts.
Add t4_mem_win_read_len() - mem-window reads for arbitrary lengths.
This is used to read the CIDX/PIDX values from EC contexts during DB
drop recovery.
Add t4_fwaddrspace_write() - sends addrspace write cmds to the fw.
Needed to flush the sge eq context cache.
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Roland Dreier <roland@purestorage.com>
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Dimitris Michailidis <dm@chelsio.com>
Cc: netdev@vger.kernel.org
Signed-off-by: David S. Miller <davem@davemloft.net>
Moves the drivers for the Chelsio chipsets into
drivers/net/ethernet/chelsio/ and the necessary Kconfig and Makefile
changes.
CC: Divy Le Ray <divy@chelsio.com>
CC: Dimitris Michailidis <dm@chelsio.com>
CC: Casey Leedom <leedom@chelsio.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Steve Wise