Commit Graph

111 Commits

Author SHA1 Message Date
Suganath Prabu 320e77acb3 scsi: mpt3sas: Irq poll to avoid CPU hard lockups
Issue Description:
We have seen cpu lock up issue from fields if system has greater (more than
96) logical cpu count.  SAS3.0 controller (Invader series) supports at max
96 msix vector and SAS3.5 product (Ventura) supports at max 128 msix
vectors.

This may be a generic issue (if PCI device supports completion on multiple
reply queues).  Let me explain it w.r.t to mpt3sas supported h/w just to
simplify the problem and possible changes to handle such issues. IT HBA
(mpt3sas) supports multiple reply queues in completion path. Driver creates
MSI-x vectors for controller as "min of (FW supported Reply queue, Logical
CPUs)". If submitter is not interrupted via completion on same CPU, there
is a loop in the IO path. This behavior can cause hard/soft CPU lockups, IO
timeout, system sluggish etc.

Example - one CPU (e.g. CPU A) is busy submitting the IOs and another CPU
(e.g. CPU B) is busy with processing the corresponding IO's reply
descriptors from reply descriptor queue upon receiving the interrupts from
HBA.  If the CPU A is continuously pumping the IOs then always CPU B (which
is executing the ISR) will see the valid reply descriptors in the reply
descriptor queue and it will be continuously processing those reply
descriptor in a loop without quitting the ISR handler.

Mpt3sas driver will exit ISR handler if it finds unused reply descriptor in
the reply descriptor queue. Since CPU A will be continuously sending the
IOs, CPU B may always see a valid reply descriptor (posted by HBA Firmware
after processing the IO) in the reply descriptor queue. In worst case,
driver will not quit from this loop in the ISR handler. Eventually, CPU
lockup will be detected by watchdog.

Above mentioned behavior is not common if "rq_affinity" set to 2 or
affinity_hint is honored by irqbalance as "exact". If rq_affinity is set
to 2, submitter will be always interrupted via completion on same CPU.  If
irqbalance is using "exact" policy, interrupt will be delivered to
submitter CPU.

If CPU counts to MSI-X vectors (reply descriptor Queues) count ratio is not
1:1, we still have exposure of issue explained above and for that we don't
have any solution.

Exposure of soft/hard lockup if CPU count is more than MSI-x supported by
device.

If CPUs count to MSI-x vectors count ratio is not 1:1, (Other way, if CPU
counts to MSI-x vector count ratio is something like X:1, where X > 1) then
'exact' irqbalance policy OR rq_affinity = 2 won't help to avoid CPU
hard/soft lockups. There won't be any one to one mapping between CPU to
MSI-x vector instead one MSI-x interrupt (or reply descriptor queue) is
shared with group/set of CPUs and there is a possibility of having a loop
in the IO path within that CPU group and may observe lockups.

For example: Consider a system having two NUMA nodes and each node having
four logical CPUs and also consider that number of MSI-x vectors enabled on
the HBA is two, then CPUs count to MSI-x vector count ratio as 4:1.  e.g.
MSIx vector 0 is affinity to CPU 0, CPU 1, CPU 2 & CPU 3 of NUMA node 0 and
MSI-x vector 1 is affinity to CPU 4, CPU 5, CPU 6 & CPU 7 of NUMA node 1.

numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3                 --> MSI-x 0
node 0 size: 65536 MB
node 0 free: 63176 MB
node 1 cpus: 4 5 6 7                 -->MSI-x 1
node 1 size: 65536 MB
node 1 free: 63176 MB

Assume that user started an application which uses all the CPUs of NUMA
node 0 for issuing the IOs.  Only one CPU from affinity list (it can be any
cpu since this behavior depends upon irqbalance) CPU0 will receive the
interrupts from MSIx vector 0 for all the IOs. Eventually, CPU 0 IO
submission percentage will be decreasing and ISR processing percentage will
be increasing as it is more busy with processing the interrupts.  Gradually
IO submission percentage on CPU 0 will be zero and it's ISR processing
percentage will be 100 percentage as IO loop has already formed within the
NUMA node 0, i.e. CPU 1, CPU 2 & CPU 3 will be continuously busy with
submitting the heavy IOs and only CPU 0 is busy in the ISR path as it
always find the valid reply descriptor in the reply descriptor
queue. Eventually, we will observe the hard lockup here.

Chances of occurring of hard/soft lockups are directly proportional to
value of X. If value of X is high, then chances of observing CPU lockups is
high.

Solution: Use IRQ poll interface defined in " irq_poll.c".  mpt3sas driver
will execute ISR routine in Softirq context and it will always quit the
loop based on budget provided in IRQ poll interface.

In these scenarios (i.e. where CPUs count to MSI-X vectors count ratio is
X:1 (where X > 1)), IRQ poll interface will avoid CPU hard lockups due to
voluntary exit from the reply queue processing based on budget.  Note -
Only one MSI-x vector is busy doing processing.

Irqstat output:

IRQs / 1 second(s)
IRQ#  TOTAL  NODE0   NODE1   NODE2   NODE3  NAME
  44    122871   122871   0       0       0  IR-PCI-MSI-edge mpt3sas0-msix0
  45        0              0           0       0       0  IR-PCI-MSI-edge mpt3sas0-msix1

We use this approach only if cpu count is more than FW supported MSI-x
vector

Signed-off-by: Suganath Prabu <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2019-03-18 17:16:43 -04:00
Suganath Prabu S c6ded86a16 scsi: mpt3sas: Update driver version to 27.102.00.00
Updated driver version to 27.102.00.00 from 27.101.00.00.

Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2019-02-04 22:52:21 -05:00
Suganath Prabu S eb9c7ce560 scsi: mpt3sas: Add support for ATLAS PCIe switch
Add Atlas PCIe Switch Management Port device PNPID,
Vendor Id: 0x1000
device Id: 0x00B2

This device is based on MPI 2.6 spec and it exposes one SES device to
accept management commands for the PCIe switch.

Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2019-02-04 22:52:21 -05:00
Suganath Prabu ed4c1136ac scsi: mpt3sas: Update driver version to 27.101.00.00
Update driver version from 27.100.00.00 to 27.101.00.00.

Signed-off-by: Suganath Prabu <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-12-12 21:29:07 -05:00
Suganath Prabu b899202901 scsi: mpt3sas: Add separate function for aero doorbell reads
Sometimes Aero controllers appears to be returning bad data (0) for
doorbell register read and if retries are performed immediately after the
bad read, they return good data.

Workaround is added to retry read from doorbell registers for maximum three
times if driver get the zero.  Added functions base_readl_aero for Aero IOC
and base_readl for gen35 and other controllers.

Signed-off-by: Suganath Prabu <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-12-12 21:29:07 -05:00
Suganath Prabu cc68e6077b scsi: mpt3sas: Introduce flag for aero based controllers
Adding flag "is_aero_ioc" to differentiate aero based controllers from
other gen35 controllers.

Signed-off-by: Suganath Prabu <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-12-12 21:29:07 -05:00
Suganath Prabu 8dbb748d4d scsi: mpt3sas: Bump driver version to 27.100.00.00
Modify driver version to 27.100.00.00 (which is equivalent to PH8 OOB
driver)

Signed-off-by: Suganath Prabu <suganath-prabu.subramani@broadcom.com>
Reviewed-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-11-06 20:16:01 -05:00
Suganath Prabu f4305749ca scsi: mpt3sas: Separate out mpt3sas_wait_for_ioc
No functional changes. This section of code "wait for IOC to be
operational" is used in many places across the driver.  Factor this code
out into a new mpt3sas_wait_for_ioc().

Signed-off-by: Suganath Prabu <suganath-prabu.subramani@broadcom.com>
Reviewed-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-11-06 20:16:01 -05:00
Suganath Prabu 02abcbc25a scsi: mpt3sas: Added new #define variable IOC_OPERATIONAL_WAIT_COUNT
Added new #define variable IOC_OPERATIONAL_WAIT_COUNT and it replaces hard
coded value '10' in all the places where driver is waiting for the IOC to
become operational.

Signed-off-by: Suganath Prabu <suganath-prabu.subramani@broadcom.com>
Reviewed-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-11-06 20:16:01 -05:00
Suganath Prabu ff92b9dd92 scsi: mpt3sas: Update MPI headers to support Aero controllers
Updating MPI headers to the latest version 2.6.7 to add support to the
driver to detect the new 3816 and 3916 chip based controllers.  Separate
out firmware image data from mpi2_ioc.h to new file mpi2_image.h

Signed-off-by: Suganath Prabu <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-11-06 20:16:01 -05:00
Joe Perches 16e8b9631a scsi: mpt3sas: Remove unused macro MPT3SAS_FMT
All the uses have been removed, delete the macro.

Signed-off-by: Joe Perches <joe@perches.com>
Acked-by: Suganath Prabu <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-10-10 22:00:44 -04:00
Joe Perches 645a20c682 scsi: mpt3sas: Add ioc_<level> logging macros
These macros can help identify specific logging uses and eventually perhaps
reduce object sizes.

Signed-off-by: Joe Perches <joe@perches.com>
Acked-by: Suganath Prabu <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-10-10 22:00:43 -04:00
Bart Van Assche c7a3570588 scsi: mpt3sas: Split _base_reset_handler(), mpt3sas_scsih_reset_handler() and mpt3sas_ctl_reset_handler()
Split each of these functions in three functions - one function per reset
phase. This patch does not change any functionality but makes the code
easier to read.

Note: it is much easier to review the git diff -w output after having
applied this patch than by reviewing the patch itself.

Signed-off-by: Bart Van Assche <bart.vanassche@wdc.com>
Cc: Sathya Prakash <sathya.prakash@broadcom.com>
Cc: Chaitra P B <chaitra.basappa@broadcom.com>
Cc: Suganath Prabu Subramani <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-06-19 22:02:25 -04:00
Bart Van Assche 982ea6f9ff scsi: mpt3sas: Fix a race condition in mpt3sas_base_hard_reset_handler()
Since ioc->shost_recovery is set after ioc->reset_in_progress_mutex is
obtained, if concurrent resets are issued there is a short time during
which ioc->reset_in_progress_mutex is locked and ioc->shost_recovery ==
0. Avoid that this can cause trouble by unconditionally locking
ioc->shost_recovery.

Signed-off-by: Bart Van Assche <bart.vanassche@wdc.com>
Cc: Sathya Prakash <sathya.prakash@broadcom.com>
Cc: Chaitra P B <chaitra.basappa@broadcom.com>
Cc: Suganath Prabu Subramani <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-06-19 22:02:25 -04:00
Bart Van Assche 84203b3561 scsi: mpt3sas: Introduce struct mpt3sas_nvme_cmd
Make _base_build_nvme_prp() easier to read by introducing a structure
to access NVMe command fields.

Signed-off-by: Bart Van Assche <bart.vanassche@wdc.com>
Cc: Sathya Prakash <sathya.prakash@broadcom.com>
Cc: Chaitra P B <chaitra.basappa@broadcom.com>
Cc: Suganath Prabu Subramani <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-06-19 22:02:25 -04:00
Chaitra P B 2cd4bf89f7 scsi: mpt3sas: Update driver version "26.100.00.00"
Updated driver version to "26.100.00.00"

Signed-off-by: Chaitra P B <chaitra.basappa@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-06-18 21:05:32 -04:00
Chaitra P B 2b48be6568 scsi: mpt3sas: As per MPI-spec, use combined reply queue for SAS3.5 controllers when HBA supports more than 16 MSI-x vectors.
Presently driver is using combined reply queue feature when MSI-x vectors >
8 for both SAS3 and SAS3.5 controllers.  But as per MPI-spec,

1. For SAS3 controllers, driver should use combined reply queue when HBA
supports more than 8 MSI-x vectors.

2. For SAS3.5 controllers, driver should use combined reply queue when HBA
supports more than 16 MSI-x vectors.

Modified driver code to use combined reply queue for SAS3 controllers when
HBA supports > 8 MSI-x vectors and for SAS3.5 controllers when HBA supports
> 16 MSI-x vectors.

Signed-off-by: Chaitra P B <chaitra.basappa@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-06-18 21:05:32 -04:00
Chaitra P B d37306ca0e scsi: mpt3sas: Fix, False timeout prints for ioctl and other internal commands during controller reset.
When an ioctl is sent to FW, and if there is a controller reset issued
before ioctl gets completed, then in controller reset path all the pending
ioctl commands are terminated from "mpt3sas_ctl_reset_handler" function.
This will wake up the waiting ioctl commands in ioctl path and print
timeouts which are actually not timeouts.

Introduced "mpt3sas_base_check_cmd_timeout" function to check and print
whether command got timed out (or) terminated due to Host reset.

Signed-off-by: Chaitra P B <chaitra.basappa@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-06-18 21:05:32 -04:00
Chaitra P B f6972d7180 scsi: mpt3sas: Update driver version "25.100.00.00"
Update driver version to match OOB/internal driver version.

Signed-off-by: Chaitra P B <chaitra.basappa@broadcom.com>
Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-05-08 00:40:05 -04:00
Chaitra P B c1a6c5ac42 scsi: mpt3sas: For NVME device, issue a protocol level reset
1) Manufacturing Page 11 contains parameters to control internal
   firmware behavior. Based on AddlFlags2 field FW/Driver behaviour can
   be changed, (flag tm_custom_handling is used for this)

a) For PCIe device, protocol level reset should be used if flag
   tm_custom_handling is 0.  Since Abort Task Set, LUN reset and Target
   reset will result in a protocol level reset. Drivers should issue
   only one type of this reset, if that fails then it should escalate to
   a controller reset (diag reset/OCR).

b) If the driver has control over the TM reset timeout value, then
   driver should use the value exposed in PCIe Device Page 2 for pcie
   device (field ControllerResetTO).

Signed-off-by: Chaitra P B <chaitra.basappa@broadcom.com>
Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-05-08 00:39:49 -04:00
Chaitra P B 3d29ed85fc scsi: mpt3sas: Report Firmware Package Version from HBA Driver.
Added function _base_display_fwpkg_version, which sends FWUpload request
to pull FW package version from FW Image Header.  Now driver prints FW
package version in addition to FW version if the PackageVersion is
valid.

Signed-off-by: Chaitra P B <chaitra.basappa@broadcom.com>
Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-05-08 00:34:20 -04:00
Chaitra P B 22a923c315 scsi: mpt3sas: Cache enclosure pages during enclosure add.
In function _scsih_add_device, for each device connected to an
enclosure, driver reads the enclosure page(To get details like enclosure
handle, enclosure logical ID, enclosure level etc.)

With this patch, instead of reading enclosure page everytime, driver
maintains a list for enclosure device(During enclosure add event,
enclosure device is added to the list and removed from the list on
delete events) and uses the enclosure page from the list.

Signed-off-by: Chaitra P B <chaitra.basappa@broadcom.com>
Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-05-08 00:34:20 -04:00
Chaitra P B 93204b782a scsi: mpt3sas: Lockless access for chain buffers.
Introduces Chain lookup table/tracker and implements accessing chain
buffer using smid.  Removed link list based access of chain buffer which
requires lock and allocated as many chains needed.

Signed-off-by: Chaitra P B <chaitra.basappa@broadcom.com>
Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-05-08 00:34:19 -04:00
Chaitra P B cd33223b59 scsi: mpt3sas: Pre-allocate RDPQ Array at driver boot time.
Instead of allocating RDPQ array (This stores the address's of each RDPQ
pools) at run time, now it will be allocated once during driver load
time and same will be reused during host reset operation also (instead
of allocating & freeing this buffer on the fly during every host reset
operation) and then freed during driver unload.

Signed-off-by: Chaitra P B <chaitra.basappa@broadcom.com>
Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-05-08 00:34:19 -04:00
Chaitra P B cf6bf9710c scsi: mpt3sas: Bug fix for big endian systems.
This patch fixes sparse warnings and bugs on big endian systems.

Signed-off-by: Chaitra P B <chaitra.basappa@broadcom.com>
Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-05-08 00:34:19 -04:00
James Bottomley 2e1f44f6ad Merge branch 'fixes' into misc
Somewhat nasty merge due to conflicts between "33b28357dd00 scsi:
qla2xxx: Fix Async GPN_FT for FCP and FC-NVMe scan" and "2b5b96473efc
scsi: qla2xxx: Fix FC-NVMe LUN discovery"

Merge is non-trivial and has been verified by Qlogic (Cavium)

Signed-off-by: James E.J. Bottomley <jejb@linux.vnet.ibm.com>
2018-04-03 17:38:39 -07:00
Arnd Bergmann 6f9e09fd64 scsi: mpt3sas: clarify mmio pointer types
The newly added code mixes up phys_addr_t/resource_size_t with dma_addr_t
and void pointers, as seen from these compiler warning:

drivers/scsi/mpt3sas/mpt3sas_base.c: In function '_base_get_chain_phys':
drivers/scsi/mpt3sas/mpt3sas_base.c:235:21: error: cast to pointer from integer of different size [-Werror=int-to-pointer-cast]
  base_chain_phys  = (void *)ioc->chip_phys + MPI_FRAME_START_OFFSET +
                     ^
drivers/scsi/mpt3sas/mpt3sas_base.c: In function '_clone_sg_entries':
drivers/scsi/mpt3sas/mpt3sas_base.c:427:20: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
    sgel->Address = (dma_addr_t)dst_addr_phys;
                    ^
drivers/scsi/mpt3sas/mpt3sas_base.c:438:7: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
       (dma_addr_t)buff_ptr_phys;
       ^
drivers/scsi/mpt3sas/mpt3sas_base.c:444:10: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
          (dma_addr_t)buff_ptr_phys;

Both dma_addr_t and phys_addr_t may be wider than a pointer, so we must
avoid the conversion to pointer types. This also helps readability.

A second problem is treating MMIO addresses from a 'struct resource'
as addresses that can be used for DMA on that device. In almost all
cases, those are the same, but on some of the more obscure architectures,
PCI memory address 0 is mapped into the CPU address space at a nonzero
offset. I don't have a good fix for that, so I'm adding a comment here,
plus a WARN_ON() that triggers whenever the phys_addr_t number is
outside of the low 32-bit address space and causes a straight overflow
when assigned to the 32-bit sgel->Address.

Fixes: 182ac784b4 ("scsi: mpt3sas: Introduce Base function for cloning.")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Sreekanth Reddy <Sreekanth.Reddy@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-03-06 13:06:10 -05:00
Suganath Prabu S 40114bde97 scsi: mpt3sas: Do not use 32-bit atomic request descriptor for Ventura controllers.
Sending I/O through 32 bit descriptors to Ventura series of controller
results in IO timeout on certain conditions. This error only occurs on
systems with high I/O activity.

Changes in this patch will prevent driver from using 32 bit descriptor
and use 64 bit Descriptors

Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Reviewed-by: Tomas Henzl <thenzl@redhat.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-02-27 22:26:16 -05:00
Suganath Prabu Subramani 182ac784b4 scsi: mpt3sas: Introduce Base function for cloning.
All scsi IO's and config request's data buffer and sgl are cloned to
system memory in _clone_sg_entries before submitting it to firmware.

Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-02-27 22:26:14 -05:00
Suganath Prabu Subramani 22ae5a3c25 scsi: mpt3sas: Introduce API to get BAR0 mapped buffer address
For MPI Endpoint/Mcpu, driver should double buffer data buffer/SGLs.
This is normally copied from host to internal memory of IOC by DMA
engine of PCI device. Since the interface to DMA from host to mCPU is
not present for Mcpu/MPI Endpoint device, driver does double copy of
those buffers directly to the mCPU memory region via BAR0 region.

Introduced API to calculate and return BAR0 mapped host buffer's
physical and virtual address for the provided smid.

Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-02-27 22:26:13 -05:00
Suganath Prabu Subramani c520691b38 scsi: mpt3sas: Add PCI device ID for Andromeda.
Add device ID and flag for Andromeda/MPI Endpoint.

[mkp: typo]

Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-02-27 22:26:12 -05:00
Sreekanth Reddy c666d3be99 scsi: mpt3sas: wait for and flush running commands on shutdown/unload
This patch finishes all outstanding SCSI IO commands (but not other commands,
e.g., task management) in the shutdown and unload paths.

It first waits for the commands to complete (this is done after setting
'ioc->remove_host = 1 ', which prevents new commands to be queued) then it
flushes commands that might still be running.

This avoids triggering error handling (e.g., abort command) for all commands
possibly completed by the adapter after interrupts disabled.

[mauricfo: introduced something in commit message.]

Signed-off-by: Sreekanth Reddy <sreekanth.reddy@broadcom.com>
Tested-by: Mauricio Faria de Oliveira <mauricfo@linux.vnet.ibm.com>
Signed-off-by: Mauricio Faria de Oliveira <mauricfo@linux.vnet.ibm.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-02-21 22:59:39 -05:00
Suganath Prabu Subramani dbec4c9040 scsi: mpt3sas: lockless command submission
Performance improvement using block layer tag.

Curent driver gets scsiio tracker and free smid from link list and array
based tracking managed by driver.  Accessing list in main io path is
performance pentaly because of protection using spinlock
"scsi_lookup_lock".

In this patch:

1. Driver removes all link list access from main io path and
   use scmd->request->tag to get free smid.

2. Instead of holding 'struct scsiio_tracker' in its own pool
   driver can embed it into the scsi command.

Driver provides cmd_size in scsi_host_template, so that struct
scsiio_tracker is preallocated by scsi mid layer for each scsi command.

Signed-off-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-01-10 23:25:04 -05:00
Hannes Reinecke 6da999fe5a scsi: mpt3sas: simplify mpt3sas_scsi_issue_tm()
Move the check for outstanding commands out of the function allowing us
to simplify the overall code.

Signed-off-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-01-10 23:25:03 -05:00
Hannes Reinecke 12e7c6782b scsi: mpt3sas: Introduce mpt3sas_get_st_from_smid()
Abstract accesses to the scsi_lookup array by introducing
mpt3sas_get_st_from_smid().

Signed-off-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-01-10 23:25:01 -05:00
Arnd Bergmann d8335ae2b4 scsi: mpt3sas: fix dma_addr_t casts
The newly added base_make_prp_nvme function triggers a build warning on
some 32-bit configurations:

drivers/scsi/mpt3sas/mpt3sas_base.c: In function 'base_make_prp_nvme':
drivers/scsi/mpt3sas/mpt3sas_base.c:1664:13: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
  msg_phys = (dma_addr_t)mpt3sas_base_get_pcie_sgl_dma(ioc, smid);

After taking a closer look, I found that the problem is that the new
code mixes up pointers and dma_addr_t values unnecessarily.

This changes it to use the correct types consistently, which lets us get
rid of a lot of type casts in the process. I'm also renaming some
variables to avoid confusion between physical and dma address spaces
that are often distinct.

Fixes: 016d5c35e2 ("scsi: mpt3sas: SGL to PRP Translation for I/Os to NVMe devices")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Sathya Prakash Veerichetty <sathya.prakash@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2017-11-06 23:01:53 -05:00
Suganath Prabu Subramani 62d2f77438 scsi: mpt3sas: Update mpt3sas driver version.
Updated mpt3sas driver version to 17.100.00.00

Signed-off-by: Chaitra P B <chaitra.basappa@broadcom.com>
Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2017-11-03 12:20:53 -04:00
Suganath Prabu Subramani d1b01d14b7 scsi: mpt3sas: Set NVMe device queue depth as 128
Sets nvme device queue depth, name and displays device capabilities

Signed-off-by: Chaitra P B <chaitra.basappa@broadcom.com>
Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2017-11-03 12:20:52 -04:00
Suganath Prabu Subramani c102e00cf4 scsi: mpt3sas: API 's to support NVMe drive addition to SML
Below Functions are added in various paths to support NVMe drive
addition.

_scsih_pcie_add_device
_scsih_pcie_device_add
_scsih_pcie_device_init_add
_scsih_check_pcie_access_status
_scsih_pcie_check_device

mpt3sas_get_pdev_by_handle

mpt3sas_config_get_pcie_device_pg0
mpt3sas_config_get_pcie_device_pg2

Signed-off-by: Chaitra P B <chaitra.basappa@broadcom.com>
Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2017-11-03 12:20:52 -04:00
Suganath Prabu Subramani aff39e6121 scsi: mpt3sas: Added support for nvme encapsulated request message.
* Mpt3sas driver uses the NVMe Encapsulated Request message to send an
  NVMe command to an NVMe device attached to the IOC.

* Normal I/O commands like reads and writes are passed to the
  controller as SCSI commands and the controller has the ability to
  translate the commands to NVMe equivalent.

* This encapsulated NVMe command is used by applications to send
  direct NVMe commands to NVMe drives.

Signed-off-by: Chaitra P B <chaitra.basappa@broadcom.com>
Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2017-11-03 12:20:52 -04:00
Suganath Prabu Subramani 016d5c35e2 scsi: mpt3sas: SGL to PRP Translation for I/Os to NVMe devices
* Added support for translating the SGLs associated with incoming
commands either to IEE SGL or NVMe PRPs for NVMe devices.

* The hardware translation of IEEE SGL to NVMe PRPs has limitations
and if a command cannot be translated by hardware then it will go to
firmware and the firmware needs to translate it. This will have a
performance impact. To avoid that, the driver proactively checks
whether the translation will be done in hardware or not. If not, then
driver translates.

[mkp: clarified commit message]

Signed-off-by: Chaitra P B <chaitra.basappa@broadcom.com>
Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2017-11-03 12:20:52 -04:00
Suganath Prabu Subramani d88e1eaba6 scsi: mpt3sas: Add nvme device support in slave alloc, target alloc and probe
1) Added support for probing pcie device and adding NVMe drives to SML
and driver's internal list pcie_device_list.

2) Added support for determing NVMe as boot device.

3) Added nvme device support for call back functions scan_finished
target_alloc,slave_alloc,target destroy and slave destroy.

 a) During scan, pcie devices are probed and added to SML to drivers
internal list.

 b) target_alloc & slave alloc API's allocates resources for
(MPT3SAS_TARGET & MPT3SAS_DEVICE) private datas and holds information
like handle, target_id etc.

 c) slave_destroy & target_destroy are called when driver unregisters
or removes device. Also frees allocated resources and info.

Signed-off-by: Chaitra P B <chaitra.basappa@broadcom.com>
Signed-off-by: Suganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2017-11-03 12:20:52 -04:00
Sreekanth Reddy 09a50f43a5 scsi: mpt3sas: Bump mpt3sas driver version to v16.100.00.00
Bump mpt3sas driver version to v16.100.00.00

Signed-off-by: Sreekanth Reddy <Sreekanth.Reddy@broadcom.com>
Reviewed-by: Tomas Henzl <thenzl@redhat.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2017-10-11 14:08:07 -04:00
Sreekanth Reddy 7588895646 scsi: mpt3sas: Display chassis slot information of the drive
Display chassis slot information along with other drive location
parameters such as slot number and connector name in the logs if
chassis slot validity bit is set in 'SAS Enclosure Page 0'.

Signed-off-by: Sreekanth Reddy <Sreekanth.Reddy@broadcom.com>
Reviewed-by: Tomas Henzl <thenzl@redhat.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2017-10-11 14:08:07 -04:00
Sreekanth Reddy 06f5f976a6 scsi: mpt3sas: Reduce memory footprint in kdump kernel
To reduce the memory footprint of the driver in the kdump kernel, we
apply the following settings when reset_devices is set:

1. Use single MSI-x vector.
2. Disable RDPQ mode.
3. Set sg_table_size to 32 by default.
4) Set SCSI IO Queue depth to 200.

[mkp: fixed commit message]

Signed-off-by: Sreekanth Reddy <Sreekanth.Reddy@broadcom.com>
Reviewed-by: Tomas Henzl <thenzl@redhat.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2017-10-11 14:08:07 -04:00
Arnd Bergmann bbfd8e8b24 scsi: mpt3sas: fix format overflow warning
We print the driver name into one string and then add and ID
and copy it into a second string of the same length, at which
point gcc complains about a possible overflow:

drivers/scsi/mpt3sas/mpt3sas_scsih.c: In function '_scsih_probe':
drivers/scsi/mpt3sas/mpt3sas_scsih.c:8884:21: error: '_cm' directive writing 3 bytes into a region of size between 1 and 32 [-Werror=format-overflow=]
printf(ioc->name, "%s_cm%d", ioc->driver_name, ioc->id);
                  ^~~~~~~~~
drivers/scsi/mpt3sas/mpt3sas_scsih.c:8884:21: note: directive argument in the range [0, 255]
drivers/scsi/mpt3sas/mpt3sas_scsih.c:8884:2: note: 'sprintf' output between 5 and 38 bytes into a destination of size 32
  sprintf(ioc->name, "%s_cm%d", ioc->driver_name, ioc->id);
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Making the first string shorter is sufficient to avoid the
warning here, as we know it can only contain either "mpt2sas"
or "mpt3sas".

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2017-08-07 14:04:01 -04:00
Stephen Boyd ad61dd303a scripts/spelling.txt: add regsiter -> register spelling mistake
This typo is quite common.  Fix it and add it to the spelling file so
that checkpatch catches it earlier.

Link: http://lkml.kernel.org/r/20170317011131.6881-2-sboyd@codeaurora.org
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2017-05-08 17:15:13 -07:00
Bart Van Assche 8893cf6cb1 scsi: mpt3sas: Avoid sleeping in interrupt context
Commit 669f044170 ("scsi: srp_transport: Move queuecommand() wait code
to SCSI core") can make scsi_internal_device_block() sleep.  However,
the mpt3sas driver can call this function from an interrupt
handler. Hence add a second argument to scsi_internal_device_block()
that restores the old behavior of this function for the mpt3sas handler.

The call chain that triggered an "IRQ handler enabled interrupts"
complaint is as follows:

_base_interrupt()
-> _base_async_event()
   -> mpt3sas_scsih_event_callback()
      -> _scsih_check_topo_delete_events()
         -> _scsih_block_io_to_children_attached_directly()
            -> _scsih_block_io_device()
               -> _scsih_internal_device_block()
                  -> scsi_internal_device_block()

Reported-by: Omar Sandoval <osandov@osandov.com>
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Cc: Omar Sandoval <osandov@osandov.com>
Cc: Hannes Reinecke <hare@suse.com>
Cc: Sagi Grimberg <sagi@grimberg.me>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sathya Prakash <sathya.prakash@broadcom.com>
Cc: Chaitra P B <chaitra.basappa@broadcom.com>
Cc: Suganath Prabu Subramani <suganath-prabu.subramani@broadcom.com>
Cc: Sreekanth Reddy <Sreekanth.Reddy@broadcom.com>
Cc: <stable@vger.kernel.org> # v4.10+
Tested-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2017-03-01 21:52:13 -05:00
Hannes Reinecke 1d55abc0e9 scsi: mpt3sas: switch to pci_alloc_irq_vectors
Cleanup the MSI-X handling allowing us to use the PCI-layer provided
vector allocation.

Signed-off-by: Hannes Reinecke <hare@suse.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Acked-by: Sreekanth Reddy <sreekanth.reddy@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2017-02-23 17:01:05 -05:00
Linus Torvalds cdc194705d SCSI misc on 20170220
This update includes the usual round of major driver updates (ncr5380,
 ufs, lpfc, be2iscsi, hisi_sas, storvsc, cxlflash, aacraid,
 megaraid_sas, ).  There's also an assortment of minor fixes and the
 major update of switching a bunch of drivers to pci_alloc_irq_vectors
 from Christoph.
 
 Signed-off-by: James E.J. Bottomley <jejb@linux.vnet.ibm.com>
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2
 
 iQIcBAABAgAGBQJYq5adAAoJEAVr7HOZEZN4bjUP/Atk7CSZVnC75pcYmncbEGCx
 ysOlEHK4uW2HhiAYk3PlYMk+pKrMHet2zsbbM9PHJfopdOHZ7Sq1+UZZVeqE1Zun
 8pe0NhON+fZx7XAnevdEvnSSULQZ+AGfjZO72iUwkJiN3ozYaFtCITOyn49l4GpR
 ra9emskBh7CQOFW2voGn1AKeDijPYGx3+TO4AUrWjVMiByR06gb1bmImx+ljiUrs
 jzRJPfrt90ORcTdpMateyN2EXxudcASMhX03SJ6fRI84hPAhMCROMbTv8RnzOTE4
 DPbnvbYUowlHt43iUhJHSwGdkRRaRBnkzQENBp1fNrNzZgF6vB7+kShxbonrYB2p
 gC4ewaJr0BNj+HsUnvTpe3WseiPOcfsnBsKilPLKBlm2dCKEXqFox/dj/T1uexxg
 HoyFrl3u8fyEqVHrzRS4M9t/njWh0NFmXxb0wBdj+lkVFTRErGSKQ8SfOqshuSGs
 P8NN88jy8vC7uqgzKBJ+UH3ehzn3qfBxasFHIC/e2awY9FqKjHGTxKMmSVpjXVxy
 wCvE2FQ3k/qEj2XSM6f7/NGytlSOlju5q1rFtHPW2M+TFSh0LJWCnmVjR/Zle9em
 pBWmtIgCv8W5b41zL2H94nLWAZbfdrrNU/XnX88l47LKnmorte/PGhpxu36NEsMS
 VCgreQmFMdMRY+WzDWl1
 =cBQx
 -----END PGP SIGNATURE-----

Merge tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi

Pull SCSI updates from James Bottomley:
 "This update includes the usual round of major driver updates (ncr5380,
  ufs, lpfc, be2iscsi, hisi_sas, storvsc, cxlflash, aacraid,
  megaraid_sas, ...).

  There's also an assortment of minor fixes and the major update of
  switching a bunch of drivers to pci_alloc_irq_vectors from Christoph"

* tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (188 commits)
  scsi: megaraid_sas: handle dma_addr_t right on 32-bit
  scsi: megaraid_sas: array overflow in megasas_dump_frame()
  scsi: snic: switch to pci_irq_alloc_vectors
  scsi: megaraid_sas: driver version upgrade
  scsi: megaraid_sas: Change RAID_1_10_RMW_CMDS to RAID_1_PEER_CMDS and set value to 2
  scsi: megaraid_sas: Indentation and smatch warning fixes
  scsi: megaraid_sas: Cleanup VD_EXT_DEBUG and SPAN_DEBUG related debug prints
  scsi: megaraid_sas: Increase internal command pool
  scsi: megaraid_sas: Use synchronize_irq to wait for IRQs to complete
  scsi: megaraid_sas: Bail out the driver load if ld_list_query fails
  scsi: megaraid_sas: Change build_mpt_mfi_pass_thru to return void
  scsi: megaraid_sas: During OCR, if get_ctrl_info fails do not continue with OCR
  scsi: megaraid_sas: Do not set fp_possible if TM capable for non-RW syspdIO, change fp_possible to bool
  scsi: megaraid_sas: Remove unused pd_index from megasas_build_ld_nonrw_fusion
  scsi: megaraid_sas: megasas_return_cmd does not memset IO frame to zero
  scsi: megaraid_sas: max_fw_cmds are decremented twice, remove duplicate
  scsi: megaraid_sas: update can_queue only if the new value is less
  scsi: megaraid_sas: Change max_cmd from u32 to u16 in all functions
  scsi: megaraid_sas: set pd_after_lb from MR_BuildRaidContext and initialize pDevHandle to MR_DEVHANDLE_INVALID
  scsi: megaraid_sas: latest controller OCR capability from FW before sending shutdown DCMD
  ...
2017-02-21 11:51:42 -08:00