This patch fixes a isoc transfer bug reported by Sander Eikelenboom.
When ep->skip is set, endpoint ring dequeue pointer should be updated
when processed every missed td. Although ring dequeue pointer will also
be updated when ep->skip is clear, leave it intact during missed tds
processing may cause two issues:
1). If the very next valid transfer following missed tds is a short
transfer, its actual_length will be miscalculated;
2). If there are too many missed tds during transfer, new inserted tds
may found the transfer ring full and urb enqueue fails.
Reported-by: Sander Eikelenboom <linux@eikelenboom.it>
Tested-by: Sander Eikelenboom <linux@eikelenboom.it>
Signed-off-by: Andiry Xu <andiry.xu@amd.com>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The code to increment the TRB pointer has a slight ambiguity that could
lead to a bug on different compilers. The ANSI C specification does not
specify the precedence of the assignment operator over the postfix
operator. gcc 4.4 produced the correct code (increment the pointer and
assign the value), but a MIPS compiler that one of John's clients used
assigned the old (unincremented) value.
Remove the unnecessary assignment to make all compilers produce the
correct assembly.
Signed-off-by: John Youn <johnyoun@synopsys.com>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
To tell the host controller that there are transfers on the endpoint
rings, we need to ring the endpoint doorbell. This is a PCI MMIO write,
which can be delayed until another register read is queued.
The previous code would flush the doorbell write by reading the doorbell
register after the write. This may take time, and it's not necessary to
force the host controller to know about the transfers right away. Don't
flush the doorbell register writes.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The interrupter register set includes a register that says whether interrupts
are pending for each event ring (the IP bit). Each MSI-X vector will get its
own interrupter set with separate IP bits. The status register includes an
"Event Interrupt (EINT)" bit that is set when an IP bit is set in any of the
interrupters.
When PCI interrupts are used, the EINT bit exactly mirrors the IP bit in the
single interrupter set, and it is a waste of time to check both registers when
trying to figure out if the xHC interrupted or another device on the shared IRQ
line interrupted. Only check the IP bit to reduce register reads.
The IP bit is automatically cleared by the xHC when MSI or MSI-X is enabled. It
doesn't make sense to read that register to check for shared interrupts (since
MSI and MSI-X aren't shared). It also doesn't make sense to write to that
register to clear the IP bit, since it is cleared by the hardware.
We can tell whether MSI or MSI-X is enabled by looking at the irq number in
hcd->irq. If it's -1, we know MSI or MSI-X is enabled.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Now that the event handler functions no longer use xhci_set_hc_event_deq()
to update the event ring dequeue pointer, that function is not used by
anything in xhci-ring.c. Move that function into xhci-mem.c and make it
static.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The xHCI specification suggests that writing the hardware event ring dequeue
pointer register too often can be an expensive operation for the xHCI hardware
to manage. It suggests minimizing the number of writes to that register.
Originally, the driver wrote the event ring dequeue pointer after each
event was processed. Depending on how the event ring moderation register
is set up and how fast the transfers are completing, there may be several
events processed for each interrupt. This patch makes the hardware event
ring dequeue pointer be written only once per interrupt.
Make the transfer event handler and port status event handler only write
the software event ring dequeue pointer. Move the updating of the
hardware event ring dequeue pointer into the interrupt function. Move the
contents of xhci_set_hc_event_deq() into the interrupt handler. The
interrupt handler must clear the event handler busy flag, so it might as
well also write the dequeue pointer to the same register. This eliminates
two 32-bit PCI reads and two 32-bit PCI writes.
Reported-by: Andiry Xu <andiry.xu@amd.com>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
xhci_handle_event() is now only called from within xhci-ring.c, so make it
static.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Remove a duplicate register read of the interrupt pending register from
xhci_irq(). Also, remove waiting on the posted write of that register.
The host will see it eventually. It will probably read the register
itself before deciding whether to interrupt the system again, forcing the
posted write to complete.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When we move xhci_work() into xhci_irq(), we don't need to read the operational
register status field twice.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Most of the work for interrupt handling is done in xhci-ring.c, so it makes
sense to move the functions that are first called when an interrupt happens
(xhci_irq() or xhci_msi_irq()) into xhci-ring.c, so that the compiler can better
optimize them.
Shorten some lines to make it pass checkpatch.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
I've been using perf to measure the top symbols while transferring 1GB of data
on a USB 3.0 drive with dd. This is using the raw disk with /dev/sdb, with a
block size of 1K.
During performance testing, the top symbol was xhci_triad_to_transfer_ring(), a
function that should return immediately if streams are not enabled for an
endpoint. It turned out that the functions to find the endpoint ring was
defined in xhci-mem.c and used in xhci-ring.c and xhci-hcd.c. I moved a copy of
xhci_triad_to_transfer_ring() and xhci_urb_to_transfer_ring() into xhci-ring.c
and declared them static. I also made a static version of
xhci_urb_to_transfer_ring() in xhci.c.
This improved throughput on a 1GB read of the raw disk with dd from
186MB/s to 195MB/s, and perf reported sampling the xhci_triad_to_transfer_ring()
0.06% of the time, rather than 9.26% of the time.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This patch implements isochronous urb enqueue and interrupt handler part.
When an isochronous urb is passed to xHCI driver, first check the transfer
ring to guarantee there is enough room for the whole urb. Then update the
start_frame and interval field of the urb. Always assume URB_ISO_ASAP
is set, and never use urb->start_frame as input.
The number of isoc TDs is equal to urb->number_of_packets. One isoc TD is
consumed every Interval. Each isoc TD consists of an Isoch TRB chained to
zero or more Normal TRBs.
Call prepare_transfer for each TD to do initialization; then calculate the
number of TRBs needed for each TD. If the data required by an isoc TD is
physically contiguous (not crosses a page boundary), then only one isoc TRB
is needed; otherwise one or more additional normal TRB shall be chained to
the isoc TRB by the host.
Set TRB_IOC to the last TRB of each isoc TD. Do not ring endpoint doorbell
to start xHC procession until all the TDs are inserted to the endpoint
transer ring.
In irq handler, update urb status and actual_length, increase
urb_priv->td_cnt. When all the TDs are completed(td_cnt is equal to
urb_priv->length), giveback the urb to usbcore.
Signed-off-by: Andiry Xu <andiry.xu@amd.com>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Add urb_priv data structure to xHCI driver. This structure allows multiple
xhci TDs to be linked to one urb, which is essential for isochronous
transfer. For non-isochronous urb, only one TD is needed for one urb;
for isochronous urb, the TD number for the urb is equal to
urb->number_of_packets.
The length field of urb_priv indicates the number of TDs in the urb.
The td_cnt field indicates the number of TDs already processed by xHC.
When td_cnt matches length, the urb can be given back to usbcore.
When an urb is dequeued or cancelled, add all the unprocessed TDs to the
endpoint's cancelled_td_list. When process a cancelled TD, increase
td_cnt field. When td_cnt equals urb_priv->length, giveback the
cancelled urb.
Signed-off-by: Andiry Xu <andiry.xu@amd.com>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This patch adds mechanism to process Missed Service Error Event.
Sometimes the xHC is unable to process the isoc TDs in time, it will
generate Missed Service Error Event. In this case some TDs on the ring are
not processed and missed. When encounter a Missed Servce Error Event, set
the skip flag of the ep, and process the missed TDs until reach the next
processed TD, then clear the skip flag.
Signed-off-by: Andiry Xu <andiry.xu@amd.com>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This patch adds new cases to trb_comp_code switch, and moves
the switch judgment ahead of fetching td.
Signed-off-by: Andiry Xu <andiry.xu@amd.com>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This patch moves the bulk and interrupt td processing part in
handle_tx_event() into a separate function process_bulk_intr_td().
Signed-off-by: Andiry Xu <andiry.xu@amd.com>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This patch moves the ctrl td processing part in handle_tx_event()
into a separate function process_ctrl_td().
Signed-off-by: Andiry Xu <andiry.xu@amd.com>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This patch moves the td universal processing part in handle_tx_event()
into a separate function finish_td().
if finish_td() returns 1, it indicates the urb can be given back.
Signed-off-by: Andiry Xu <andiry.xu@amd.com>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Commit 6c12db90f1 also seems to have
introduced a bug that is triggered when the command ring is about to wrap.
The inc_enq() function will not have moved the enqueue pointer past the
link TRB. It is supposed to be moved past the link TRB in prepare_ring(),
which should be called before a TD is enqueued. However, the
queue_command() function never calls the prepare_ring() function because
prepare_ring() is only supposed to be used for endpoint rings. That means
the enqueue pointer will not be moved past the link TRB, and will get
overwritten.
The fix is to make queue_command() call prepare_ring() with a fake
endpoint status (set to running). Then the enqueue pointer will get moved
past the link TRB.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Commit 6c12db90f1 introduced a bug for
control transfers. The patch was supposed to change when the link TRBs at
the end of each ring segment were given to the hardware. If a transfer
descriptor (TD) ended just before the link TRB, the code wouldn't give
back the link TRB to the hardware; instead it would be given back in
prepare_ring() just before the next TD was enqueued at the top of the
ring.
Unfortunately, the code relied on checking the chain bit of the TRB to
determine whether the TD ended just before the link TRB. It assumed that
the ring enqueuing code would call prepare_ring() before enqueuing the
next TD. However, control transfers are made of multiple TDs, and
prepare_ring() is only called once before enqueuing two or three TDs.
If the first or second TD of the control transfer ended just before the
link TRB, then the code in inc_enq() would not move the enqueue pointer
past the link TRB, and the link TRB would get overwritten. This would
cause the xHCI driver to start writing to memory past the ring segment,
and eventually the system would crash or hang.
The fix is to add a flag to inc_enq() that says whether the caller will
enqueue more TDs before calling prepare_ring(). If the chain bit is
cleared (meaning this is the last TRB in a TD), and the caller will not
enqueue more TDs, then we defer giving back the link TRB.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The NEC xHCI host controller firmware version can be found by putting a
vendor-specific command on the command ring and extracting the BCD
encoded-version out of the vendor-specific event TRB.
The firmware version debug line in dmesg will look like:
xhci_hcd 0000:05:00.0: NEC firmware version 30.21
(NEC merged with Renesas Technologies and became Renesas Electronics on
April 1, 2010. I have their OK to merge this vendor-specific code.)
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: Satoshi Otani <satoshi.otani.xm@renesas.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Change transfer ring behavior to not follow/activate link TRBs
until active TRBs are queued after it. This change affects
the behavior when a TD ends just before a link TRB.
Signed-off-by: John Youn <johnyoun@synopsys.com>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
After using state stored in xhci_virt_ep to clean up a stalled endpoint,
be sure to set the stalled stream ID back to 0.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Change the type of the URB's 'sg' pointer from a usb_sg_request to
a scatterlist. This allows drivers to submit scatter-gather lists
without using the usb_sg_wait() interface. It has the added benefit
of removing the typecasts that were added as part of patch as1368 (and
slightly decreasing the number of pointer dereferences).
Signed-off-by: Matthew Wilcox <willy@linux.intel.com>
Reviewed-by: Alan Stern <stern@rowland.harvard.edu>
Tested-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Fix usb sparse warnings:
drivers/usb/host/isp1362-hcd.c:2220:50: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-mem.c:43:24: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-mem.c:49:24: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-mem.c:161:24: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-mem.c:198:16: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-mem.c:319:31: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-mem.c:1231:33: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-pci.c:177:23: warning: non-ANSI function declaration of function 'xhci_register_pci'
drivers/usb/host/xhci-pci.c:182:26: warning: non-ANSI function declaration of function 'xhci_unregister_pci'
drivers/usb/host/xhci-ring.c:342:32: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-ring.c:525:34: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-ring.c:1009:32: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-ring.c:1031:32: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-ring.c:1041:16: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-ring.c:1096:30: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-ring.c:1100:27: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-mem.c:224:27: warning: symbol 'xhci_alloc_container_ctx' was not declared. Should it be static?
drivers/usb/host/xhci-mem.c:242:6: warning: symbol 'xhci_free_container_ctx' was not declared. Should it be static?
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Cc: Lothar Wassmann <LW@KARO-electronics.de>
Signed-off By: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Much of the xHCI driver code assumes that endpoints only have one ring.
Now an endpoint can have one ring per enabled stream ID, so correct that
assumption. Use functions that translate the stream_id field in the URB
or the DMA address of a TRB into the correct stream ring.
Correct the polling loop to print out all enabled stream rings. Make the
URB cancellation routine find the correct stream ring if the URB has
stream_id set. Make sure the URB enqueueing routine does the same. Also
correct the code that handles stalled/halted endpoints.
Check that commands and registers that can take stream IDs handle them
properly. That includes ringing an endpoint doorbell, resetting a
stalled/halted endpoint, and setting a transfer ring dequeue pointer
(since that command can set the dequeue pointer in a stream context or an
endpoint context).
Correct the transfer event handler to translate a TRB DMA address into the
stream ring it was enqueued to. Make the code to allocate and prepare TD
structures adds the TD to the right td_list for the stream ring. Make
sure the code to give the first TRB in a TD to the hardware manipulates
the correct stream ring.
When an endpoint stalls, store the stream ID of the stream ring that
stalled in the xhci_virt_ep structure. Use that instead of the stream ID
in the URB, since an URB may be re-used after it is given back after a
non-control endpoint stall.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Add support for allocating streams for USB 3.0 bulk endpoints. See
Documentation/usb/bulk-streams.txt for more information about how and why
you would use streams.
When an endpoint has streams enabled, instead of having one ring where all
transfers are enqueued to the hardware, it has several rings. The ring
dequeue pointer in the endpoint context is changed to point to a "Stream
Context Array". This is basically an array of pointers to transfer rings,
one for each stream ID that the driver wants to use.
The Stream Context Array size must be a power of two, and host controllers
can place a limit on the size of the array (4 to 2^16 entries). These
two facts make calculating the size of the Stream Context Array and the
number of entries actually used by the driver a bit tricky.
Besides the Stream Context Array and rings for all the stream IDs, we need
one more data structure. The xHCI hardware will not tell us which stream
ID a transfer event was for, but it will give us the slot ID, endpoint
index, and physical address for the TRB that caused the event. For every
endpoint on a device, add a radix tree to map physical TRB addresses to
virtual segments within a stream ring.
Keep track of whether an endpoint is transitioning to using streams, and
don't enqueue any URBs while that's taking place. Refuse to transition an
endpoint to streams if there are already URBs enqueued for that endpoint.
We need to make sure that freeing streams does not fail, since a driver's
disconnect() function may attempt to do this, and it cannot fail.
Pre-allocate the command structure used to issue the Configure Endpoint
command, and reserve space on the command ring for each stream endpoint.
This may be a bit overkill, but it is permissible for the driver to
allocate all streams in one call and free them in multiple calls. (It is
not advised, however, since it is a waste of resources and time.)
Even with the memory and ring room pre-allocated, freeing streams can
still fail because the xHC rejects the configure endpoint command. It is
valid (by the xHCI 0.96 spec) to return a "Bandwidth Error" or a "Resource
Error" for a configure endpoint command. We should never see a Bandwidth
Error, since bulk endpoints do not effect the reserved bandwidth. The
host controller can still return a Resource Error, but it's improbable
since the xHC would be going from a more resource-intensive configuration
(streams) to a less resource-intensive configuration (no streams).
If the xHC returns a Resource Error, the endpoint will be stuck with
streams and will be unusable for drivers. It's an unavoidable consequence
of broken host controller hardware.
Includes bug fixes from the original patch, contributed by
John Youn <John.Youn@synopsys.com> and Andy Green <AGreen@PLXTech.com>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This patch (as1368) fixes a rather obscure bug in usbmon: When tracing
URBs sent by the scatter-gather library, it accesses the data buffers
while they are still mapped for DMA.
The solution is to move the mapping and unmapping out of the s-g
library and into the usual place in hcd.c. This requires the addition
of new URB flag bits to describe the kind of mapping needed, since we
have to call dma_map_sg() if the HCD supports native scatter-gather
operation and dma_map_page() if it doesn't. The nice thing about
having the new flags is that they simplify the testing for unmapping.
The patch removes the only caller of usb_buffer_[un]map_sg(), so those
functions are #if'ed out. A later patch will remove them entirely.
As a result of this change, urb->sg will be set in situations where
it wasn't set previously. Hence the xhci and whci drivers are
adjusted to test urb->num_sgs instead, which retains its original
meaning and is nonzero only when the HCD has to handle a scatterlist.
Finally, even when a submission error occurs we don't want to hand
URBs to usbmon before they are unmapped. The submission path is
rearranged so that map_urb_for_dma() is called only for non-root-hub
URBs and unmap_urb_for_dma() is called immediately after a submission
error. This simplifies the error handling.
Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
CC: <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The length of the scatter gather list a driver can enqueue is limited by
the bus' sg_tablesize to 62 entries. Each entry will be described by at
least one transfer request block (TRB). If the entry's buffer crosses a
64KB boundary, then that entry will have to be described by two or more
TRBs. So even if the USB device driver respects sg_tablesize, the whole
scatter list may take more than 62 TRBs to describe, and won't fit on
the ring.
Don't assume that an empty ring means there is enough room on the
transfer ring. The old code would unconditionally queue this too-large
transfer, and over write the beginning of the transfer. This would mean
the cycle bit was unchanged in those overwritten transfers, causing the
hardware to think it didn't own the TRBs, and the host would seem to
hang.
Now drivers may see submit_urb() fail with -ENOMEM if the transfers are
too big to fit on the ring.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When the USB core installs a new interface, it unconditionally clears the
halts on all the endpoints on the new interface. Usually the xHCI host
needs to know when an endpoint is reset, so it can change its internal
endpoint state. In this case, it doesn't care, because the endpoints were
never halted in the first place.
To avoid issuing a redundant Reset Endpoint command, the xHCI driver looks
at xhci_virt_ep->stopped_td to determine if the endpoint was actually
halted. However, the functions that handle the stall never set that
variable to NULL after it dealt with the stall. So if an endpoint stalled
and a Reset Endpoint command completed, and then the class driver tried to
install a new alternate setting, the xHCI driver would access the old
xhci_virt_ep->stopped_td pointer. A similar problem occurs if the
endpoint has been stopped to cancel a transfer.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.
percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.
http://userweb.kernel.org/~tj/misc/slabh-sweep.py
The script does the followings.
* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.
* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.
* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.
The conversion was done in the following steps.
1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.
2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.
3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.
4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.
5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.
6. percpu.h was updated not to include slab.h.
7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).
* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig
8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.
Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
When a USB device is reset, the xHCI hardware must know, in order to match
the device state and disable all endpoints except control endpoint 0.
Issue a Reset Device command after a USB device is successfully reset.
Wait on the command to finish, and then cache or free the disabled
endpoint rings.
There are four different USB device states that the xHCI hardware tracks:
- disabled/enabled - device connection has just been detected,
- default - the device has been reset and has an address of 0,
- addressed - the device has a non-zero address but no configuration has
been set,
- configured - a set configuration succeeded.
The USB core may issue a port reset when a device is in any state, but the
Reset Device command will fail for a 0.96 xHC if the device is not in the
addressed or configured state. Don't consider this failure as an error,
but don't free any endpoint rings if this command fails.
A storage driver may request that the USB device be reset during error
handling, so use GPF_NOIO instead of GPF_KERNEL while allocating memory
for the Reset Device command.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
All commands that can be issued to the xHCI hardware can come back with
vendor-specific "informational" completion codes. These are to be treated
like a successful completion code. Refactor out the code to test for the
range of these codes and print debugging messages.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The xHCI driver issues a Configure Endpoint command for two reasons:
- a new configuration or alternate interface setting is selected
- a quirky Fresco Logic prototype requires the command after a Reset
Endpoint command.
The xHCI driver only waits on the command in the first case.
When a configure endpoint command completes, the driver needs to know why
the command was generated. When the driver only supported selecting an
initial configuration, the check was simple. Unfortunately that check
doesn't work now that the driver supports alternate interfaces. If an
endpoint must be dropped (because it's not in the new alternate setting)
and no new endpoints are added, the math involving
xhci_last_valid_endpoint() will assign -1 to an unsigned integer and cause
an out-of-bounds array access.
Move the check for the quirky hardware sooner and avoid the bad array
access.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The xHCI 0.95 and 0.96 specification defines several transfer buffer
request completion codes that indicate a USB transaction error occurred.
When a stall, babble, transaction, or split transaction error completion code
is set, the xHCI has halted that endpoint ring. Software must issue a
Reset Endpoint command and a Set Transfer Ring Dequeue Pointer command
to clean up the halted ring.
The USB device driver is supposed to call into usb_reset_endpoint() when
an endpoint stalls. That calls into the xHCI driver to issue the proper
commands. However, drivers don't call that function for the other
errors that cause the xHC to halt the endpoint ring. If a babble,
transaction, or split transaction error occurs, check if the endpoint
context reports a halted condition, and clean up the endpoint ring if it
does.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
An xHCI host controller manufacturer can choose to implement several
vendor-specific informational completion codes. These are all to be
treated like a successful transfer completion.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When the xHCI hardware says a transfer completed with a split
transaction error, set the URB status to -EPROTO.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The transfer descriptor (TD) is a series of transfer request buffers
(TRBs) that describe the buffer pointer, length, and other
characteristics. The xHCI controllers want to know an estimate of how
long the TD is, for caching reasons. In each TRB, there is a "TD size"
field that provides a rough estimate of the remaining buffers to be
transmitted, including the buffer pointed to by that TRB.
The TD size is 5 bits long, and contains the remaining size in bytes,
right shifted by 10 bits. So a remaining TD size less than 1024 would get
a zero in the TD size field, and a remaining size greater than 32767 would
get 31 in the field.
This patches fixes a bug in the TD_REMAINDER macro that is triggered when
the URB has a scatter gather list with a size bigger than 32767 bytes.
Not all host controllers pay attention to the TD size field, so the bug
will not appear on all USB 3.0 hosts.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
It's not surprising that the transfer request buffer (TRB) physical to
virtual address translation function has bugs in it, since I wrote most of
it at 4am last October. Add a test suite to check the TRB math. This
runs at memory initialization time, and causes the driver to fail to load
if the TRB math fails.
Please excuse the excessively long lines in the test vectors; they can't
really be made shorter and still be readable.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
CONFIG_USB_HCD_STAT was used in an abandoned patch to track host
controller throughput statistics. Since CONFIG_USB_HCD_STAT will never be
defined, remove code that can never run.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
In order to giveback a canceled URB, we must ensure that the xHCI
hardware will not access the buffer in an URB. We can't modify the
buffer pointers on endpoint rings without issuing and waiting for a stop
endpoint command. Since URBs can be canceled in interrupt context, we
can't wait on that command. The old code trusted that the host
controller would respond to the command, and would giveback the URBs in
the event handler. If the hardware never responds to the stop endpoint
command, the URBs will never be completed, and we might hang the USB
subsystem.
Implement a watchdog timer that is spawned whenever a stop endpoint
command is queued. If a stop endpoint command event is found on the
event ring during an interrupt, we need to stop the watchdog timer with
del_timer(). Since del_timer() can fail if the timer is running and
waiting on the xHCI lock, we need a way to signal to the timer that
everything is fine and it should exit. If we simply clear
EP_HALT_PENDING, a new stop endpoint command could sneak in and set it
before the watchdog timer can grab the lock.
Instead we use a combination of the EP_HALT_PENDING flag and a counter
for the number of pending stop endpoint commands
(xhci_virt_ep->stop_cmds_pending). If we need to cancel the watchdog
timer and del_timer() succeeds, we decrement the number of pending stop
endpoint commands. If del_timer() fails, we leave the number of pending
stop endpoint commands alone. In either case, we clear the
EP_HALT_PENDING flag.
The timer will decrement the number of pending stop endpoint commands
once it obtains the lock. If the timer is the tail end of the last stop
endpoint command (xhci_virt_ep->stop_cmds_pending == 0), and the
endpoint's command is still pending (EP_HALT_PENDING is set), we assume
the host is dying. The watchdog timer will set XHCI_STATE_DYING, try to
halt the xHCI host, and give back all pending URBs.
Various other places in the driver need to check whether the xHCI host
is dying. If the interrupt handler ever notices, it should immediately
stop processing events. The URB enqueue function should also return
-ESHUTDOWN. The URB dequeue function should simply return the value
of usb_hcd_check_unlink_urb() and the watchdog timer will take care of
giving the URB back. When a device is disconnected, the xHCI hardware
structures should be freed without issuing a disable slot command (since
the hardware probably won't respond to it anyway). The debugging
polling loop should stop polling if the host is dying.
When a device is disconnected, any pending watchdog timers are killed
with del_timer_sync(). It must be synchronous so that the watchdog
timer doesn't attempt to access the freed endpoint structures.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
In the old code, there was a race condition between the stop endpoint
command and the URB submission process. When the stop endpoint command is
handled by the event handler, the endpoint ring is assumed to be stopped.
When a stop endpoint command is queued, URB submissions are to not ring
the doorbell. The old code would check the number of pending URBs to be
canceled, and would not ring the doorbell if it was non-zero.
However, the following race condition could occur with the old code:
1. Cancel an URB, add it to the list of URBs to be canceled, queue the stop
endpoint command, and increment ep->cancels_pending to 1.
2. The URB finishes on the HW, and an event is enqueued to the event ring
(at the same time as 1).
3. The stop endpoint command finishes, and the endpoint is halted. An
event is queued to the event ring.
4. The event handler sees the finished URB, notices it was to be
canceled, decrements ep->cancels_pending to 0, and removes it from the to
be canceled list.
5. The event handler drops the lock and gives back the URB. The
completion handler requeues the URB (or a different driver enqueues a new
URB). This causes the endpoint's doorbell to be rung, since
ep->cancels_pending == 0. The endpoint is now running.
6. A second URB is canceled, and it's added to the canceled list.
Since ep->cancels_pending == 0, a new stop endpoint command is queued, and
ep->cancels_pending is incremented to 1.
7. The event handler then sees the completed stop endpoint command. The
handler assumes the endpoint is stopped, but it isn't. It attempts to
move the dequeue pointer or change TDs to cancel the second URB, while the
hardware is actively accessing the endpoint ring.
To eliminate this race condition, a new endpoint state bit is introduced,
EP_HALT_PENDING. When this bit is set, a stop endpoint command has been
queued, and the command handler has not begun to process the URB
cancellation list yet. The endpoint doorbell should not be rung when this
is set. Set this when a stop endpoint command is queued, clear it when
the handler for that command runs, and check if it's set before ringing a
doorbell. ep->cancels_pending is eliminated, because it is no longer
used.
Make sure to ring the doorbell for an endpoint when the stop endpoint
command handler runs, even if the canceled URB list is empty. All
canceled URBs could have completed and new URBs could have been enqueued
without the doorbell being rung before the command was handled.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The trb_in_td() function in the xHCI driver is supposed to translate a
physical transfer buffer request (TRB) into a virtual pointer to the ring
segment that TRB is in.
Unfortunately, a mistake in this function may cause endless loops as the
driver searches through the linked list of ring segments over and over
again. Fix a couple bugs that may lead to loops or bad output:
1. Bail out if we get a NULL pointer when translating the segment's
private structure and the starting DMA address of the segment chunk. If
this happens, we've been handed a starting TRB pointer from a different
ring.
2. Make sure the function works when there's multiple segments in the
ring. In the while loop to search through the ring segments, use the
current segment variable (cur_seg), rather than the starting segment
variable (start_seg) that is passed in.
3. Stop searching the ring if we've run through all the segments in the
ring.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
For a USB hub to work under an xHCI host controller, the xHC's internal
scheduler must be made aware of the hub's characteristics. Add an xHCI
hook that the USB core will call after it fetches the hub descriptor.
This hook will add hub information to the slot context for that device,
including whether it has multiple TTs or a single TT, the number of ports
on the hub, and TT think time.
Setting up the slot context for the device is different for 0.95 and 0.96
xHCI host controllers.
Some of the slot context reserved fields in the 0.95 specification were
changed into hub fields in the 0.96 specification. Don't set the TT think
time or number of ports for a hub if we're dealing with a 0.95-compliant
xHCI host controller.
The 0.95 xHCI specification says that to modify the hub flag, we need to
issue an evaluate context command. The 0.96 specification says that flag
can be set with a configure endpoint command. Issue the correct command
based on the version reported by the hardware.
This patch does not add support for multi-TT hubs. Multi-TT hubs expose
a single TT on alt setting 0, and multi-TT on alt setting 1. The xHCI
driver can't handle setting alternate interfaces yet.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
In the xHCI driver, configure endpoint commands that are submitted to the
hardware may involve one of two data structures. If the configure
endpoint command is setting up a new configuration or modifying max packet
sizes, the data structures and completions are statically allocated in the
xhci_virt_device structure. If the command is being used to set up
streams or add hub information, then the data structures are dynamically
allocated, and placed on a device command waiting list.
Break out the code to check whether a completed command is in the device
command waiting list. Fix a subtle bug in the old code: continue
processing the command if the command isn't in the wait list. In the old
code, if there was a command in the wait list, but it didn't match the
completed command, the completed command event would be dropped.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Some commands to the xHCI hardware cannot be allowed to fail due to out of
memory issues or the command ring being full.
Add a way to reserve a TRB on the command ring, and make all command
queueing functions indicate whether they are using a reserved TRB.
Add a way to pre-allocate all the memory a command might need. A command
needs an input context, a variable to store the status, and (optionally) a
completion for the caller to wait on. Change all code that assumes the
input device context, status, and completion for a command is stored in
the xhci virtual USB device structure (xhci_virt_device).
Store pending completions in a FIFO in xhci_virt_device. Make the event
handler for a configure endpoint command check to see whether a pending
command in the list has completed. We need to use separate input device
contexts for some configure endpoint commands, since multiple drivers can
submit requests at the same time that require a configure endpoint
command.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The xhci_ring structure contained information that is really related to an
endpoint, not a ring. This will cause problems later when endpoint
streams are supported and there are multiple rings per endpoint.
Move the endpoint state and cancellation information into a new virtual
endpoint structure, xhci_virt_ep. The list of TRBs to be cancelled should
be per endpoint, not per ring, for easy access. There can be only one TRB
that the endpoint stopped on after a stop endpoint command (even with
streams enabled); move the stopped TRB information into the new virtual
endpoint structure. Also move the 31 endpoint rings and temporary ring
storage from the virtual device structure (xhci_virt_device) into the
virtual endpoint structure (xhci_virt_ep).
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Andiry Xu