We should be returning -ENOMEM here instead of success.
Fixes: ba612aa8b4 ('misc: mic: SCIF memory registration and unregistration')
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Reviewed-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
The caller expects that we take this lock again before returning
otherwise it you get double unlocks and races.
Fixes: ba612aa8b4 ('misc: mic: SCIF memory registration and unregistration')
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Reviewed-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
In scif_node_connect() we were returning if the initialization of p2p_ji
fails. But at that time p2p_ij has already been initialized and
resources allocated for it. And since p2p_ij is not added to the list
till now so we will have a leak.
Lets deinitialize and release the resources connected to p2p_ij.
Signed-off-by: Sudip Mukherjee <sudip@vectorindia.org>
Reviewed-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
This patch adds the SCIF kernel node QP control messages required to
enable SCIF RMAs. Examples of such node QP control messages include
registration, unregistration, remote memory allocation requests,
remote memory unmap and SCIF remote fence requests.
The patch also updates the SCIF driver with minor changes required to
enable SCIF RMAs by adding the new files to the build, initializing
RMA specific information during SCIF endpoint creation, reserving SCIF
DMA channels, initializing SCIF RMA specific global data structures,
adding the IOCTL hooks required for SCIF RMAs and updating RMA
specific debugfs hooks.
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Reviewed-by: Nikhil Rao <nikhil.rao@intel.com>
Signed-off-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
This patch implements the fence APIs required to synchronize
DMAs. SCIF provides an interface to return a "mark" for all DMAs
programmed at the instant the API was called. Users can then "wait" on
the mark provided previously by blocking inside the kernel. Upon
receipt of a DMA completion interrupt the waiting thread is woken
up. There is also an interface to signal DMA completion by polling for
a location to be updated via a "signal" cookie to avoid the interrupt
overhead in the mark/wait interface. SCIF allows programming fences on
both the local and the remote node for both the mark/wait or the fence
signal APIs.
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Reviewed-by: Nikhil Rao <nikhil.rao@intel.com>
Signed-off-by: Jacek Lawrynowicz <jacek.lawrynowicz@intel.com>
Signed-off-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
SCIF allows users to read from or write to registered remote memory
via CPU copies or DMA. The API verifies that both local and remote
windows are valid before initiating the CPU or DMA transfers. SCIF has
optimized algorithms for handling byte aligned as well as cache line
aligned DMA engines. A registration cache is maintained to avoid the
overhead of pinning pages repeatedly if buffers are reused. The
registration cache is invalidated upon receipt of MMU notifier
callbacks. SCIF windows are destroyed and the pages are unpinned only
once all prior DMAs initiated using that window are drained. Users can
request synchronous DMA operations as well as tail byte ordering if
required. CPU copies are always performed synchronously.
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Reviewed-by: Nikhil Rao <nikhil.rao@intel.com>
Signed-off-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
This patch implements the SCIF mmap/munmap interface. A similar
capability is provided to kernel clients via the
scif_get_pages()/scif_put_pages() APIs. The SCIF mmap interface
queries to check if a window is valid and then remaps the local
virtual address to the remote physical pages. These mappings are
subsequently destroyed upon receipt of the VMA close operation or
scif_get_pages(). This functionality allows SCIF users to directly
access remote memory without any driver interaction once the mappings
are created thereby providing bare-metal PCIe latency. These mappings
are zapped to avoid RMA accesses from user space, if a Coprocessor is
reset.
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Reviewed-by: Nikhil Rao <nikhil.rao@intel.com>
Signed-off-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
This patch adds the implementation for operations performed on the
list of SCIF windows. Examples of such operations includes adding the
windows to the list of registered (or cached) windows, querying the
list of self or remote windows and unregistering windows. The query
operation is used by SCIF APIs which initiate DMAs, CPU copies or
fences to ensure that a window remains valid during a transfer.
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Nikhil Rao <nikhil.rao@intel.com>
Signed-off-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
This patch implements the SCIF APIs required to pin and unpin
pages. SCIF registration locks down the pages. It then sends a remote
window allocation request to the peer. Once the peer has allocated
memory, the local SCIF endpoint copies the pinned page information to
the peer and notifies the peer once the copy has complete. The peer
upon receipt of the registration notification adds the new remote
window to its list. At this point the window page information is
available on both self and remote nodes so that they can start
performing SCIF DMAs, CPU copies and fences. The unregistration API
tears down the registration at both self and remote nodes.
Reviewed-by: Nikhil Rao <nikhil.rao@intel.com>
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
This patch adds the internal data structures required to perform SCIF
RMAs. The data structures required to maintain per SCIF endpoint, RMA
information are contained in scif_endpt_rma_info. scif_pinned_pages
describes a set of SCIF pinned pages maintained locally. The
scif_window is a data structure which contains all the fields required
to describe a SCIF registered window on self and remote nodes. It
contains an offset which is used as a key to perform SCIF DMAs and CPU
copies between self and remote registered windows.
Reviewed-by: Nikhil Rao <nikhil.rao@intel.com>
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Add support for registration/de-registration of kernel mode SCIF
clients. SCIF clients are probed with new and existing SCIF peer
devices. Similarly the client remove method is called when SCIF
peer devices are removed.
Changes to SCIF peer device framework necessitated by supporting
kernel mode SCIF clients are also included in this patch.
Reviewed-by: Nikhil Rao <nikhil.rao@intel.com>
Reviewed-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
SCIF poll allows both user and kernel mode clients to wait on
events on a SCIF endpoint. These events include availability of
space or data in the SCIF ring buffer, availability of connection
requests on a listening endpoint and completion of connections
when using async connects.
Reviewed-by: Nikhil Rao <nikhil.rao@intel.com>
Reviewed-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
v4.2-rc1 enabled huge page support for ioremap(..).
Calling vmalloc_to_page after v4.2-rc1 results in the
crash shown below on the host upon booting X100 coprocessors:
BUG: unable to handle kernel paging request at ffffc47c00000000
IP: [<ffffffff811a2c0c>] vmalloc_to_page+0x6c/0xb0
This patch fixes this crash by obtaining the fake struct page
pointer which is required to be passed into dma_map_sg(..)
by calling pfn_to_page(..) instead of vmalloc_to_page(..).
Link: https://lkml.org/lkml/2015/7/18/110
Reviewed-by: Nikhil Rao <nikhil.rao@intel.com>
Suggested-by: Toshi Kani <toshi.kani@hp.com>
Signed-off-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Delete unnecessary prints resulting in an "spdev could be null"
warning from a static checker in scif_peer_remove(..).
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
This issue was reported @ https://lkml.org/lkml/2015/6/9/731
Reported-by: Jim Davis <jim.epost@gmail.com>
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
SCIF messaging APIs which allow sending messages between the SCIF
endpoints via a byte stream based ring buffer which has been
optimized to avoid reads across PCIe. The SCIF messaging APIs
are typically used for short < 1024 byte messages for best
performance while the RDMA APIs which will be submitted in a future
patch series is recommended for larger transfers. The node
enumeration API enables a user to query for the number of nodes
online in the SCIF network and their node ids.
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Nikhil Rao <nikhil.rao@intel.com>
Signed-off-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
SCIF connection APIs which establish a SCIF connection between
a pair of SCIF endpoints. A SCIF connection consists of a
dedicated queue-pair between the endpoints. Client messages are
sent over the queue-pair whereas the signaling associated with the
message is multiplexed over the node queue-pair. Similarly other
control messages such as exposing registered memory are also sent
over the node queue-pair. The SCIF endpoints must be in connected
state to exchange messages, register memory, map remote memory and
trigger DMA transfers. SCIF connections can be set up
asynchronously or synchronously.
Thanks to Johnnie S Peters for authoring parts of this patch during
early bring up of the SCIF driver.
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Nikhil Rao <nikhil.rao@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
SCIF character device file operations and kernel APIs for opening and
closing a user and kernel mode SCIF endpoint. This patch also enables
binding to a SCIF port and listening for incoming SCIF connections.
Reviewed-by: Nikhil Rao <nikhil.rao@intel.com>
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
SCIF node queue pair setup creates the SCIF driver kernel
mode private node queue pairs between all the nodes to enable
internal control message communication once SCIF gets probed
by the SCIF hardware bus. Peer to peer communication between
MIC Coprocessor nodes is supported.
Reviewed-by: Nikhil Rao <nikhil.rao@intel.com>
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
SCIF module initialization, DMA mapping, ioremap wrapper APIs
and debugfs hooks. SCIF gets probed by the SCIF hardware bus
if SCIF devices were registered by base drivers. A MISC device
is registered to provide the SCIF character device interface.
Reviewed-by: Nikhil Rao <nikhil.rao@intel.com>
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
The SCIF peer bus is used to register and unregister SCIF peer devices
internally by the SCIF driver to signify the addition and removal of
peer nodes respectively from the SCIF network. This simplifies remote node
handling within SCIF and will also be used to support device probe/remove
for SCIF client drivers (e.g. netdev over SCIF)
Reviewed-by: Nikhil Rao <nikhil.rao@intel.com>
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
SCIF ring buffer is a single producer, single consumer byte stream
ring buffer optimized for avoiding reads across the PCIe bus while
adding the required barriers and hardware workarounds for the MIC
Coprocessor. The ring buffer is used to implement a receive queue for
SCIF driver messaging between two nodes and for byte stream messaging
between SCIF endpoints. The existing in-kernel ring buffer was not
reused since it has not been designed for our use across the PCIe bus
where each node runs an independent OS. Each SCIF node has a receive
queue for every other SCIF node, and each connected endpoint has a
receive queue for messages from its peer. This pair of receive
queues is referred to as a SCIF queue pair.
Reviewed-by: Nikhil Rao <nikhil.rao@intel.com>
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Dan Carpenter