openkylin
/
qemu
mirror of https://gitee.com/openkylin/qemu.git
9
0
Fork 0
Code Issues Projects Releases Wiki Activity
qemu/contrib/rdmacm-mux/rdmacm-mux.h

62 lines
1.4 KiB
C
Raw Normal View History

contrib/rdmacm-mux: Add implementation of RDMA User MAD multiplexer RDMA MAD kernel module (ibcm) disallow more than one MAD-agent for a given MAD class. This does not go hand-by-hand with qemu pvrdma device's requirements where each VM is MAD agent. Fix it by adding implementation of RDMA MAD multiplexer service which on one hand register as a sole MAD agent with the kernel module and on the other hand gives service to more than one VM. Design Overview: Reviewed-by: Shamir Rabinovitch <shamir.rabinovitch@oracle.com> ---------------- A server process is registered to UMAD framework (for this to work the rdma_cm kernel module needs to be unloaded) and creates a unix socket to listen to incoming request from clients. A client process (such as QEMU) connects to this unix socket and registers with its own GID. TX: ---- When client needs to send rdma_cm MAD message it construct it the same way as without this multiplexer, i.e. creates a umad packet but this time it writes its content to the socket instead of calling umad_send(). The server, upon receiving such a message fetch local_comm_id from it so a context for this session can be maintain and relay the message to UMAD layer by calling umad_send(). RX: ---- The server creates a worker thread to process incoming rdma_cm MAD messages. When an incoming message arrived (umad_recv()) the server, depending on the message type (attr_id) looks for target client by either searching in gid->fd table or in local_comm_id->fd table. With the extracted fd the server relays to incoming message to the client. Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com> Reviewed-by: Shamir Rabinovitch <shamir.rabinovitch@oracle.com> Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
2018-12-21 22:40:15 +08:00
/*
* QEMU paravirtual RDMA - rdmacm-mux declarations
*
* Copyright (C) 2018 Oracle
* Copyright (C) 2018 Red Hat Inc
*
* Authors:
* Yuval Shaia <yuval.shaia@oracle.com>
* Marcel Apfelbaum <marcel@redhat.com>
*
* This work is licensed under the terms of the GNU GPL, version 2 or later.
* See the COPYING file in the top-level directory.
*
*/
#ifndef RDMACM_MUX_H
#define RDMACM_MUX_H
#include "linux/if.h"
Use #include "..." for our own headers, <...> for others Also delete a few redundant #include. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190315145123.28030-2-armbru@redhat.com>
2019-03-15 22:51:16 +08:00
#include <infiniband/verbs.h>
#include <infiniband/umad.h>
#include <rdma/rdma_user_cm.h>
contrib/rdmacm-mux: Add implementation of RDMA User MAD multiplexer RDMA MAD kernel module (ibcm) disallow more than one MAD-agent for a given MAD class. This does not go hand-by-hand with qemu pvrdma device's requirements where each VM is MAD agent. Fix it by adding implementation of RDMA MAD multiplexer service which on one hand register as a sole MAD agent with the kernel module and on the other hand gives service to more than one VM. Design Overview: Reviewed-by: Shamir Rabinovitch <shamir.rabinovitch@oracle.com> ---------------- A server process is registered to UMAD framework (for this to work the rdma_cm kernel module needs to be unloaded) and creates a unix socket to listen to incoming request from clients. A client process (such as QEMU) connects to this unix socket and registers with its own GID. TX: ---- When client needs to send rdma_cm MAD message it construct it the same way as without this multiplexer, i.e. creates a umad packet but this time it writes its content to the socket instead of calling umad_send(). The server, upon receiving such a message fetch local_comm_id from it so a context for this session can be maintain and relay the message to UMAD layer by calling umad_send(). RX: ---- The server creates a worker thread to process incoming rdma_cm MAD messages. When an incoming message arrived (umad_recv()) the server, depending on the message type (attr_id) looks for target client by either searching in gid->fd table or in local_comm_id->fd table. With the extracted fd the server relays to incoming message to the client. Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com> Reviewed-by: Shamir Rabinovitch <shamir.rabinovitch@oracle.com> Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
2018-12-21 22:40:15 +08:00
typedef enum RdmaCmMuxMsgType {
RDMACM_MUX_MSG_TYPE_REQ = 0,
RDMACM_MUX_MSG_TYPE_RESP = 1,
} RdmaCmMuxMsgType;
typedef enum RdmaCmMuxOpCode {
RDMACM_MUX_OP_CODE_REG = 0,
RDMACM_MUX_OP_CODE_UNREG = 1,
RDMACM_MUX_OP_CODE_MAD = 2,
} RdmaCmMuxOpCode;
typedef enum RdmaCmMuxErrCode {
RDMACM_MUX_ERR_CODE_OK = 0,
RDMACM_MUX_ERR_CODE_EINVAL = 1,
RDMACM_MUX_ERR_CODE_EEXIST = 2,
RDMACM_MUX_ERR_CODE_EACCES = 3,
RDMACM_MUX_ERR_CODE_ENOTFOUND = 4,
} RdmaCmMuxErrCode;
typedef struct RdmaCmMuxHdr {
RdmaCmMuxMsgType msg_type;
RdmaCmMuxOpCode op_code;
union ibv_gid sgid;
RdmaCmMuxErrCode err_code;
} RdmaCmUHdr;
typedef struct RdmaCmUMad {
struct ib_user_mad hdr;
char mad[RDMA_MAX_PRIVATE_DATA];
} RdmaCmUMad;
typedef struct RdmaCmMuxMsg {
RdmaCmUHdr hdr;
int umad_len;
RdmaCmUMad umad;
} RdmaCmMuxMsg;
#endif