Device information
Here are few options to get your InfiniBand HCA device information
ibv_devinfo
Print information about RDMA devices available for use from userspace.
$ ibv_devinfo
hca_id: mlx5_0
transport: InfiniBand (0)
fw_ver: 20.26.4012
node_guid: 9803:9b03:0067:a614
sys_image_guid: 9803:9b03:0067:a614
vendor_id: 0x02c9
vendor_part_id: 4123
hw_ver: 0x0
board_id: MT_0000000228
phys_port_cnt: 1
Device ports:
port: 1
state: PORT_ACTIVE (4)
max_mtu: 4096 (5)
active_mtu: 4096 (5)
sm_lid: 1
port_lid: 38
port_lmc: 0x00
link_layer: InfiniBand
ibv_devinfo -v
Print verbos information about RDMA devices available for use from userspace.
$ ibv_devinfo -v
hca_id: mlx5_0
transport: InfiniBand (0)
fw_ver: 20.26.4012
node_guid: 9803:9b03:0067:a614
sys_image_guid: 9803:9b03:0067:a614
vendor_id: 0x02c9
vendor_part_id: 4123
hw_ver: 0x0
board_id: MT_0000000228
phys_port_cnt: 1
max_mr_size: 0xffffffffffffffff
page_size_cap: 0xfffffffffffff000
max_qp: 262144
max_qp_wr: 32768
device_cap_flags: 0xe17e1c36
BAD_PKEY_CNTR
BAD_QKEY_CNTR
AUTO_PATH_MIG
CHANGE_PHY_PORT
PORT_ACTIVE_EVENT
SYS_IMAGE_GUID
RC_RNR_NAK_GEN
XRC
Unknown flags: 0xe16e0000
device_cap_exp_flags: 0x5648F8F100000000
EXP_DC_TRANSPORT
EXP_CROSS_CHANNEL
EXP_MR_ALLOCATE
EXT_ATOMICS
EXT_SEND NOP
EXP_UMR
EXP_ODP
EXP_RX_CSUM_TCP_UDP_PKT
EXP_RX_CSUM_IP_PKT
EXP_DC_INFO
EXP_MASKED_ATOMICS
EXP_RX_TCP_UDP_PKT_TYPE
EXP_PHYSICAL_RANGE_MR
EXP_UMR_FIXED_SIZE
EXP_PACKET_BASED_CREDIT_MODE
Unknown flags: 0x200000000000
max_sge: 30
max_sge_rd: 30
max_cq: 16777216
max_cqe: 4194303
max_mr: 16777216
max_pd: 16777216
max_qp_rd_atom: 16
max_ee_rd_atom: 0
max_res_rd_atom: 4194304
max_qp_init_rd_atom: 16
max_ee_init_rd_atom: 0
atomic_cap: ATOMIC_HCA (1)
log atomic arg sizes (mask) 0x8
masked_log_atomic_arg_sizes (mask) 0x3c
masked_log_atomic_arg_sizes_network_endianness (mask) 0x34
max fetch and add bit boundary 64
log max atomic inline 5
max_ee: 0
max_rdd: 0
max_mw: 16777216
max_raw_ipv6_qp: 0
max_raw_ethy_qp: 0
max_mcast_grp: 2097152
max_mcast_qp_attach: 240
max_total_mcast_qp_attach: 503316480
max_ah: 2147483647
max_fmr: 0
max_srq: 8388608
max_srq_wr: 32767
max_srq_sge: 31
max_pkeys: 128
local_ca_ack_delay: 16
hca_core_clock: 156250
max_klm_list_size: 65536
max_send_wqe_inline_klms: 20
max_umr_recursion_depth: 4
max_umr_stride_dimension: 1
general_odp_caps:
ODP_SUPPORT
ODP_SUPPORT_IMPLICIT
max_size: 0xFFFFFFFFFFFFFFFF
rc_odp_caps:
SUPPORT_SEND
SUPPORT_RECV
SUPPORT_WRITE
SUPPORT_READ
SUPPORT_SRQ_RECV
uc_odp_caps:
NO SUPPORT
ud_odp_caps:
SUPPORT_SEND
dc_odp_caps:
SUPPORT_SEND
SUPPORT_WRITE
SUPPORT_READ
xrc_odp_caps:
NO SUPPORT
raw_eth_odp_caps:
NO SUPPORT
max_dct: 262144
max_device_ctx: 1020
Multi-Packet RQ supported
Supported for objects type:
IBV_EXP_MP_RQ_SUP_TYPE_SRQ_TM
IBV_EXP_MP_RQ_SUP_TYPE_WQ_RQ
Supported payload shifts:
2 bytes
Log number of strides for single WQE: 3 - 16
Log number of bytes in single stride: 6 - 13
rx_pad_end_addr_align: 64
tso_caps:
max_tso: 0
packet_pacing_caps:
qp_rate_limit_min: 0kbps
qp_rate_limit_max: 0kbps
ooo_caps:
ooo_rc_caps = 0x1
ooo_xrc_caps = 0x1
ooo_dc_caps = 0x1
ooo_ud_caps = 0x0
SUPPORT_RC_RW_DATA_PLACEMENT
SUPPORT_XRC_RW_DATA_PLACEMENT
SUPPORT_DC_RW_DATA_PLACEMENT
sw_parsing_caps:
supported_qp:
max_rndv_hdr_size: 0x40
max_num_tags: 0x7f
max_ops: 0x8000
max_sge: 0x1
capability_flags:
IBV_EXP_TM_CAP_RC
IBV_EXP_TM_CAP_DC
tunnel_offloads_caps:
UMR fixed size:
max entity size: 2147483648
Device ports:
port: 1
state: PORT_ACTIVE (4)
max_mtu: 4096 (5)
active_mtu: 4096 (5)
sm_lid: 1
port_lid: 38
port_lmc: 0x00
link_layer: InfiniBand
max_msg_sz: 0x40000000
port_cap_flags: 0x2251e848
max_vl_num: 4 (3)
bad_pkey_cntr: 0x0
qkey_viol_cntr: 0x0
sm_sl: 0
pkey_tbl_len: 128
gid_tbl_len: 8
subnet_timeout: 18
init_type_reply: 0
active_width: 4X (2)
active_speed: 50.0 Gbps (64)
phys_state: LINK_UP (5)
GID[ 0]: fe80:0000:0000:0000:9803:9b03:0067:a614
ibstatus
ibstatus is a script which displays basic information obtained from the local IB driver. Output includes LID, SMLID, port state, link width active, and port physical state.
$ ibstatus
Infiniband device 'mlx5_0' port 1 status:
default gid: fe80:0000:0000:0000:9803:9b03:0067:a614
base lid: 0x26
sm lid: 0x1
state: 4: ACTIVE
phys state: 5: LinkUp
rate: 200 Gb/sec (4X HDR)
link_layer: InfiniBand
ibstat
ibstat is a binary which displays basic information obtained from the local IB driver. Output includes LID, SMLID, port state, link width active, and port physical state.
It is similar to the ibstatus utility but implemented as a binary rather than a script. It has options to list CAs and/or ports and displays more information than ibstatus.
ibv_devices
Get the device list on the node.