According to the Open Fabrics Alliance Documents, Open Fabrics Enterprise Distribution (OFED) ConnectX driver (mlx4) in OFED 1.4 Release Notes.
It is recommended to use the QSA Adapter (QSFP+ to SFP+ adapter) which is the world’s first solution for the QSFP to SFP+ conversion challenge for 40GB/Infiniband to 10G/1G. For more information, see Quad to Serial Small Form Factor Pluggable (QSA) Adapter to allow for the hardware
Here is the summary of the excerpts from the document.
mlx4 is the low level driver implementation for the ConnectX adapters designed by Mellanox Technologies. The ConnectX can operate as an InfiniBand adapter, as an Ethernet NIC, or as a Fibre Channel HBA. The driver in OFED 1.4 supports Infiniband and Ethernet NIC configurations. To accommodate the supported configurations, the driver is split into three modules:
Handles low-level functions like device initialization and firmware commands processing. Also controls resource allocation so that the InfiniBand and Ethernet functions can share the device without interfering with each other.
Handles InfiniBand-specific functions and plugs into the InfiniBand midlayer
A new 10G driver named mlx4_en was added to drivers/net/mlx4. It handles Ethernet specific functions and plugs into the netdev mid-layer.
Using Virtual Protocol Interconnect (VPI) to switch between Ethernet and Infiniband
- The VPI driver is a combination of the Mellanox ConnectX HCA Ethernet and Infiniband drivers. It supplies the user with the ability to run Infiniband and Ethernet protocols on the same HCA.
- Check the MLX4 Driver is loaded, ensure that the
# vim /etc/infiniband/openib.conf
# Load MLX4_EN module MLX4_EN_LOAD=yes
- If the MLX4_EN_LOAD=no, the Ethernet Driver can be loaded by running
# /sbin/modprobe mlx4_en
Port Management / Driver Switching
- Show Port Configuration
# /sbin/connectx_port_config -s
-------------------------------- Port configuration for PCI device: 0000:16:00.0 is: eth eth --------------------------------
- Looking at saved configuration
# vim /etc/infiniband/connectx.conf
- Switching between Ethernet and Infiniband
- Configuration supported by VPI
- The following configurations are supported by VPI: Port1 = eth Port2 = eth Port1 = ib Port2 = ib Port1 = auto Port2 = auto Port1 = ib Port2 = eth Port1 = ib Port2 = auto Port1 = auto Port2 = eth Note: the following options are not supported: Port1 = eth Port2 = ib Port1 = eth Port2 = auto Port1 = auto Port2 = ib
For more information, see
- ConnectX -3 VPI Single and Dual QSFP+ Port Adapter Card User Manual (pdf)
- Open Fabrics Enterprise Distribution (OFED) ConnectX driver (mlx4) in OFED 1.4 Release Notes