...
Make sure to use the core local to the HCA, in this example the HDR InfiniBand adapter is local to core 80
More iterations helps to make the output more smooth. In this example, I am We are using also 10000.
Expected RDMA write latency is around 1usec (0.97-1.02) for Rome 7742 2.25GHz using HDR InfiniBand adapter over single HDR switch.
NPS Configuration is not critical here
10000 iterations.
Command Example:
| Code Block |
|---|
# numactl --physcpubind=80 ib_write_lat -a -d mlx5_2 -i 1 --report_gbits -F -n 10000 & ssh rome002 numactl --physcpubind=80 ib_write_lat -a -d mlx5_2 -i 1 --report_gbits -F rome001 -n 10000 |
...
Make sure to use the core local to the HCA, in this example the HDR InfiniBand adapter is local to core 80
More iterations helps to make the output more smooth. In this example, I am We are using also 10000.Expected RDMA write bandwidth line rate around 8-16K message size for Rome 7742 2.25GHz using HDR InfiniBand adapter over single HDR switch10000 iterations.
NPS Configuration should set to 1 (or 2) for HDR for maximum bandwidth.
...