Skip to content

Commit

Permalink
d
Browse files Browse the repository at this point in the history
  • Loading branch information
weizhoublue committed Dec 14, 2023
1 parent deaab5c commit 1b89e23
Show file tree
Hide file tree
Showing 2 changed files with 31 additions and 29 deletions.
58 changes: 30 additions & 28 deletions docs/usage/rdma-ib-zh_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -206,34 +206,36 @@

开启一个终端,进入另一个 Pod 访问服务:

# 能看到宿主机上的所有 RDMA 网卡
~# rdma link
link mlx5_8/1 subnet_prefix fe80:0000:0000:0000 lid 7 sm_lid 1 lmc 0 state ACTIVE physical_state LINK_UP

# 访问对方 Pod 的 RDMA 服务
~# ib_read_lat 172.91.0.115
---------------------------------------------------------------------------------------
RDMA_Read Latency Test
Dual-port : OFF Device : mlx5_8
Number of qps : 1 Transport type : IB
Connection type : RC Using SRQ : OFF
PCIe relax order: ON
ibv_wr* API : ON
TX depth : 1
Mtu : 4096[B]
Link type : IB
Outstand reads : 16
rdma_cm QPs : OFF
Data ex. method : Ethernet
---------------------------------------------------------------------------------------
local address: LID 0x07 QPN 0x012e PSN 0x7eb74 OUT 0x10 RKey 0x030509 VAddr 0x005560e826f000
remote address: LID 0x08 QPN 0x00ee PSN 0x7eb74 OUT 0x10 RKey 0x020509 VAddr 0x005560f99dc000
---------------------------------------------------------------------------------------
#bytes #iterations t_min[usec] t_max[usec] t_typical[usec] t_avg[usec] t_stdev[usec] 99% percentile[usec] 99.9% percentile[usec]
Conflicting CPU frequency values detected: 1000.085000 != 2200.000000. CPU Frequency is not max.
Conflicting CPU frequency values detected: 1000.383000 != 2200.000000. CPU Frequency is not max.
2 1000 1.84 12.20 1.90 1.97 0.47 2.24 12.20
---------------------------------------------------------------------------------------
```shell

Check failure on line 209 in docs/usage/rdma-ib-zh_CN.md

View workflow job for this annotation

GitHub Actions / Markdown Lint

Code block style [Expected: indented; Actual: fenced]
# 能看到宿主机上的所有 RDMA 网卡
~# rdma link
link mlx5_8/1 subnet_prefix fe80:0000:0000:0000 lid 7 sm_lid 1 lmc 0 state ACTIVE physical_state LINK_UP

# 访问对方 Pod 的 RDMA 服务
~# ib_read_lat 172.91.0.115
---------------------------------------------------------------------------------------
RDMA_Read Latency Test
Dual-port : OFF Device : mlx5_8
Number of qps : 1 Transport type : IB
Connection type : RC Using SRQ : OFF
PCIe relax order: ON
ibv_wr* API : ON
TX depth : 1
Mtu : 4096[B]
Link type : IB
Outstand reads : 16
rdma_cm QPs : OFF
Data ex. method : Ethernet
---------------------------------------------------------------------------------------
local address: LID 0x07 QPN 0x012e PSN 0x7eb74 OUT 0x10 RKey 0x030509 VAddr 0x005560e826f000
remote address: LID 0x08 QPN 0x00ee PSN 0x7eb74 OUT 0x10 RKey 0x020509 VAddr 0x005560f99dc000
---------------------------------------------------------------------------------------
#bytes #iterations t_min[usec] t_max[usec] t_typical[usec] t_avg[usec] t_stdev[usec] 99% percentile[usec] 99.9% percentile[usec]
Conflicting CPU frequency values detected: 1000.085000 != 2200.000000. CPU Frequency is not max.
Conflicting CPU frequency values detected: 1000.383000 != 2200.000000. CPU Frequency is not max.
2 1000 1.84 12.20 1.90 1.97 0.47 2.24 12.20
---------------------------------------------------------------------------------------
```

## 基于 IPoIB 的常规网卡

Expand Down
2 changes: 1 addition & 1 deletion docs/usage/rdma-roce.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ RDMA devices' network namespaces have two modes: shared and exclusive. Container

(2) Mellanox OFED version 4.7 or newer is required. In this case it is not required to use a Kernel based on 5.3.0 or newer.

Moreover, for applications using clusterIP for RDMA communication, it must take the underlay network card to forwarded RDMA traffic, so it needs to implement the clusterIP by cgroup eBPF in the container network namespace. For specific details, please refer to [cgroup eBPF Resolving ClusterIP](./underlay_cni_service zh-CN.md)
Moreover, for applications using clusterIP for RDMA communication, it must take the underlay network card to forwarded RDMA traffic, so it needs to implement the clusterIP by cgroup eBPF in the container network namespace. For specific details, please refer to [cgroup eBPF Resolving ClusterIP](./underlay_cni_service.md#access-service-by-cgroup-ebpf)

## Shared RoCE NIC with macvlan or ipvlan

Expand Down

0 comments on commit 1b89e23

Please sign in to comment.