带有AF_XDP的DPDK :无法创建xsk套接字

Question

我正在尝试运行AF_XDP轮询模式驱动程序。内核版本为5.4(CONFIG_XDP_SOCKETS=y)。当我运行DPDK网站提供的示例时，发生了一个错误。

root@n211-203-164:~# dpdk-testpmd --vdev=net_af_xdp0,iface=eth3 -- -i --total-num-mbufs=10240
EAL: Detected 96 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: No available 1048576 kB hugepages reported
EAL: VFIO support initialized
EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:5e:00.0 (socket 0)
mlx5_net: Default miss action is not supported.
EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:5e:00.1 (socket 0)
mlx5_net: Default miss action is not supported.
EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:86:00.0 (socket 1)
mlx5_net: Default miss action is not supported.
EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:86:00.1 (socket 1)
mlx5_net: Default miss action is not supported.
Interactive-mode selected
testpmd: create a new mbuf pool <mb_pool_0>: n=10240, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
testpmd: create a new mbuf pool <mb_pool_1>: n=10240, size=2176, socket=1
testpmd: preferred mempool ops selected: ring_mp_mc

Warning! port-topology=paired and odd forward ports number, the last port will pair with itself.

Configuring Port 0 (socket 0)
Port 0: B8:CE:F6:35:DB:3A
Configuring Port 1 (socket 0)
Port 1: B8:CE:F6:35:DB:3B
Configuring Port 2 (socket 1)
Port 2: B8:CE:F6:3B:31:4A
Configuring Port 3 (socket 1)
Port 3: B8:CE:F6:3B:31:4B
Configuring Port 4 (socket 0)
xsk_configure(): Failed to create xsk socket.
eth_rx_queue_setup(): Failed to configure xdp socket
Fail to configure port 4 rx queues
EAL: Error - exiting with code: 1
  Cause: Start ports failed

似乎XSK套接字调用bind()失败，因此eth_rx_queue_setup()也失败了。

如何正确配置EAL参数以使用vdev=af_net_xdp0运行dpdk-testpmd？任何帮助都是非常感谢的。

Answer 1

这是一个关于rlimit的问题。我写了一个简单的测试程序。

#include <bpf.h>
#include <xsk.h>
#include <stdio.h>
//#include <sys/resource.h>

//static struct rlimit r = {RLIM_INFINITY, RLIM_INFINITY};

int main() {
        //setrlimit(RLIMIT_MEMLOCK, & r);

        int fd = bpf_create_map_name(BPF_MAP_TYPE_XSKMAP, "xsks_map", sizeof(int), sizeof(int), 1, 0);
        printf("fd:%d\n", fd);
        return 0;
}

执行命令strace -e bpf ./test-prog，结果可能如下所示：

root@n211-203-164:~# strace -e bpf ./test 
bpf(BPF_MAP_CREATE, {map_type=0x11 /* BPF_MAP_TYPE_??? */, key_size=4, value_size=4, max_entries=1}, 112) = -1 EPERM (Operation not permitted)
fd:-1 1
+++ exited with 0 +++

这意味着max locked memory导致了资源分配不足。执行命令ulimit -a可以发现max locked memory是64k字节。有两种解决方案，一种是通过执行cmd ulimit -S -l <a big value>来全局放大max locked memory，另一种是在创建BPF map时调用setrlimit()到libbpf.so中。