错误附加装置到DPDK

Question

在Ubuntu上向OVS键添加接口时，将设备附加到DPDK时出错。

• Ubuntu 20.04.3
• DPDK 19.11.10
• OVS 2.13.3
• 英特尔x710 (fw 7.10)

命令：

ovs-vsctl add-bond ovsbr0 bond0  ens2f0 ens2f1 ens3f0 ens3f1 \
   -- set Interface ens2f0 type=dpdk  "options:dpdk-devargs=0000:5e:00.0" \
   -- set Interface ens2f1 type=dpdk  "options:dpdk-devargs=0000:5e:00.1" \
   -- set Interface ens3f0 type=dpdk  "options:dpdk-devargs=0000:d8:00.0" \
   -- set Interface ens3f1 type=dpdk  "options:dpdk-devargs=0000:d8:00.1"

输出：

ovs-vsctl: Error detected while setting up 'ens2f0': Error attaching device '0000:5e:00.0' to DPDK.  See ovs-vswitchd log for details.
ovs-vsctl: Error detected while setting up 'ens2f1': Error attaching device '0000:5e:00.1' to DPDK.  See ovs-vswitchd log for details.
ovs-vsctl: Error detected while setting up 'ens3f0': Error attaching device '0000:d8:00.0' to DPDK.  See ovs-vswitchd log for details.
ovs-vsctl: Error detected while setting up 'ens3f1': Error attaching device '0000:d8:00.1' to DPDK.  See ovs-vswitchd log for details.
ovs-vsctl: The default log directory is "/var/log/openvswitch".

/var/log/openvswitch/ovs-vSwitd.log

2022-01-03T16:00:05.776Z|00083|dpdk|ERR|EAL: Driver cannot attach the device (0000:d8:00.1)
2022-01-03T16:00:05.776Z|00084|dpdk|ERR|EAL: Failed to attach device on primary process
2022-01-03T16:00:05.776Z|00085|netdev_dpdk|WARN|Error attaching device '0000:d8:00.1' to DPDK
2022-01-03T16:00:05.776Z|00086|netdev|WARN|ens3f1: could not set configuration (Invalid argument)
2022-01-03T16:00:05.776Z|00087|dpdk|ERR|Invalid port_id=32

启用了VT支持：

cat /proc/cmdline 
BOOT_IMAGE=/boot/vmlinuz-5.4.0-91-generic root=UUID=2849776f-b167-447f-a5d1-ea4b5c831c35 ro iommu=pt intel_iommu=on

我用vfio驱动程序。

DMAR：

$ grep -i dmar charlie-dmesg2.yml | grep -i iommu
[    0.703534] DMAR: IOMMU enabled
[    1.353766] DMAR-IR: IOAPIC id 12 under DRHD base  0xc5ffc000 IOMMU 6
[    1.353767] DMAR-IR: IOAPIC id 11 under DRHD base  0xb87fc000 IOMMU 5
[    1.353768] DMAR-IR: IOAPIC id 10 under DRHD base  0xaaffc000 IOMMU 4
[    1.353770] DMAR-IR: IOAPIC id 18 under DRHD base  0xfbffc000 IOMMU 3
[    1.353771] DMAR-IR: IOAPIC id 17 under DRHD base  0xee7fc000 IOMMU 2
[    1.353773] DMAR-IR: IOAPIC id 16 under DRHD base  0xe0ffc000 IOMMU 1
[    1.353774] DMAR-IR: IOAPIC id 15 under DRHD base  0xd37fc000 IOMMU 0
[    1.353775] DMAR-IR: IOAPIC id 8 under DRHD base  0x9d7fc000 IOMMU 7
[    1.353777] DMAR-IR: IOAPIC id 9 under DRHD base  0x9d7fc000 IOMMU 7

此问题仅在Ubuntu自动安装的情况下发生。Ubuntu手动安装很好。

Answer 1

将pci=realloc=on添加到内核引导参数为我解决了这个问题