在使用SR-IOV时，为什么我的主机操作系统在执行网络连接的核心上遇到高系统cpu使用率？

Question

我正在尝试确定为什么我的KVM主机显示特定来宾的系统CPU使用率很高。

我已经设置了一个KVM主机(Ubuntu 20.04)来托管来宾VM (Ubuntu 20.04)。我将来宾配置为使用插槽1中的处理器内核(通过vcpu / vcpupin /模拟器NIC )、连接到插槽1中的处理器通道的内存(通过numatune)以及连接到插槽1中的处理器通道的NIC(通过接口/地址源)，以确保VM在最佳状态下运行。即使我已经完成了配置IOMMU、SR-IOV和Intel VF以将NIC的虚拟功能PCI设备直接传递到虚拟机的过程，主机实例仍然会在来宾寄存其NIC中断的核心上看到高系统cpu使用率。

我不知道为什么涉及主机系统，因为PCI设备与主机断开连接，并直接连接到来宾系统。这会导致巨大的性能问题。有没有人经历过这种情况？我这一端有没有配置错误？请查看下面的配置和mpstat数据：

  <vcpu placement='static' cpuset='0,2,4,6,8,10,12,14,16,18,20,22,24,52,54,56,58,60,62,64,66,68,70,72,74,76'>26</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='0'/>
    <vcpupin vcpu='1' cpuset='2'/>
    <vcpupin vcpu='2' cpuset='4'/>
    <vcpupin vcpu='3' cpuset='6'/>
    <vcpupin vcpu='4' cpuset='8'/>
    <vcpupin vcpu='5' cpuset='10'/>
    <vcpupin vcpu='6' cpuset='12'/>
    <vcpupin vcpu='7' cpuset='14'/>
    <vcpupin vcpu='8' cpuset='16'/>
    <vcpupin vcpu='9' cpuset='18'/>
    <vcpupin vcpu='10' cpuset='20'/>
    <vcpupin vcpu='11' cpuset='22'/>
    <vcpupin vcpu='12' cpuset='24'/>
    <vcpupin vcpu='13' cpuset='52'/>
    <vcpupin vcpu='14' cpuset='54'/>
    <vcpupin vcpu='15' cpuset='56'/>
    <vcpupin vcpu='16' cpuset='58'/>
    <vcpupin vcpu='17' cpuset='60'/>
    <vcpupin vcpu='18' cpuset='62'/>
    <vcpupin vcpu='19' cpuset='64'/>
    <vcpupin vcpu='20' cpuset='66'/>
    <vcpupin vcpu='21' cpuset='68'/>
    <vcpupin vcpu='22' cpuset='70'/>
    <vcpupin vcpu='23' cpuset='72'/>
    <vcpupin vcpu='24' cpuset='74'/>
    <vcpupin vcpu='25' cpuset='76'/>
    <emulatorpin cpuset='0,2,4,6,8,10,12,14,16,18,20,22,24,52,54,56,58,60,62,64,66,68,70,72,74,76'/>
  </cputune>
  <numatune>
    <memory mode='strict' nodeset='0'/>
  </numatune>
  <cpu mode='host-passthrough' check='none'/>
  
    <interface type='hostdev' managed='yes'>
      <mac address='52:54:08:06:48:01'/>
      <driver name='vfio'/>
      <source>
        <address type='pci' domain='0x0000' bus='0x19' slot='0x02' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </interface>
    <interface type='hostdev' managed='yes'>
      <mac address='52:54:08:06:48:01'/>
      <driver name='vfio'/>
      <source>
        <address type='pci' domain='0x0000' bus='0x19' slot='0x0a' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
    </interface>
    <interface type='hostdev' managed='yes'>
      <mac address='52:54:08:06:48:02'/>
      <driver name='vfio'/>
      <source>
        <address type='pci' domain='0x0000' bus='0x19' slot='0x02' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/>
    </interface>
    <interface type='hostdev' managed='yes'>
      <mac address='52:54:08:06:48:02'/>
      <driver name='vfio'/>
      <source>
        <address type='pci' domain='0x0000' bus='0x19' slot='0x0a' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/>
    </interface>

22:26:38     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
22:26:43     all    0.00    0.00    3.39    0.00    0.00    0.09    0.00    7.30    0.00   89.22
22:26:43       0    0.00    0.00    7.30    0.00    0.00    0.00    0.00   26.57    0.00   66.13
22:26:43       1    0.00    0.00    0.00    0.00    0.00    8.43    0.00    0.00    0.00   91.57
22:26:43       2    0.00    0.00   10.38    0.00    0.00    0.00    0.00   29.34    0.00   60.28
22:26:43       4    0.00    0.00    8.69    0.00    0.00    0.00    0.00   29.09    0.00   62.22
22:26:43       6    0.00    0.00    5.00    0.00    0.00    0.00    0.00   23.20    0.00   71.80
22:26:43       8    0.20    0.00   39.31    0.00    0.00    0.00    0.00   35.03    0.00   25.46
22:26:43      10    0.00    0.00    6.96    0.00    0.00    0.00    0.00   31.21    0.00   61.83
22:26:43      12    0.00    0.00   44.90    0.00    0.00    0.00    0.00   35.31    0.00   19.80
22:26:43      14    0.00    0.00   13.18    0.00    0.00    0.00    0.00   32.05    0.00   54.77
22:26:43      16    0.20    0.00    5.31    0.00    0.00    0.00    0.00   26.12    0.00   68.37
22:26:43      18    0.00    0.00   15.95    0.00    0.00    0.20    0.00   30.47    0.00   53.37
22:26:43      20    0.00    0.00    7.04    0.00    0.00    0.00    0.00   26.16    0.00   66.80
22:26:43      22    0.00    0.00   15.76    0.00    0.00    0.00    0.00   31.92    0.00   52.32
22:26:43      24    0.00    0.00    7.35    0.00    0.00    0.00    0.00   27.96    0.00   64.69
22:26:43      52    0.00    0.00   10.46    0.00    0.00    0.00    0.00   29.78    0.00   59.76
22:26:43      54    0.00    0.00    7.35    0.00    0.00    0.00    0.00   26.12    0.00   66.53
22:26:43      56    0.00    0.00    6.91    0.00    0.00    0.00    0.00   29.67    0.00   63.41
22:26:43      58    0.00    0.00    7.82    0.00    0.00    0.00    0.00   29.42    0.00   62.76
22:26:43      60    0.00    0.00   37.17    0.00    0.00    0.00    0.00   35.15    0.00   27.68
22:26:43      62    0.00    0.00   14.99    0.00    0.00    0.00    0.00   28.54    0.00   56.47
22:26:43      64    0.00    0.00   37.50    0.00    0.00    0.00    0.00   33.53    0.00   28.97
22:26:43      66    0.00    0.00    7.30    0.00    0.00    0.00    0.00   29.61    0.00   63.08
22:26:43      68    0.00    0.00    8.08    0.00    0.00    0.00    0.00   27.88    0.00   64.04
22:26:43      70    0.00    0.00    7.09    0.00    0.00    0.00    0.00   24.49    0.00   68.42
22:26:43      72    0.20    0.00    6.94    0.00    0.00    0.00    0.00   29.96    0.00   62.90
22:26:43      74    0.00    0.00    7.63    0.00    0.00    0.00    0.00   27.91    0.00   64.46
22:26:43      76    0.00    0.00    8.63    0.00    0.00    0.00    0.00   29.32    0.00   62.05
22:26:43      98    0.00    0.00    0.20    0.00    0.00    0.00    0.00    0.00    0.00   99.80
22:26:43     100    0.00    0.00    0.20    0.00    0.00    0.00    0.00    0.00    0.00   99.80

有没有人知道什么配置不正确，或者为什么8、12、60、64核的%sys如此之高？

Answer 1

这最终成为默认kvm.halt_poll_ns值的问题。将/sys/module/kvm/parameters/halt_poll_ns设置为0会导致此问题停止发生。