.apply_on_packets的超时是不可预测的吗？

Question

使用以下advice时，以下代码会产生不可预知的结果：

import pyshark
import pandas as pd
import asyncio

def ProcessPackets(packet):
    global packet_list
    packet_version = packet.layers[1].version
    layer_name = packet.layers[2].layer_name
    packet_list.append([packet_version, layer_name, packet.length, packet.sniff_time])

def Capture(timeOrPath):
    global packet_list
    packet_list=[]
    try:
        timeout=int(timeOrPath)
        capture = pyshark.LiveCapture()          
        capture.apply_on_packets(ProcessPackets, timeout=timeout)
    except asyncio.TimeoutError:
        pass
    except ValueError:
        capture = pyshark.FileCapture(timeOrPath)
        capture.load_packets()
        capture.apply_on_packets(ProcessPackets)
    data = pd.DataFrame(packet_list, columns=['vIP', 'protocol', 'length','timestamp']) 
    print(data['timestamp'].iloc[-1]-data['timestamp'].iloc[0])

def main(): 
    Capture(6)

if __name__ == '__main__':
    main()

有时计算的时间会超过给定的超时时间。(timestamp是packet.sniff_time)

Answer 1

更新06-03-2021

在对这个捕获延迟问题做了一些研究之后，我确定这个问题很可能与pyshark等待dumpcap加载有关。dumpcap在LiveCapture模式下加载

  def _get_dumpcap_parameters(self):
        # Don't report packet counts.
        params = ["-q"]
        if self._get_tshark_version() < LooseVersion("2.5.0"):
            # Tshark versions older than 2.5 don't support pcapng. This flag forces dumpcap to output pcap.
            params += ["-P"]
        if self.bpf_filter:
            params += ["-f", self.bpf_filter]
        if self.monitor_mode:
            params += ["-I"]
        for interface in self.interfaces:
            params += ["-i", interface]
        # Write to STDOUT
        params += ["-w", "-"]
        return params

    async def _get_tshark_process(self, packet_count=None, stdin=None):
        read, write = os.pipe()

        dumpcap_params = [get_process_path(process_name="dumpcap", tshark_path=self.tshark_path)] + self._get_dumpcap_parameters()

        self._log.debug("Creating Dumpcap subprocess with parameters: %s" % " ".join(dumpcap_params))
        dumpcap_process = await asyncio.create_subprocess_exec(*dumpcap_params, stdout=write,
                                                               stderr=self._stderr_output())
        self._created_new_process(dumpcap_params, dumpcap_process, process_name="Dumpcap")

        tshark = await super(LiveCapture, self)._get_tshark_process(packet_count=packet_count, stdin=read)
        return tshark

上面的代码在我的系统上启动：

 /usr/local/bin/dumpcap -q -i en0 -w -

还有这个：

/usr/local/bin/tshark -l -n -T pdml -r -

我尝试将一些自定义参数传递给LiveCapture

capture = pyshark.LiveCapture(interface='en0', custom_parameters=["-q", "--no-promiscuous-mode", "-l"])

但是仍然有大约1/2秒的延迟。

10.015577793121338
0 days 00:00:09.371264

在dumpcap文档中有一个-a模式，它允许持续时间超时，但我无法将该参数传递到pyshark中而不导致错误。

Tshark也有一个-a模式，但它也会在pyshark中导致一个错误

capture = pyshark.LiveCapture(interface='en0', override_prefs={'': '-r'}, custom_parameters={'': '-a duration:20'})

可能有办法修改pyshark代码库中的超时参数，以允许-a模式。要做到这一点需要一些测试，我现在没有时间做这些。

我和pyshark的开发人员一起打开了一个issue on this problem。

原文06-02-2021

我重新编写了您的代码，将提取的项目写入到pandas数据帧中。如果这不是您想要的，请根据您的确切要求更新您的问题。

import pyshark
import asyncio
import pandas as pd

packet_list = []


def process_packets(packet):
    global packet_list
    try:
        packet_version = packet.layers[1].version
        layer_name = packet.layers[2].layer_name
        packet_list.append([packet_version, layer_name, packet.length, str(packet.sniff_time)])
    except AttributeError:
        pass


def capture_packets(timeout):
    capture = pyshark.LiveCapture(interface='en0')
    try:
        capture.apply_on_packets(process_packets, timeout=timeout)
    except asyncio.TimeoutError:
        pass
    finally:
        return packet_list


def main():
    capture_packets(6)
    df = pd.DataFrame(packet_list, columns=['packet version', 'layer type', 'length', 'capture time'])
    print(df)
    # output 
          packet version layer type length                capture time
    0                 4        udp     75  2021-06-02 16:22:36.463805
    1                 4        udp     67  2021-06-02 16:22:36.517076
    2                 4        udp   1388  2021-06-02 16:22:36.706240
    3                 4        udp   1392  2021-06-02 16:22:36.706245
    4                 4        udp   1392  2021-06-02 16:22:36.706246
    truncated...


if __name__ == '__main__':
    main()