如何从shell脚本中的日志文件中获取特定ip地址并转换为Python

Question

我是python的新手，我有一个日志文件，其中包含数百个目标ip，我从日志文件中只获取了目标ip地址，这是代码

f = open('/Users/kiya/Desktop/webgw logs/check/1st.log', 'r')
list_ip = sorted([])
for line in f:
     k = line.split()[3]
     list_ip.append(k)
ips = list(set(list_ip))
for ip in ips:
    #print(ip,'Count',list_ip.count(ip))
    print (ip)

在python中，如何根据下面的shell脚本过滤IP地址

cat 1st.log | grep -e "proto=6" -e "proto=17" | grep -e "srcip=10\." -e "srcip=172\." -e "srcip=192\.168" | grep -v -e "dstip=10\." -e "dstip=169\.254" -e "dstip=17\." -e "dstip=208\.91\.11[2-3]\." -e "dstip=172\." | cut -d ' ' -f 7 | sort | uniq | awk 'match($0,/[0-9]+.[0-9]+.[0-9]+.[0-9]+/){print substr($0, RSTART, RLENGTH)}' > list.txt

这是我的基本日志文件，里面有很多数据

#time_stamp "auth_user" dest_ip src_ip status_code "req_line" "categories" "rep_level" "media_type" bytes_to_client bytes_from_client bytes_to_server bytes_from_server "user_agent" "virus_name" block_res "application_name"
[30/Dec/2017:23:59:47 +0900] "" 255.255.255.255 172.16.23.177 407 "CONNECT dapi.ds-intel.net:443 HTTP/1.1" "" "-" "" 2935 243 0 0 "Devicescape-Agent/2.0.999 (Windows 7 - 34525574-f744-a01f-f5a2-bbca6285294e) tmobile_wispr1" "" 81 ""
[30/Dec/2017:23:59:47 +0900] "" 255.255.255.255 172.16.23.177 407 "CONNECT dapi.ds-intel.net:443 HTTP/1.1" "" "-" "" 2946 327 0 0 "Devicescape-Agent/2.0.999 (Windows 7 - 34525574-f744-a01f-f5a2-bbca6285294e) tmobile_wispr1" "" 81 "

Answer 1

IIUC：

import pandas as pd

ips = pd.read_csv('/Users/kiya/Desktop/webgw logs/check/1st.log',
                  delim_whitespace=True, usecols=[3], squeeze=True)
print(ips.unique())