如何从日志How输出中提取多个值

Question

我正在尝试调试一个ethereum beacon节点，该节点输出具有可变列数据的日志：

-- Logs begin at Sat 2021-01-30 19:05:03 CET, end at Wed 2021-02-24 11:19:14 CET. --
Feb 23 11:19:23 server beacon-chain[793]: time="2021-02-23 11:19:23" level=info msg="Synced new block" block=0x8f117f30... chainServiceProcessedTime=97.852794ms epoch=18884 finalizedEpoch=18882 finalizedRoot=0x9c26b9db... prefix=blockchain sinceSlotStartTime=504.745454ms slot=604295 slotInEpoch=7
Feb 23 11:19:23 server beacon-chain[793]: time="2021-02-23 11:19:23" level=info msg="Finished applying state transition" attestations=85 attesterSlashings=0 deposits=0 prefix=blockchain proposerSlashings=0 voluntaryExits=0
Feb 23 11:19:36 server beacon-chain[793]: time="2021-02-23 11:19:36" level=info msg="Synced new block" block=0xdf506bde... chainServiceProcessedTime=63.515885ms epoch=18884 finalizedEpoch=18882 finalizedRoot=0x9c26b9db... prefix=blockchain sinceSlotStartTime=1.492164357s slot=604296 slotInEpoch=8
Feb 23 11:19:36 server beacon-chain[793]: time="2021-02-23 11:19:36" level=info msg="Finished applying state transition" attestations=25 attesterSlashings=0 deposits=0 prefix=blockchain proposerSlashings=0 voluntaryExits=0
Feb 23 11:19:47 server beacon-chain[793]: time="2021-02-23 11:19:47" level=info msg="Synced new block" block=0x1785e975... chainServiceProcessedTime=81.050651ms epoch=18884 finalizedEpoch=18882 finalizedRoot=0x9c26b9db... prefix=blockchain sinceSlotStartTime=413.217351ms slot=604297 slotInEpoch=9
Feb 23 11:19:47 server beacon-chain[793]: time="2021-02-23 11:19:47" level=info msg="Finished applying state transition" attestations=61 attesterSlashings=0 deposits=0 prefix=blockchain proposerSlashings=0 voluntaryExits=0
...

我正在尝试提取epoch、slot和sinceSlotStartTime的值

journalctl -u servicename | sed -E -n 's/epoch=(\d*)|slot=(\d*)|sinceSlotStartTime=(\S*)ms/\1,\2,\3/p'

但是，这只是打印所有内容，任何指导都将不胜感激。

Answer 1

在awk中解析日志文件要容易得多

awk '{s=""; for (i=1; i<=NF; ++i) if ($i ~ /^(epoch|slot|sinceSlotStartTime)=/) {sub(/[^=]+=/, "", $i); s = s (s == "" ? "" : ",") $i+0} if (s) print s}' file

18884,504.745,604295
18884,1.49216,604296
18884,413.217,604297

PS: POSIX不支持\d、\S等sed正则表达式属性，我们必须使用它们的POSIX等价物，比如[[:digit:]]和[^[:blank:]]，但是sed在这方面仍然比awk更具挑战性。

扩展的awk命令：

awk '{
   s = ""
   for (i=1; i<=NF; ++i)
      if ($i ~ /^(epoch|slot|sinceSlotStartTime)=/) {
         sub(/[^=]+=/, "", $i)
         s = s (s == "" ? "" : ",") $i+0
      }
   if (s)
      print s
}' file

Answer 2

每当您的输入中有标记-值对时，我发现最好首先填充该映射的数组(下面的tag2val)，然后您可以打印或对任何您想要的值执行任何其他操作，只需通过它们的标记(名称)来引用它们：

$ cat tst.awk
BEGIN {
    OFS = ","
    numTags = split("epoch slot sinceSlotStartTime",tags)
    for (tagNr=1; tagNr<=numTags; tagNr++) {
        tag = tags[tagNr]
        printf "%s%s", tag, (tagNr<numTags ? OFS : ORS)
    }
}
/Synced new block/ {
    delete tag2val
    while ( match($0,/[^[:space:]]+=([^[:space:]]+|"[^"]+")/) ) {
        tag = val = substr($0,RSTART,RLENGTH)
        sub(/=.*/,"",tag)
        sub(/[^=]*=/,"",val)
        tag2val[tag] = val
        $0 = substr($0,RSTART+RLENGTH)
    }

    for (tagNr=1; tagNr<=numTags; tagNr++) {
        tag = tags[tagNr]
        val = tag2val[tag]
        printf "%s%s", val, (tagNr<numTags ? OFS : ORS)
    }
}

$ awk -f tst.awk file
epoch,slot,sinceSlotStartTime
18884,604295,504.745454ms
18884,604296,1.492164357s
18884,604297,413.217351ms

如果您想打印其他字段或以不同的顺序打印，则只需更新split()调用中的字符串"epoch slot sinceSlotStartTime"即可。

Answer 3

使用sed：

 journalctl -u servicename | sed -En 's/(^.*)(epoch=)([[:digit:]]+)(.*)(sinceSlotStartTime=)([[:digit:]]+)(.*)(slot=)([[:digit:]]+)(.*$)/\3,\6,\9/p'

使用-E或-r启用正则表达式解释，然后将该行拆分为用括号指定的部分。用此行替换第三、第六和第九部分，用逗号分隔。