如何忽略jq中破损的JSON行？

Question

当使用jq处理日志文件时，可能会中断一些行，因此jq抛出错误并停止处理。

例如完整的日志：

{"level":"debug","time":"2021-09-24T19:42:47.140+0800","message":"sent send binary to ws server1","pid":41491,"cid":"32likw","num":1,"count":5120}
{"level":"debug","time":"2021-09-24T19:42:47.305+0800","message":"sent send binary to ws server2","pid":41491,"cid":"32likw","num":1,"count":5120}
{"level":"debug","time":"2021-09-24T19:42:47.469+0800","message":"sent send binary to ws server3","pid":41491,"cid":"32likw","num":1,"count":5120}
{"level":"debug","time":"2021-09-24T19:42:47.499+0800","message":"sent send binary to ws server4","pid":41491,"cid":"32likw","num":1,"count":5120}
{"level":"debug","time":"2021-09-24T19:42:47.581+0800","message":"sent send binary to ws server5","pid":41491,"cid":"32likw","num":1,"count":5120}

jq处理得很好：

< snippet1.json jq -C -r '.message'
sent send binary to ws server1
sent send binary to ws server2
sent send binary to ws server3
sent send binary to ws server4
sent send binary to ws server5

坏掉的(第3行的最后一部分不见了)：

{"level":"debug","time":"2021-09-24T19:42:47.140+0800","message":"sent send binary to ws server1","pid":41491,"cid":"32likw","num":1,"count":5120}
{"level":"debug","time":"2021-09-24T19:42:47.305+0800","message":"sent send binary to ws server2","pid":41491,"cid":"32likw","num":1,"count":5120}
{"level":"debug","time":"2021-09-24T19:42:47.469+0800","message":"sent send binary to ws server3","pi
{"level":"debug","time":"2021-09-24T19:42:47.499+0800","message":"sent send binary to ws server4","pid":41491,"cid":"32likw","num":1,"count":5120}
{"level":"debug","time":"2021-09-24T19:42:47.581+0800","message":"sent send binary to ws server5","pid":41491,"cid":"32likw","num":1,"count":5120}

jq停在断线处：

< snippet2.json jq -C -r '.message'
sent send binary to ws server1
sent send binary to ws server2
parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 4, column 2

我希望jq可以忽略第三行，继续，就像这样：

< snippet2.json jq -C -r '.message'
sent send binary to ws server1
sent send binary to ws server2
sent send binary to ws server4
sent send binary to ws server5

我试着使用在-R中提到的另一个职位，但这并没有帮助解决这个问题。

< snippet2.json jq -C -R -r '.message'
jq: error (at <stdin>:1): Cannot index string with string "message"
jq: error (at <stdin>:2): Cannot index string with string "message"
jq: error (at <stdin>:3): Cannot index string with string "message"
jq: error (at <stdin>:4): Cannot index string with string "message"
jq: error (at <stdin>:5): Cannot index string with string "message"

您能告诉我是否有任何解决方案/技能可以忽略/跳过/抑制这样的错误并得到其他的结果吗？

Answer 1

对匹克的回答再做几个解释。(学分达到顶峰)

解决方案1：

❯ cat bad.json | jq -r -R 'fromjson? | .message'
sent send binary to ws server1
sent send binary to ws server2
sent send binary to ws server4
sent send binary to ws server5

解决方案2：

❯ cat bad.json | jq -r -R '. as $line | try fromjson catch $line | .message'
sent send binary to ws server1
sent send binary to ws server2
jq: error (at <stdin>:3): Cannot index string with string "message"
sent send binary to ws server4
sent send binary to ws server5

jq仍然输出错误，但它位于stderr上，您可以重定向它：

❯ cat bad.json | jq -r -R '. as $line | try fromjson catch $line | .message' 2>/dev/null
sent send binary to ws server1
sent send binary to ws server2
sent send binary to ws server4
sent send binary to ws server5

值得注意的是，它可以同时使用-R和-r。(谢谢你的“峰”!)

Answer 2

要跳过断行，您可以使用：

jq -Rr 'fromjson? | .message'

如果你想用它们做其他的事情，你可以从这样的事情开始：

jq -R '. as $line | try fromjson catch $line'

关于其他选择，见：

：有什么方法可以让jq在遇到输入文件中的错误后继续运行吗？jq能处理坏的JSON吗？

在jq常见问题里。