如何将Stdout解析为CSV和JSON的混合？

Question

我目前正在编写一个类，它让我们将代码提交给自动平分器，然后返回结果。它返回的格式很难直观地解析，所以我想编写一个脚本，我可以在管道中使用它，使它更容易阅读。

这是自动平地机的输出：

Problem,Correct?,Correct Answer,Agent's Answer
"Challenge Problem B-04",0,4,-1
"Basic Problem B-12",0,1,-1
"Challenge Problem B-05",0,6,-1
"Challenge Problem B-07",0,6,-1
"Challenge Problem B-06",0,3,-1
"Basic Problem B-11",0,1,-1
"Basic Problem B-10",0,3,-1
"Challenge Problem B-03",0,3,-1
"Challenge Problem B-02",0,1,-1
"Challenge Problem B-01",0,6,-1
"Challenge Problem B-09",0,4,-1
"Challenge Problem B-08",0,4,-1
"Basic Problem B-08",0,6,-1
"Basic Problem B-09",0,5,-1
"Basic Problem B-04",0,3,-1
"Basic Problem B-05",0,4,-1
"Basic Problem B-06",0,5,-1
"Basic Problem B-07",0,6,-1
"Basic Problem B-01",0,2,-1
"Basic Problem B-02",0,5,-1
"Basic Problem B-03",0,1,-1
"Challenge Problem B-10",0,4,-1
"Challenge Problem B-11",0,5,-1
"Challenge Problem B-12",0,1,-1
{
    "Basic Problems B": {
        "Incorrect": "0",
        "Skipped": "12",
        "Correct": "0",
        "Set": "Basic Problems B"
    },
    "Challenge Problems B": {
        "Incorrect": "0",
        "Skipped": "12",
        "Correct": "0",
        "Set": "Challenge Problems B"
    }
}

它是逗号分隔的值和JSON的混合。把这些都放在我能读到的漂亮的桌子上会很好。

目前，我有一件事

python submit.py --provider gt --assignment error-check | column -t -s, | less -S

其中产出：

{
    "Basic Problems B": {
        "Incorrect": "0",
        "Skipped": "12",
        "Correct": "0",
        "Set": "Basic Problems B"
    },
    "Challenge Problems B": {
        "Incorrect": "0",
        "Skipped": "12",
        "Correct": "0",
        "Set": "Challenge Problems B"
    }
}
Problem                   Correct?  Correct Answer  Agent's Answer
"Challenge Problem B-04"  0         4               -1
"Basic Problem B-12"      0         1               -1
"Challenge Problem B-05"  0         6               -1
"Challenge Problem B-07"  0         6               -1
"Challenge Problem B-06"  0         3               -1
"Basic Problem B-11"      0         1               -1
"Basic Problem B-10"      0         3               -1
"Challenge Problem B-03"  0         3               -1
"Challenge Problem B-02"  0         1               -1
"Challenge Problem B-01"  0         6               -1
"Challenge Problem B-09"  0         4               -1
"Challenge Problem B-08"  0         4               -1
"Basic Problem B-08"      0         6               -1
"Basic Problem B-09"      0         5               -1
"Basic Problem B-04"      0         3               -1
"Basic Problem B-05"      0         4               -1
"Basic Problem B-06"      0         5               -1
"Basic Problem B-07"      0         6               -1
"Basic Problem B-01"      0         2               -1
"Basic Problem B-02"      0         5               -1
"Basic Problem B-03"      0         1               -1
"Challenge Problem B-10"  0         4               -1
"Challenge Problem B-11"  0         5               -1
"Challenge Problem B-12"  0         1               -1

我大部分的路都是这样的。现在，我想知道是否有一种方法可以处理JSON？

我不能依赖于将输出拆分到一个特定的行号，但我想当它第一次找到一个{时，我可以分割输出。

我想尽可能少地这样做，这样我就可以和同学们分享了。所以依赖性越小越好。

我见过其他JSON解析文章建议使用外部代码。

理想的输出应该如下所示：

Problem                   Correct?  Correct Answer  Agent's Answer
"Challenge Problem B-04"  0         4               -1
"Basic Problem B-12"      0         1               -1
"Challenge Problem B-05"  0         6               -1
"Challenge Problem B-07"  0         6               -1
"Challenge Problem B-06"  0         3               -1
"Basic Problem B-11"      0         1               -1
"Basic Problem B-10"      0         3               -1
"Challenge Problem B-03"  0         3               -1
"Challenge Problem B-02"  0         1               -1
"Challenge Problem B-01"  0         6               -1
"Challenge Problem B-09"  0         4               -1
"Challenge Problem B-08"  0         4               -1
"Basic Problem B-08"      0         6               -1
"Basic Problem B-09"      0         5               -1
"Basic Problem B-04"      0         3               -1
"Basic Problem B-05"      0         4               -1
"Basic Problem B-06"      0         5               -1
"Basic Problem B-07"      0         6               -1
"Basic Problem B-01"      0         2               -1
"Basic Problem B-02"      0         5               -1
"Basic Problem B-03"      0         1               -1
"Challenge Problem B-10"  0         4               -1
"Challenge Problem B-11"  0         5               -1
"Challenge Problem B-12"  0         1               -1

Set                   Incorrect Skipped Correct
Basic Problems B      0         12      0
Challenge Problems B  0         12      0

Answer 1

使用Miller (https://github.com/johnkerl/miller)并运行

# get the CSV and transform it into a pretty print table
<input grep -P '^("|\w)' | mlr --c2p cat >out
# add a carriage return
echo "" >> out
# convert the json into a pretty print table and add it to the output
<input grep -vP '^("|\w)'  | mlr --j2p cat -n then reshape -r "(Basi|Chal)" -o i,v \
then nest --explode --values --across-fields --nested-fs ":" -f i \
then reshape -s i_2,v \
then cut -x -f i_1,n \
then reorder -f Set >>out

你将会有

Problem                Correct? Correct Answer Agent's Answer
Challenge Problem B-04 0        4              -1
Basic Problem B-12     0        1              -1
Challenge Problem B-05 0        6              -1
Challenge Problem B-07 0        6              -1
Challenge Problem B-06 0        3              -1
Basic Problem B-11     0        1              -1
Basic Problem B-10     0        3              -1
Challenge Problem B-03 0        3              -1
Challenge Problem B-02 0        1              -1
Challenge Problem B-01 0        6              -1
Challenge Problem B-09 0        4              -1
Challenge Problem B-08 0        4              -1
Basic Problem B-08     0        6              -1
Basic Problem B-09     0        5              -1
Basic Problem B-04     0        3              -1
Basic Problem B-05     0        4              -1
Basic Problem B-06     0        5              -1
Basic Problem B-07     0        6              -1
Basic Problem B-01     0        2              -1
Basic Problem B-02     0        5              -1
Basic Problem B-03     0        1              -1
Challenge Problem B-10 0        4              -1
Challenge Problem B-11 0        5              -1
Challenge Problem B-12 0        1              -1

Set                  Incorrect Skipped Correct
Basic Problems B     0         12      0
Challenge Problems B 0         12      0