以正确的格式以json的形式获取passwd输出

Question

为了尝试将bash命令输出转换为json，我使用了jq。然而，转换失败了。

使用这一行：

hostname && getent passwd | egrep -v '/s?bin/(nologin|shutdown|sync|halt)' | cut -d: -f1 | sudo xargs -I % sh -c  'echo %; chage -l %; echo' | jq --raw-input . | jq --slurp .

• 主机名必须是json的根元素。
• 用户名必须是第一个元素
• getent passwd输出必须是用户名的子元素。

就像这样：JSON

在不使用jq的情况下输出上述命令，结果是：命令输出

有人试过正确地将bash输出转换为json吗？谢谢

Answer 1

我用的是jq。

我意识到jq在这里很受欢迎，但是我会推荐西德尔来完成这个任务。

虽然主要是一个HTML/XML/JSON解析器(使用CSS、XPath、XQuery、JSONiq和模式模板)，但是XML也可以处理基本文本。

你说你想要一个像这样的儿子：

{
  "hostname": "example",
  "users": [
    {
      "username": "example1",
      "details": {
        "LastPasswordChange": "Nov 12, 2018",
        "PasswordExpires": "never",
        "PasswordInactive": "never",
        "AccountExpires": "never",
        "MinDaysBetweenPasswordChange": "0",
        "MaxDaysBetweenPasswordChange": "99999",
        "NumOfDaysOfWarningBeforePasswordExpires": "7"
      }
    },
    {
      "username": "example2",
      "details": {
        "LastPasswordChange": "Nov 13, 2018",
        "PasswordExpires": "never",
        "PasswordInactive": "never",
        "AccountExpires": "never",
        "MinDaysBetweenPasswordChange": "0",
        "MaxDaysBetweenPasswordChange": "99999",
        "NumOfDaysOfWarningBeforePasswordExpires": "7"
      }
    }
  ]
}

这意味着users数组需要被对象填充(从这个chage程序的输出中创建)，用于每个用户单独的。这就是为什么我会选择另一种方法。

假设getent passwd | egrep -v '/s?bin/(nologin|shutdown|sync|halt)' | cut -d: -f1返回：

root
user1
user2

首先，我们将管道到Xidel，创建一个基本的json：

getent passwd | egrep -v '/s?bin/(nologin|shutdown|sync|halt)' | cut -d: -f1 | xidel -s - --xquery '
  {
    "hostname":"'$(hostname)'",
    "users":x:lines($raw) ! {
      "username":.
    }
  }
'

• 没有必要在原始命令前面使用hostname &&，因为您只需在Xidel中调用它(这也使查询更加容易)。
• 使用x:lines($raw)，您可以从西德尔的输入中创建每个新行的序列(本例中为stdin)。另一个要使用的函数是tokenize($raw,"\n")。

查询输出：

{
  "hostname": "user",
  "users": [
    {
      "username": "root"
    },
    {
      "username": "user1"
    },
    {
      "username": "user2"
    }
  ]
}

接下来，我们将从call中调用chage来创建details对象：

getent passwd | egrep -v '/s?bin/(nologin|shutdown|sync|halt)' | cut -d: -f1 | xidel -s - --xquery '
  {
    "hostname":"'$(hostname)'",
    "users":x:lines($raw) ! {
      "username":.,
      "details":{|
        for $x in x:lines(
          system(x"chage -l {.}")
        )
        let $a:=tokenize(
          $x,
          ":"
        )
        return {
          normalize-space($a[1]):normalize-space($a[2])
        }
      |}
    }
  }
'

• system(x"chage -l {.}")返回chage -l root、chage -l user1和chage -l user2的输出。
• 有了x:lines()，您将再次创建每个新行的序列。例如，user2输出的第一行：

Last password change                                  : Oct 23, 2018

• 对于此序列(或行)中的每一项，将创建变量$a，该变量包含该项的序列被分割成两部分，并以:作为分隔符。因此，对于第一项，$a：

Last password change                                  # white-space up untill this point! 
 Oct 23, 2018

• 最后，创建属性值对，并对空白进行规范化：

{
  "Last password change": "Oct 23, 2018"
}

查询输出：

{
  "hostname": "user",
  "users": [
    {
      "username": "root",
      "details": {
        "Last password change": "never",
        "Password expires": "never",
        "Password inactive": "never",
        "Account expires": "never",
        "Minimum number of days between password change": "0",
        "Maximum number of days between password change": "99999",
        "Number of days of warning before password expires": "7"
      }
    },
    {
      "username": "user1",
      "details": {
        "Last password change": "never",
        "Password expires": "never",
        "Password inactive": "never",
        "Account expires": "never",
        "Minimum number of days between password change": "0",
        "Maximum number of days between password change": "99999",
        "Number of days of warning before password expires": "7"
      }
    },
    {
      "username": "user2",
      "details": {
        "Last password change": "Oct 23, 2018",
        "Password expires": "never",
        "Password inactive": "never",
        "Account expires": "never",
        "Minimum number of days between password change": "0",
        "Maximum number of days between password change": "99999",
        "Number of days of warning before password expires": "7"
      }
    }
  ]
}

你可以再走一步：

getent passwd | egrep -v '/s?bin/(nologin|shutdown|sync|halt)' | cut -d: -f1 | xidel -s - --xquery '
  {
    "hostname":"'$(hostname)'",
    "users":x:lines($raw) ! {
      "username":.,
      "details":{|
        for $x in x:lines(
          system(x"chage -l {.}")
        )
        let $a:=tokenize(
          $x,
          ":"
        )
        return {
          string-join(
            for $x in tokenize(
              normalize-space($a[1]),
              " "
            )
            return
            upper-case(
              substring(
                $x,
                1,
                1
              )
            )||substring(
              $x,
              2
            )
          ):normalize-space($a[2]) ! (
            if (. castable as decimal) then
              number(.)
            else
              .
          )
        }
      |}
    }
  }
'

• 而不是Last password change，用..。

string-join(for $x in tokenize(normalize-space($a[1])," ") return upper-case(substring($x,1,1))||substring($x,2))

...it返回：LastPasswordChange。

• 而不是"99999" (字符串)，用..。

normalize-space($a[2]) ! (if (. castable as decimal) then number(.) else .)

...it返回：99999作为xs:十进制原子类型。

查询输出：

{
  "hostname": "user",
  "users": [
    {
      "username": "root",
      "details": {
        "LastPasswordChange": "never",
        "PasswordExpires": "never",
        "PasswordInactive": "never",
        "AccountExpires": "never",
        "MinimumNumberOfDaysBetweenPasswordChange": 0,
        "MaximumNumberOfDaysBetweenPasswordChange": 99999,
        "NumberOfDaysOfWarningBeforePasswordExpires": 7
      }
    },
    {
      "username": "user1",
      "details": {
        "LastPasswordChange": "never",
        "PasswordExpires": "never",
        "PasswordInactive": "never",
        "AccountExpires": "never",
        "MinimumNumberOfDaysBetweenPasswordChange": 0,
        "MaximumNumberOfDaysBetweenPasswordChange": 99999,
        "NumberOfDaysOfWarningBeforePasswordExpires": 7
      }
    },
    {
      "username": "user2",
      "details": {
        "LastPasswordChange": "Oct 23, 2018",
        "PasswordExpires": "never",
        "PasswordInactive": "never",
        "AccountExpires": "never",
        "MinimumNumberOfDaysBetweenPasswordChange": 0,
        "MaximumNumberOfDaysBetweenPasswordChange": 99999,
        "NumberOfDaysOfWarningBeforePasswordExpires": 7
      }
    }
  ]
}

“美化”查询更容易阅读，但如果您更喜欢“缩小”版本：

getent passwd | egrep -v '/s?bin/(nologin|shutdown|sync|halt)' | cut -d: -f1 | xidel -s - --xquery '{"hostname":"'$(hostname)'","users":x:lines($raw) ! {"username":.,"details":{|for $x in x:lines(system(x"chage -l {.}")) let $a:=tokenize($x,":") return {string-join(for $x in tokenize(normalize-space($a[1])," ") return upper-case(substring($x,1,1))||substring($x,2)):normalize-space($a[2]) ! (if (. castable as decimal) then number(.) else .)}|}}}'

Answer 2

下面是一个perl脚本，它以JSON格式显示/etc/passwd的主机名和内容(在我熟悉的任何操作系统上，输出都不像您的示例)。

注意，当然，当讨论JSON对象的键时，‘第一个元素’是没有意义的；它们是无序的。如果您依赖于某个特定的键在某个对象中的特定位置，那么您就错了。

#!/usr/bin/env perl
use warnings;
use strict;
use Sys::Hostname;
use User::pwent;
# Might have to install through your OS's package manager or
# with your favorite CPAN client
use JSON;

my %pwdb = ( hostname => hostname(), users => [] );

while (my $pw = getpwent) {
  # Ignore users with certain shells
  next if
    defined $pw->shell && $pw->shell =~ m!s?bin/(nologin|shutdown|sync|halt)!;
  push @{$pwdb{"users"}}, {
                           username => $pw->name,
                           details => {
                                       passwd => $pw->passwd,
                                       uid => $pw->uid,
                                       gid => $pw->gid,
                                       change => $pw->change,
                                       age => $pw->age,
                                       quota => $pw->quota,
                                       comment => $pw->comment,
                                       class => $pw->class,
                                       gecos => $pw->gecos,
                                       dir => $pw->dir,
                                       shell => $pw->shell,
                                       expire => $pw->expire
                                      }
                          };
}

my $pp = JSON->new->ascii->pretty;
print $pp->encode(\%pwdb);