Python YAML转JSON转YAML

Question

我是python的新手，所以我正在构建一个简单的程序来将YAML解析为JSON，将JSON解析为YAML。

yaml2json在一行中将YAML转换为JSON，但是JSON验证器说它是正确的。

这是我到目前为止的代码：

def parseyaml(inFileType, outFileType):
   infile = input('Please enter a {} filename to parse: '.format(inFileType))
   outfile = input('Please enter a {} filename to output: '.format(outFileType))

   with open(infile, 'r') as stream:
       try:
           datamap = yaml.safe_load(stream)
           with open(outfile, 'w') as output:
               json.dump(datamap, output)
       except yaml.YAMLError as exc:
           print(exc)

    print('Your file has been parsed.\n\n')


def parsejson(inFileType, outFileType):
   infile = input('Please enter a {} filename to parse: '.format(inFileType))
   outfile = input('Please enter a {} filename to output: '.format(outFileType))

   with open(infile, 'r') as stream:
       try:
           datamap = json.load(stream)
           with open(outfile, 'w') as output:
               yaml.dump(datamap, output)
       except yaml.YAMLError as exc:
           print(exc)

   print('Your file has been parsed.\n\n')

原始YAML与新YAML的示例

原件：

inputs:
  webTierCpu:
    type: integer
    minimum: 2
    default: 2
    maximum: 5
    title: Web Server CPU Count
    description: The number of CPUs for the Web nodes

新的：

inputs:
  dbTierCpu: {default: 2, description: The number of CPUs for the DB node, maximum: 5,
    minimum: 2, title: DB Server CPU Count, type: integer}

它看起来不像是在解码所有的JSON，所以我不确定下一步该怎么做……

Answer 1

您的文件正在丢失其格式，因为原始的dump例程默认以YAML flow样式写入所有叶节点，而您的输入一直是块样式。

还会丢失键的顺序，这首先是因为JSON解析器使用了dict，其次是因为dump对输出进行了排序。

如果您查看您的中间JSON，您已经看到在这一点上键顺序已经消失了。为了保持这一点，使用新的API加载YAML，并用一个特殊的JSON编码器代替dump，它可以处理加载YAML的Mapping的子类，类似于标准Python文档中的this示例。

假设您的YAML存储在input.yaml中

import sys
import json
from collections.abc import Mapping, Sequence
from collections import OrderedDict
import ruamel.yaml

# if you instantiate a YAML instance as yaml, you have to explicitly import the error
from ruamel.yaml.error import YAMLError


yaml = ruamel.yaml.YAML()  # this uses the new API
# if you have standard indentation, no need to use the following
yaml.indent(sequence=4, offset=2)

input_file = 'input.yaml'
intermediate_file = 'intermediate.json'
output_file = 'output.yaml'


class OrderlyJSONEncoder(json.JSONEncoder):
    def default(self, o):
        if isinstance(o, Mapping):
            return OrderedDict(o)
        elif isinstance(o, Sequence):
            return list(o)
        return json.JSONEncoder.default(self, o)


def yaml_2_json(in_file, out_file):
    with open(in_file, 'r') as stream:
        try:
            datamap = yaml.load(stream)
            with open(out_file, 'w') as output:
                output.write(OrderlyJSONEncoder(indent=2).encode(datamap))
        except YAMLError as exc:
            print(exc)
            return False
    return True


yaml_2_json(input_file, intermediate_file)
with open(intermediate_file) as fp:
    sys.stdout.write(fp.read())

这就给出了：

{
  "inputs": {
    "webTierCpu": {
      "type": "integer",
      "minimum": 2,
      "default": 2,
      "maximum": 5,
      "title": "Web Server CPU Count",
      "description": "The number of CPUs for the Web nodes"
    }
  }
}

您可以看到您的JSON具有适当的键顺序，我们还需要在加载时保留该顺序。通过指定将JSON对象加载到Mapping的子类中(通过提供object_pairs_hook，YAML解析器在内部使用该子类)，就可以做到这一点，而不需要对任何东西进行子类化。

from ruamel.yaml.comments import CommentedMap


def json_2_yaml(in_file, out_file):
    with open(in_file, 'r') as stream:
        try:
            datamap = json.load(stream, object_pairs_hook=CommentedMap)
            # if you need to "restore" literal style scalars, etc.
            # walk_tree(datamap)
            with open(out_file, 'w') as output:
                yaml.dump(datamap, output)
        except yaml.YAMLError as exc:
            print(exc)
            return False
    return True


json_2_yaml(intermediate_file, output_file)
with open(output_file) as fp:
    sys.stdout.write(fp.read())

以下哪项输出：

inputs:
  webTierCpu:
    type: integer
    minimum: 2
    default: 2
    maximum: 5
    title: Web Server CPU Count
    description: The number of CPUs for the Web nodes

我希望这与您最初的输入足够相似，以便可以接受。

备注：

• 在使用新的API时，我倾向于使用yaml作为ruamel.yaml.YAML()实例的名称，而不是from ruamel import yaml。由于error类不是YAML()
• If的属性，因此我建议您至少从实际功能中删除用户输入。编写parseyaml和parsejson来调用yaml_2_json resp应该很简单。原始YAML文件中的json_2_yaml.
• Any注释将会丢失，尽管ruamel.yaml可以加载它们。JSON最初确实允许注释，但它不在规范中，而且据我所知没有解析器可以输出注释。

由于您的实际文件具有文字块标量，因此您必须使用一些魔法才能将其取回。

包括以下函数，这些函数遍历树，递归为dict值和列表元素，并将任何带有嵌入换行符的行转换为一个类型，该类型作为原地文字块样式标量输出到YAML (因此没有返回值)：

from ruamel.yaml.scalarstring import PreservedScalarString, SingleQuotedScalarString
from ruamel.yaml.compat import string_types, MutableMapping, MutableSequence

def preserve_literal(s):
    return PreservedScalarString(s.replace('\r\n', '\n').replace('\r', '\n'))

def walk_tree(base):
    if isinstance(base, MutableMapping):
        for k in base:
            v = base[k]  # type: Text
            if isinstance(v, string_types):
                if '\n' in v:
                    base[k] = preserve_literal(v)
                elif '${' in v or ':' in v:
                    base[k] = SingleQuotedScalarString(v)
            else:
                walk_tree(v)
    elif isinstance(base, MutableSequence):
        for idx, elem in enumerate(base):
            if isinstance(elem, string_types):
                if '\n' in elem:
                    base[idx] = preserve_literal(elem)
                elif '${' in elem or ':' in elem:
                    base[idx] = SingleQuotedScalarString(elem)
            else:
                walk_tree(elem)

然后做

    walk_tree(datamap)

从JSON加载数据之后。

完成以上所有操作后，您的Wordpress.yaml文件中应该只有一行不同。

Answer 2

function yaml_validate {
  python -c 'import sys, yaml, json; yaml.safe_load(sys.stdin.read())'
}

function yaml2json {
  python -c 'import sys, yaml, json; print(json.dumps(yaml.safe_load(sys.stdin.read())))'
}

function yaml2json_pretty {
  python -c 'import sys, yaml, json; print(json.dumps(yaml.safe_load(sys.stdin.read()), indent=2, sort_keys=False))'
}

function json_validate {
  python -c 'import sys, yaml, json; json.loads(sys.stdin.read())'
}

function json2yaml {
  python -c 'import sys, yaml, json; print(yaml.dump(json.loads(sys.stdin.read())))'
}

在http://github.com/frgomes/bash-scripts上有更多有用的洗浴技巧