如何优雅地序列化和反序列化OpenCV YAML校准数据？

Question

我试图使用正式的OpenCV绑定以YAML格式加载/保存OpenCV校准数据。我知道OpenCV (至少c++版本)可以序列化为XML和JSON，但我希望支持较旧的YAML校准文件。

校准文件如下所示：

%YAML:1.0
cameraMatrix: !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [ 6.6278599887122368e+02, 0., 3.1244256016006659e+02, 0.,
       6.6129276875199082e+02, 2.2747179767124251e+02, 0., 0., 1. ]
imageSize_width: 640
imageSize_height: 480
sensorSize_width: 0
sensorSize_height: 0
distCoeffs: !!opencv-matrix
   rows: 5
   cols: 1
   dt: d
   data: [ -1.8848338341464690e-01, 1.0721890419183855e+00,
       -3.5244467228016116e-03, -7.0195032848241403e-04,
       -2.0412827999027101e+00 ]
reprojectionError: 2.1723265945911407e-01

我已经看了几个这里和这里的答案，但是我正在寻找一个优雅的解决方案，因为我还没有完全理解如何最好地将java类映射到YAML和返回。我尝试过一些库，如jyaml、yamlbean(来自SourceForge的1.0和通过Maven Central的1.13 )和SnakeYAML。

我目前试图对某些作品进行反序列化，但感觉相当乏味：

CalibrationParseTest.java

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Paths;

import org.opencv.core.Core;
import org.yaml.snakeyaml.Yaml;
import org.yaml.snakeyaml.constructor.Constructor;

public class CalibrationParseTest {

    public static void main(String[] args) {
        // load OpenCV native
        System.loadLibrary(Core.NATIVE_LIBRARY_NAME);

        String yamlPath = "./data/calibration.yml";

        try{

          String yamlString = new String(Files.readAllBytes(Paths.get(yamlPath)), StandardCharsets.UTF_8);
          // remove %YAML:1.0 to avoid scan directive error
          yamlString = yamlString.replaceAll("%YAML:1.0", "");
          // map custom class
          yamlString = yamlString.replaceAll("opencv-matrix", "MatYAML");

          System.out.println("<loaded>");
          System.out.println(yamlString);
          System.out.println("</loaded>");

          Yaml yaml = new Yaml(new Constructor(CalibrationData.class));
          CalibrationData data = yaml.load(yamlString);
          // currently manually parsing data from the HashMap: can this be better ?
          data.populateCV();
          // double check data
          System.out.println("<deserialized>");
          System.out.println(data);
          System.out.println("</deserialized>");

        }catch (IOException e) { 
          e.printStackTrace(); 
        } 
    }

}

CalibrationData.java

import java.util.HashMap;

import org.opencv.core.Mat;
import org.opencv.core.Size;

public class CalibrationData extends HashMap{

    public Mat cameraMatrix;
    public Size imageSize;
    public Size sensorSize;
    public Mat distCoeffs;
    public float reprojectionError;

    public CalibrationData(){}

    public void populateCV(){
        cameraMatrix      = ((MatYAML)get("cameraMatrix")).toMat();
        imageSize         = new Size((int)get("imageSize_width"),(int)get("imageSize_height"));
        sensorSize        = new Size((int)get("sensorSize_width"),(int)get("sensorSize_height"));
        distCoeffs        = ((MatYAML)get("distCoeffs")).toMat();
        reprojectionError = (float)((double)get("reprojectionError"));
    }

    public String toString(){
        if(cameraMatrix == null){
            return String.format("[CalibrationData (not parsed to CV-> call populateCV()\n\tdata: %s\n]",super.toString());
        }
        return String.format("[CalibrationData\n" + 
                             "\tcalibrationMatrix: %s\n" + 
                             "\timageSize: %s\n" + 
                             "\tsensorSize: %s\n" + 
                             "\tdistCoeffs: %s\n" + 
                             "\treprojectionError: %f\n]", cameraMatrix.dump(), imageSize.toString(), sensorSize.toString(), distCoeffs.dump(), reprojectionError);
    }

}

MatYAML.java

import java.util.List;

import org.opencv.core.CvType;
import org.opencv.core.Mat;

public class MatYAML{

    public int rows;
    public int cols;
    public String dt;
    public List<Double> data;

    Mat toMat(){
        Mat out = new Mat(rows, cols, dt.equals("d") ? CvType.CV_64F : CvType.CV_32F);

        int index = 0;

        for (int row = 0; row < rows; row++) {
            for (int col = 0; col < cols; col++) {
                out.put(row, col, data.get(index++));
            }
        }

        return out;
    }

}

这将产生预期结果：

<loaded>

cameraMatrix: !!MatYAML
   rows: 3
   cols: 3
   dt: d
   data: [ 6.6278599887122368e+02, 0., 3.1244256016006659e+02, 0.,
       6.6129276875199082e+02, 2.2747179767124251e+02, 0., 0., 1. ]
imageSize_width: 640
imageSize_height: 480
sensorSize_width: 0
sensorSize_height: 0
distCoeffs: !!MatYAML
   rows: 5
   cols: 1
   dt: d
   data: [ -1.8848338341464690e-01, 1.0721890419183855e+00,
       -3.5244467228016116e-03, -7.0195032848241403e-04,
       -2.0412827999027101e+00 ]
reprojectionError: 2.1723265945911407e-01

</loaded>
<deserialized>
[CalibrationData
    calibrationMatrix: [662.7859988712237, 0, 312.4425601600666;
  0, 661.2927687519908, 227.4717976712425;
  0, 0, 1]
    imageSize: 640x480
    sensorSize: 0x0
    distCoeffs: [-0.1884833834146469; 1.072189041918385; -0.003524446722801612; -0.000701950328482414; -2.04128279990271]
    reprojectionError: 0.217233
]
</deserialized>

在没有这些黑客的情况下，Java类和OpenCV之间是否有一种更优雅的序列化/反序列化方法？

我所说的黑客是指：

• 手动删除yaml版本指令
• 用MatYAML字符串交换opencv-矩阵
• 手工铸造HashMap值
• 是否可能避免手动填充OpenCV Mat数据？(如果可能的话)

Update 2 amanin的答案更简洁，可以避免恶意替换“！！opencv-矩阵”，但是它没有序列化/反序列化Mat。

OpenCVConfig{imageSize_width=640, imageSize_height=480, sensorSize_width=0, sensorSize_height=0, camerMatrix=Matrix{rows=3, cols=3, dt=d, data=[662.7859988712237, 0.0, 312.4425601600666, 0.0, 661.2927687519908, 227.4717976712425, 0.0, 0.0, 1.0]}, distCoeffs=Matrix{rows=5, cols=1, dt=d, data=[-0.1884833834146469, 1.0721890419183855, -0.0035244467228016116, -7.01950328482414E-4, -2.04128279990271]}}
---
imageSize_width: 640
imageSize_height: 480
sensorSize_width: 0
sensorSize_height: 0
reprojectionError: 0.21723265945911407
cameraMatrix:
  rows: 3
  cols: 3
  dt: "d"
  data:
 - 662.7859988712237
 - 0.0
 - 312.4425601600666
 - 0.0
 - 661.2927687519908
 - 227.4717976712425
 - 0.0
 - 0.0
 - 1.0
distCoeffs:
  rows: 5
  cols: 1
  dt: "d"
  data:
 - -0.1884833834146469
 - 1.0721890419183855
 - -0.0035244467228016116
 - -7.01950328482414E-4
 - -2.04128279990271

请建议如何将解决方案与org.opencv.core.Mat集成

Answer 1

你看过杰克逊图书馆了吗？它允许将JSON/Yaml内容映射到Java。

我做了一个小例子，解决了你的两个问题：

• 用MatYAML字符串交换opencv-矩阵
• 手工铸造HashMap值

然而，对于yaml版本指令，因为它看起来是无效的Yaml，我不知道如何处理它。在我的例子中，我已经手动删除了它。当然，可以找到更好的解决办法，但我不知道。

EDIT2：对于矩阵对象，我做了一个哑POJO，杰克逊在内部使用它来读取原始的YAML。然后，我添加了一个转换层(请参阅OpenCVConfig类上的@JsonSerialize和@)，将这个简单的POJO转换为专门的OpenCV矩阵。Jackson提供了各种技术(流、自定义转换器/反序列化器、指导注释等)。映射，因此您可以探索它的功能，以找到最适合您需要的解决方案。

因此，要使示例工作，您需要两个依赖项(在这里以maven格式提供)：

        <!-- (De)serialization engine -->
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>2.10.0</version>
        </dependency>

        <!-- Yaml support -->
        <dependency>
            <groupId>com.fasterxml.jackson.dataformat</groupId>
            <artifactId>jackson-dataformat-yaml</artifactId>
            <version>2.10.0</version>
        </dependency>

下面是maven存储库描述页面：

Jackson数据库：通过maven搜索或通过Via存储库

杰克逊-数据格式-yaml：通过maven搜索还是通过Via存储库

注意:我的示例包含了很多样板，因为我手动(嗯，IDE)生成了getter/setters。到目前为止，您应该将代码数量减少到以下任一项：

• 使用Lombok lib
• 用Kotlin编码(数据类)
• 使用不带getter/setter的公共属性
• 使用java 14记录(不确定此时它是否有效)

package fr.amanin.stackoverflow;

import java.util.Arrays;
import java.util.stream.Stream;

import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.annotation.JsonSerialize;
import com.fasterxml.jackson.databind.type.TypeFactory;
import com.fasterxml.jackson.databind.util.Converter;
import com.fasterxml.jackson.dataformat.yaml.YAMLGenerator;
import com.fasterxml.jackson.dataformat.yaml.YAMLMapper;
import org.opencv.core.CvType;
import org.opencv.core.Mat;

public class YAMLOpenCV {
    /**
     * Engine in charge of YAML decoding.
     */
    public static final ObjectMapper OPENCV_YAML_MAPPER = new YAMLMapper();

    public static void main(String[] args) throws Exception {
        nu.pattern.OpenCV.loadShared();
        System.loadLibrary(org.opencv.core.Core.NATIVE_LIBRARY_NAME);
        final String confStr =
                "cameraMatrix: !!opencv-matrix\n" +
                        "   rows: 3\n" +
                        "   cols: 3\n" +
                        "   dt: d\n" +
                        "   data: [ 6.6278599887122368e+02, 0., 3.1244256016006659e+02, 0.,\n" +
                        "       6.6129276875199082e+02, 2.2747179767124251e+02, 0., 0., 1. ]\n" +
                        "imageSize_width: 640\n" +
                        "imageSize_height: 480\n" +
                        "sensorSize_width: 0\n" +
                        "sensorSize_height: 0\n" +
                        "distCoeffs: !!opencv-matrix\n" +
                        "   rows: 5\n" +
                        "   cols: 1\n" +
                        "   dt: d\n" +
                        "   data: [ -1.8848338341464690e-01, 1.0721890419183855e+00,\n" +
                        "       -3.5244467228016116e-03, -7.0195032848241403e-04,\n" +
                        "       -2.0412827999027101e+00 ]\n" +
                        "reprojectionError: 2.1723265945911407e-01";

        OpenCVConfig conf = OPENCV_YAML_MAPPER.readValue(confStr, OpenCVConfig.class);
        System.out.println(conf);

        String serialized = OPENCV_YAML_MAPPER.writeValueAsString(conf);
        System.out.println(serialized);
    }

    /**
     * Java model mirroring YAML configuration. Jackson will fill it 
     * with values read from YAML configuration file, matching YAML 
     * fields with this class property names.
     */
    public static class OpenCVConfig {
        int imageSize_width;
        int imageSize_height;
        int sensorSize_width;
        int sensorSize_height;

        double reprojectionError;

        /* Special case: Matrix objects are decoded in two passes:
         * 1. Jackson will check below converters, and use their input 
         * type to decode YAML to `Matrix` object (intermediate step). 
         * 2. Jackson uses converter to delegate to user the mapping 
         * from this intermediate POJO to specialized target (`Mat` here) 
         */
        @JsonDeserialize(converter = ToMatConverter.class)
        @JsonSerialize(converter = FromMatConverter.class)
        Mat cameraMatrix;
        @JsonDeserialize(converter = ToMatConverter.class)
        @JsonSerialize(converter = FromMatConverter.class)
        Mat distCoeffs;

        public int getImageSize_width() {
            return imageSize_width;
        }

        public OpenCVConfig setImageSize_width(int imageSize_width) {
            this.imageSize_width = imageSize_width;
            return this;
        }

        public int getImageSize_height() {
            return imageSize_height;
        }

        public OpenCVConfig setImageSize_height(int imageSize_height) {
            this.imageSize_height = imageSize_height;
            return this;
        }

        public int getSensorSize_width() {
            return sensorSize_width;
        }

        public OpenCVConfig setSensorSize_width(int sensorSize_width) {
            this.sensorSize_width = sensorSize_width;
            return this;
        }

        public int getSensorSize_height() {
            return sensorSize_height;
        }

        public OpenCVConfig setSensorSize_height(int sensorSize_height) {
            this.sensorSize_height = sensorSize_height;
            return this;
        }

        public double getReprojectionError() {
            return reprojectionError;
        }

        public OpenCVConfig setReprojectionError(double reprojectionError) {
            this.reprojectionError = reprojectionError;
            return this;
        }

        public Mat getCameraMatrix() {
            return cameraMatrix;
        }

        public OpenCVConfig setCameraMatrix(Mat cameraMatrix) {
            this.cameraMatrix = cameraMatrix;
            return this;
        }

        public Mat getDistCoeffs() {
            return distCoeffs;
        }

        public OpenCVConfig setDistCoeffs(Mat distCoeffs) {
            this.distCoeffs = distCoeffs;
            return this;
        }

        @Override
        public String toString() {
            return "OpenCVConfig{" +
                    "imageSize_width=" + imageSize_width +
                    ", imageSize_height=" + imageSize_height +
                    ", sensorSize_width=" + sensorSize_width +
                    ", sensorSize_height=" + sensorSize_height +
                    ", camerMatrix=" + cameraMatrix +
                    ", distCoeffs=" + distCoeffs +
                    '}';
        }
    }

    /**
     * Converter used for serialization of Mat objects into YAML.
     */
    private static class FromMatConverter implements Converter<Mat, Matrix> {

        @Override
        public Matrix convert(Mat value) {
            final Matrix result = new Matrix();
            result.cols = value.cols();
            result.rows = value.rows();
            final int type = value.type();
            result.dt = Stream.of(MatrixDataType.values())
                    .filter(dt -> dt.mapping == type)
                    .findAny()
                    .orElseThrow(() -> new UnsupportedOperationException("No matching datatype found for "+type));
            int idx = 0;
            result.data = new double[result.rows * result.cols];
            for (int r = 0 ; r < result.rows ; r++) {
                for (int c = 0; c < result.cols; c++) {
                    final double[] v = value.get(r, c);
                    result.data[idx++] = v[0];
                }
            }
            return result;
        }

        @Override
        public JavaType getInputType(TypeFactory typeFactory) {
            return typeFactory.constructType(new TypeReference<Mat>() {});
        }

        @Override
        public JavaType getOutputType(TypeFactory typeFactory) {
            return typeFactory.constructType(new TypeReference<Matrix>() {});
        }
    }

    /**
     * Converter used at read time, to map YAML object to OpenCV Mat.
     */
    private static class ToMatConverter implements Converter<Matrix, Mat> {

        @Override
        public Mat convert(Matrix in) {
            final Mat result = new Mat(in.rows, in.cols, in.dt.mapping);

            int idx = 0;
            for (int r = 0 ; r < in.rows ; r++) {
                for (int c = 0; c < in.cols; c++) {
                    result.put(r, c, in.data[idx++]);
                }
            }

            return result;
        }

        @Override
        public JavaType getInputType(TypeFactory typeFactory) {
            return typeFactory.constructType(new TypeReference<Matrix>() {});
        }

        @Override
        public JavaType getOutputType(TypeFactory typeFactory) {
            return typeFactory.constructType(new TypeReference<Mat>() {});
        }
    }

    public static class Matrix {
        int rows;
        int cols;
        MatrixDataType dt;
        double[] data;

        public int getRows() {
            return rows;
        }

        public Matrix setRows(int rows) {
            this.rows = rows;
            return this;
        }

        public int getCols() {
            return cols;
        }

        public Matrix setCols(int cols) {
            this.cols = cols;
            return this;
        }

        public MatrixDataType getDt() {
            return dt;
        }

        public Matrix setDt(MatrixDataType dt) {
            this.dt = dt;
            return this;
        }

        public double[] getData() {
            return data;
        }

        public Matrix setData(double[] data) {
            this.data = data;
            return this;
        }

        double at(int x, int y) {
            if (x >= cols || y >= rows) throw new IllegalArgumentException("Bad coordinate");
            return data[y*rows + x];
        }

        @Override
        public String toString() {
            return "Matrix{" +
                    "rows=" + rows +
                    ", cols=" + cols +
                    ", dt=" + dt +
                    ", data=" + Arrays.toString(data) +
                    '}';
        }
    }
/*
    public static class MatDeserializer extends StdDeserializer<Mat> {

        protected MatDeserializer() {
            super(Mat.class);
        }

        @Override
        public Mat deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JsonProcessingException {
            final int rows, cols;
            final MatrixDataType dtype;
            final double[] data;
        }
    }

    public static class MatSerializer extends StdSerializer<Mat> {

        protected MatSerializer() {
            super(Mat.class);
        }

        @Override
        public void serialize(Mat value, JsonGenerator gen, SerializerProvider provider) throws IOException {
            gen.writeNumberField("rows", value.rows());
            gen.writeNumberField("cols", value.cols());
            gen.writeFieldName("data");
            gen.writeStartArray();
            gen.writeEndArray();
        }
    }
*/
    public enum MatrixDataType {
        d(CvType.CV_64F),
        f(CvType.CV_32F);

        public final int mapping;
        MatrixDataType(int mapping) {
            this.mapping = mapping;
        }
    }
}

希望能帮上忙

编辑：

对不起，但我还没有找到可靠的方法：

• 选择数组样式。它似乎是关于杰克森·吉特布的公开问题
• 至于YAML标签，我还没有找到任何线索。如果你决定坚持下去的话，也许你将不得不填补杰克逊的一个问题。在寻找一种编码方法之后

编辑2: --我修改了上面的代码示例来序列化/反序列化Mat对象。然而，我像您一样使用循环来填充/获取矩阵值，因为我无法找到通过ByteBuffers传递值的方法(相信我，我试过了。但是Java和文档在这方面都没有很大的帮助)。

您可以看到引入的矩阵对象仍然存在，因为它简化了IMHO的转换工作。如果你想要的话，如果你使用杰克逊StdSerializer对象而不是转换器，你可以摆脱它。然而，我不确定它是否会更干净。

最后一句话：

• 我在搜索数据传输时找到了另一篇文章是关于OpenCV矩阵的。也许它会对你有用。
• 请注意，我的代码仅正确管理64位浮点值。如果需要，您必须从这里开始添加其他CvType案例。

嗯，这次肯定，我不能再帮你了。祝你好运-)