将XML消息从Zerocode发布到Kafka主题

Question

我正在尝试使用zerocode进行流媒体平台( Kafka / spark )的集成测试

零代码(发布XML消息)-->使用Apache AVRO序列化--> Kafka topic --> spark job (processing) --> Kafka topic -->使用AVRO反序列化-->零代码消费者测试(断言消息内容)

当我试图将消息放在零代码的Kafka主题上时，零代码测试用例在XML消息的开头引入了特殊的字符，这导致在spark中解析XML消息时出现错误。

当我使用字符串序列化程序和"hello world“时，它可以工作。

我需要让它与Apache Avro和XML消息一起工作。

{
  "scenario Name": "Produce a message to Kafka topic - vanilla",
  "steps": [
    {
      "name": "produce_step",
      "URL": "Kafka-topic: my-topic ",
      "operation": "produce",
      "request": {
        "record Type" : "J SON",
        "records": [
          {
            "key":"123456789",
            "value": "Schema Test"
          }
        ]
      },
      "assertions": {
        "record Metadata" : "$NOT.NULL",
        "status" : "OK"
      }
    }
  ]
}

当零代码测试用例在Kafka主题上放消息时，不应该在消息的开头引入任何特殊的字符。

Answer 1

您只需要将“记录类型”更新为RAW，而不是JSON。它应该像预期的那样工作。

"record Type" : "RAW"

例如：

您希望向Kafka主题生成以下XML消息，然后使用Kafka主题中的XML并验证XML是否相同。

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
    <ConversionRate xmlns="http://www.webserviceX.NET/">
      <FromCurrency>AFA</FromCurrency>
      <ToCurrency>GBP</ToCurrency>
    </ConversionRate>
  </soap:Body>
</soap:Envelope>

下面是完整的测试用例供您参考。它有“生产”和“消费”两个步骤。

{
    "scenarioName": "Produce and Consume XML as RAW message",
    "steps": [
        {
            "name": "load_kafka",
            "url": "kafka-topic:demo-raw1",
            "operation": "load",
            "request": {
                "record Type" : "RAW",
                "records": [
                    {
                        "key": "${RANDOM.NUMBER}",
                        "value": "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<soap:Envelope xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns:soap=\"http://schemas.xmlsoap.org/soap/envelope/\">\n  <soap:Body>\n    <ConversionRate xmlns=\"http://www.webserviceX.NET/\">\n      <FromCurrency>AFA</FromCurrency>\n      <ToCurrency>GBP</ToCurrency>\n    </ConversionRate>\n  </soap:Body>\n</soap:Envelope>"
                    }
                ]
            },
            "assertions": {
                "status": "Ok"
            }
        },
        {
            "name": "onload_kafka",
            "url": "kafka-topic:demo-raw1",
            "operation": "unload",
            "request": {
                "consumerLocalConfigs": {
                    "recordType": "RAW",
                    "commitSync": true,
                    "showRecordsConsumed": true,
                    "maxNoOfRetryPollsOrTimeouts": 3
                }
            },
            "assertions": {
                "size": 1,
                "records": [
                    {
                        "key" : "$NOT.NULL",
                        "value": "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<soap:Envelope xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns:soap=\"http://schemas.xmlsoap.org/soap/envelope/\">\n  <soap:Body>\n    <ConversionRate xmlns=\"http://www.webserviceX.NET/\">\n      <FromCurrency>AFA</FromCurrency>\n      <ToCurrency>GBP</ToCurrency>\n    </ConversionRate>\n  </soap:Body>\n</soap:Envelope>"
                    }
                ]
            }
        }
    ]
}

Zerocode在使用记录后测试运行下面的控制台日志，即使用的XML与生成的XML完全相同。

org.jsmart.zerocode.core.kafka.receive.KafkaReceiver - Consumer chosen recordType: RAW
2020-05-19 12:32:28,986 [main] INFO  
org.jsmart.zerocode.core.kafka.helper.KafkaConsumerHelper - 
Record Key - 6784799632078391362 , 
Record value - <?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
    <ConversionRate xmlns="http://www.webserviceX.NET/">
      <FromCurrency>AFA</FromCurrency>
      <ToCurrency>GBP</ToCurrency>
    </ConversionRate>
  </soap:Body>
</soap:Envelope>, 

Record partition - 0, Record offset - 1

您可以在下面看到完整的日志：

2020-05-19 12:32:25,358 [main] INFO  org.jsmart.zerocode.core.runner.ZeroCodeMultiStepsScenarioRunnerImpl - 
-------------------------- BDD: Scenario:consume as RAW message -------------------------

2020-05-19 12:32:25,461 [main] INFO  org.jsmart.zerocode.core.kafka.helper.KafkaConsumerHelper - 
---------------------------------------------------------
kafka.bootstrap.servers - localhost:9092
---------------------------------------------------------
returned default type 'RAW'.
2020-05-19 12:32:25,677 [main] INFO  org.jsmart.zerocode.core.kafka.send.KafkaSender - Sending record number: 0

2020-05-19 12:32:25,678 [main] INFO  org.jsmart.zerocode.core.kafka.send.KafkaSender - Synchronous Producer sending record - ProducerRecord(topic=demo-raw1, partition=null, headers=RecordHeaders(headers = [], isReadOnly = false), key=6784799632078391362, value=<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
    <ConversionRate xmlns="http://www.webserviceX.NET/">
      <FromCurrency>AFA</FromCurrency>
      <ToCurrency>GBP</ToCurrency>
    </ConversionRate>
  </soap:Body>
</soap:Envelope>, timestamp=null)


2020-05-19 12:32:25,873 [main] INFO  org.jsmart.zerocode.core.engine.validators.ZeroCodeValidatorImpl - Comparing results via LENIENT matchers
2020-05-19 12:32:25,878 [main] INFO  org.jsmart.zerocode.core.runner.StepNotificationHandler - 
***Step PASSED - Scenario:consume as RAW message -> load_kafka
2020-05-19 12:32:25,882 [main] INFO  org.jsmart.zerocode.core.runner.ZeroCodeMultiStepsScenarioRunnerImpl - 
--------- TEST-STEP-CORRELATION-ID: a8ebe4c6-c1fb-434c-9c00-b2cbdf2f7fe6 ---------
*requestTimeStamp:2020-05-19T12:32:25.455
step:load_kafka
url:kafka-topic:demo-raw1
method:load
request:
{
  "record Type" : "RAW",
  "records" : [ {
    "key" : "6784799632078391362",
    "value" : "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<soap:Envelope xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns:soap=\"http://schemas.xmlsoap.org/soap/envelope/\">\n  <soap:Body>\n    <ConversionRate xmlns=\"http://www.webserviceX.NET/\">\n      <FromCurrency>AFA</FromCurrency>\n      <ToCurrency>GBP</ToCurrency>\n    </ConversionRate>\n  </soap:Body>\n</soap:Envelope>"
  } ]
} 
--------- TEST-STEP-CORRELATION-ID: a8ebe4c6-c1fb-434c-9c00-b2cbdf2f7fe6 ---------
Response:
{
  "status" : "Ok",
  "recordMetadata" : {
    "offset" : 1,
    "timestamp" : 1589887945831,
    "serializedKeySize" : 19,
    "serializedValueSize" : 407,
    "topicPartition" : {
      "hash" : 749906548,
      "partition" : 0,
      "topic" : "demo-raw1"
    }
  }
}
*responseTimeStamp:2020-05-19T12:32:25.872 
*Response delay:417.0 milli-secs 
---------> Expected Response: <----------
{
  "status" : "Ok"
} 

-done-

2020-05-19 12:32:25,884 [main] INFO  org.jsmart.zerocode.core.kafka.helper.KafkaConsumerHelper - 
---------------------------------------------------------
kafka.bootstrap.servers - localhost:9092
---------------------------------------------------------

***Step PASSED - Scenario:consume as RAW message -> onload_kafka
2020-05-19 12:32:33,025 [main] INFO  org.jsmart.zerocode.core.runner.ZeroCodeMultiStepsScenarioRunnerImpl - 
--------- TEST-STEP-CORRELATION-ID: 53092b7e-ae92-4631-ad79-0602280650b9 ---------
*requestTimeStamp:2020-05-19T12:32:25.884
step:onload_kafka
url:kafka-topic:demo-raw1
method:unload
request:
{
  "consumerLocalConfigs" : {
    "recordType" : "RAW",
    "commitSync" : true,
    "showRecordsConsumed" : true,
    "maxNoOfRetryPollsOrTimeouts" : 3
  }
} 
--------- TEST-STEP-CORRELATION-ID: 53092b7e-ae92-4631-ad79-0602280650b9 ---------
Response:
{
  "records" : [ {
    "topic" : "demo-raw1",
    "partition" : 0,
    "offset" : 1,
    "timestamp" : 1589887945831,
    "timestampType" : "CREATE_TIME",
    "serializedKeySize" : 19,
    "serializedValueSize" : 407,
    "key" : "6784799632078391362",
    "value" : "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<soap:Envelope xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns:soap=\"http://schemas.xmlsoap.org/soap/envelope/\">\n  <soap:Body>\n    <ConversionRate xmlns=\"http://www.webserviceX.NET/\">\n      <FromCurrency>AFA</FromCurrency>\n      <ToCurrency>GBP</ToCurrency>\n    </ConversionRate>\n  </soap:Body>\n</soap:Envelope>",
    "leaderEpoch" : {
      "value" : 0
    }
  } ],
  "size" : 1
}
*responseTimeStamp:2020-05-19T12:32:33.020 
*Response delay:7136.0 milli-secs 
---------> Expected Response: <----------
{
  "size" : 1,
  "records" : [ {
    "key" : "$NOT.NULL",
    "value" : "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<soap:Envelope xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns:soap=\"http://schemas.xmlsoap.org/soap/envelope/\">\n  <soap:Body>\n    <ConversionRate xmlns=\"http://www.webserviceX.NET/\">\n      <FromCurrency>AFA</FromCurrency>\n      <ToCurrency>GBP</ToCurrency>\n    </ConversionRate>\n  </soap:Body>\n</soap:Envelope>"
  } ]
} 

-done-


2020-05-19 12:32:33,233 [main] INFO  org.jsmart.zerocode.core.domain.builders.ExtentReportsFactory - Where were the tests fired? Ans: OS:Mac OS X, Architecture:x86_64, Java:1.8.0_91, Vendor:Oracle Corporation

Process finished with exit code 0

这是Kafka Produce Consume XML messages的维基和帮助文档