Azure数据流不删除AlterRow中的行

Question

我不知道是什么错误的数据流没有删除它导致此错误的行“Activity dataflow38 failed：”

在预览选项卡中，它显示了我要删除的行，但它与表没有任何关系。

​

​

​

​

误差

 "message": "Job 'c688a5bd-34dd-44e2-8292-724f0ea5f98a failed due to reason: DF-EXEC-1 Conversion failed when converting date and/or time from character string.\ncom.microsoft.sqlserver.jdbc.SQLServerException: Conversion failed when converting date and/or time from character string.\n\tat com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:258)\n\tat com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:256)\n\tat com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:108)\n\tat com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:28)\n\tat com.microsoft.sqlserver.jdbc.SQLServerBulkCopy.doInsertBulk(SQLServerBulkCopy.java:1611)\n\tat com.microsoft.sqlserver.jdbc.SQLServerBulkCopy.access$200(SQLServerBulkCopy.java:58)\n\tat com.microsoft.sqlserver.jdbc.SQLServerBulkCopy$1InsertBulk.doExecute(SQLServerBulkCopy.java:709)\n\tat com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7151)\n\tat com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2478)\n\tat com.microsoft.sqlserver.jdbc.SQLServerBulkCopy.sendBulkLoadBCP(SQLServerBulkCopy.java:739)\n\tat com.microsoft.sqlserver.jdbc.SQLServerBulkCopy.writeToServer(SQLServerBulkCopy.java:1684)\n\tat com.microsoft.sqlserver.jdbc.SQLServerBulkCopy.writeToServer(SQLServerBulkCopy.java:669)\n\tat com.microsoft.azure.sqldb.spark.connect.DataFrameFunctions.com$microsoft$azure$sqldb$spark$connect$DataFrameFunctions$$bulkCopy(DataFrameFunctions.scala:127)\n\tat com.microsoft.azure.sqldb.spark.connect.DataFrameFunctions$$anonfun$bulkCopyToSqlDB$1.apply(DataFrameFunctions.scala:72)\n\tat com.microsoft.azure.sqldb.spark.connect.DataFrameFunctions$$anonfun$bulkCopyToSqlDB$1.apply(DataFrameFunctions.scala:72)\n\tat org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:951)\n\tat org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:951)\n\tat org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2284)\n\tat org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2284)\n\tat org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)\n\tat org.apache.spark.scheduler.Task.doRunTask(Task.scala:139)\n\tat org.apache.spark.scheduler.Task.run(Task.scala:112)\n\tat org.apache.spark.executor.Executor$TaskRunner$$anonfun$13.apply(Executor.scala:497)\n\tat org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1526)\n\tat org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:503)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat java.lang.Thread.run(Thread.java:748)\n",
    "failureType": "UserError",
    "target": "dataflow38"
}

Answer 1

此Alter策略将删除所有行，因为您在表达式中使用true()。你确定这是你想要的吗？

此错误可能来自您的Sink字段映射。如果您只想删除行，则不要设置任何接收器映射。

只需映射你的键列。并确保数据类型与您的键映射匹配。如果它没有将其转换到派生列中。

为了使这更好，我不认为我们应该默认的自动映射，如果你所做的只是删除。