resultset.next() com.simba.spark.jdbc42驱动程序的问题

Question

我在Unix服务器中使用JDK1.8，并使用spark.jdbc42连接到Azure。

它在resultset.next()的Unix服务器中提供了以下错误消息

 Exception in thread "main" java.lang.ExceptionInInitializerError
         at com.simba.spark.jdbc42.internal.apache.arrow.memory.BaseAllocator$Config.getAllocationManagerFactory(BaseAllocator.java:776)
         at com.simba.spark.jdbc42.internal.apache.arrow.memory.ImmutableConfig.access$801(ImmutableConfig.java:24)
         at com.simba.spark.jdbc42.internal.apache.arrow.memory.ImmutableConfig$InitShim.getAllocationManagerFactory(ImmutableConfig.java:83)
         at com.simba.spark.jdbc42.internal.apache.arrow.memory.ImmutableConfig.<init>(ImmutableConfig.java:47)
         at com.simba.spark.jdbc42.internal.apache.arrow.memory.ImmutableConfig.<init>(ImmutableConfig.java:24)
         at com.simba.spark.jdbc42.internal.apache.arrow.memory.ImmutableConfig$Builder.build(ImmutableConfig.java:485)
         at com.simba.spark.jdbc42.internal.apache.arrow.memory.BaseAllocator.<clinit>(BaseAllocator.java:51)
         at com.simba.spark.spark.arrow.ArrowBuffer.<init>(Unknown Source)
         at com.simba.spark.spark.jdbc.SparkJDBC42Driver.createArrowBuffer(Unknown
 Source)
         at com.simba.spark.hivecommon.api.HS2Client.makeRowsetBuffer(Unknown
 Source)
         at com.simba.spark.hivecommon.dataengine.BackgroundFetcher.<init>(Unknown
 Source)
         at com.simba.spark.hivecommon.dataengine.HiveJDBCResultSet.moveToNextRow(Unknown
 Source)
         at com.simba.spark.jdbc.common.BaseForwardResultSet.next(Unknown Source)
 Caused by: java.lang.IllegalStateException: Arrow only runs on LittleEndian systems.
         at com.simba.spark.jdbc42.internal.io.netty.buffer.UnsafeDirectLittleEndian.<init>(UnsafeDirectLittleEndian.java:64)
         at com.simba.spark.jdbc42.internal.io.netty.buffer.UnsafeDirectLittleEndian.<init>(UnsafeDirectLittleEndian.java:50)
         at com.simba.spark.jdbc42.internal.io.netty.buffer.PooledByteBufAllocatorL.<init>(PooledByteBufAllocatorL.java:49)
         at com.simba.spark.jdbc42.internal.apache.arrow.memory.NettyAllocationManager.<clinit>(NettyAllocationManager.java:33)
         ... 14 more

我使用的是从这里借用的源代码：http://sedeks.blogspot.com/2019/05/how-to-connect-to-databricks-delta.html

在我的本地系统中，相同的源代码在JDK11.0.15中运行得很好。

Answer 1

看起来，您使用的是不支持使用Apache编码数据的非小终端系统。您可以尝试将EnableArrow=0;添加到连接字符串中，就在ssl=true;部件之后。