如何从斯诺帕克scala maven应用程序构建可执行的jar并从命令行运行

Question

我能够为spark应用程序和java应用程序构建一个jar，但同样的情况并不适用于斯诺登应用程序。我想知道，如何从斯诺帕克scala应用程序构建可执行的jar并从命令行运行。我能够构建jar，但不能执行表单命令行。

以下是我的错误

Exception in thread "main" net.snowflake.client.jdbc.SnowflakeSQLException: User Error Report: 
Java Stack Trace:
java.lang.RuntimeException: java.lang.ClassNotFoundException: us.company.snowpark.etl.HashProcessor
        at function_handler_0//com.snowflake.snowpark.internal.JavaUtils$.doDeserializeAndCloseInputStream(JavaUtils.scala:351)
        at function_handler_0//com.snowflake.snowpark.internal.JavaUtils$.deserialize(JavaUtils.scala:335)
        at function_handler_0//com.snowflake.snowpark.internal.JavaUtils.deserialize(JavaUtils.scala)
        at function_handler_0//SnowUDF.<init>(InlineCode.java:12)
Caused by: java.lang.ClassNotFoundException: us.company.snowpark.etl.HashProcessor
        at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        at java.base/java.lang.Class.forName0(Native Method)
        at java.base/java.lang.Class.forName(Class.java:398)
        at java.base/java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:745)
        at java.base/java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1965)
        at java.base/java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1851)
        at java.base/java.io.ObjectInputStream.readClass(ObjectInputStream.java:1814)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1639)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2434)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2328)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2166)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1668)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:482)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:440)
        at function_handler_0//com.snowflake.snowpark.internal.JavaUtils$.doDeserializeAndCloseInputStream(JavaUtils.scala:348)
        ... 3 more
 in function SNOWPARK_TEMP_FUNCTION_KTVPWBIRM0FSHTU with handler SnowUDF.compute
        at net.snowflake.client.jdbc.SnowflakeUtil.checkErrorAndThrowExceptionSub(SnowflakeUtil.java:127)
        at net.snowflake.client.jdbc.SnowflakeUtil.checkErrorAndThrowException(SnowflakeUtil.java:67)
        at net.snowflake.client.core.StmtUtil.pollForOutput(StmtUtil.java:442)
        at net.snowflake.client.core.StmtUtil.execute(StmtUtil.java:345)
        at net.snowflake.client.core.SFStatement.executeHelper(SFStatement.java:487)
        at net.snowflake.client.core.SFStatement.executeQueryInternal(SFStatement.java:198)
        at net.snowflake.client.core.SFStatement.executeQuery(SFStatement.java:135)
        at net.snowflake.client.core.SFStatement.execute(SFStatement.java:781)
        at net.snowflake.client.core.SFStatement.execute(SFStatement.java:677)
        at net.snowflake.client.jdbc.SnowflakeStatementV1.executeQueryInternal(SnowflakeStatementV1.java:238)
        at net.snowflake.client.jdbc.SnowflakePreparedStatementV1.executeQuery(SnowflakePreparedStatementV1.java:117)
        at com.snowflake.snowpark.internal.ServerConnection.$anonfun$runQueryGetResult$1(ServerConnection.scala:358)
        at com.snowflake.snowpark.internal.ServerConnection.withValidConnection(ServerConnection.scala:810)
        at com.snowflake.snowpark.internal.ServerConnection.runQueryGetResult(ServerConnection.scala:353)
        at com.snowflake.snowpark.internal.ServerConnection.runQuery(ServerConnection.scala:336)
        at com.snowflake.snowpark.Session.runQuery(Session.scala:781)
        at com.snowflake.snowpark.internal.UDXRegistrationHandler.createJavaUDF(UDXRegistrationHandler.scala:735)
        at com.snowflake.snowpark.internal.UDXRegistrationHandler.$anonfun$registerUDF$5(UDXRegistrationHandler.scala:117)
        at com.snowflake.snowpark.internal.UDXRegistrationHandler.retryAfterFixingClassPath(UDXRegistrationHandler.scala:54)
        at com.snowflake.snowpark.internal.UDXRegistrationHandler.$anonfun$registerUDF$4(UDXRegistrationHandler.scala:99)
        at com.snowflake.snowpark.internal.UDXRegistrationHandler.withUploadFailureCleanup(UDXRegistrationHandler.scala:169)
        at com.snowflake.snowpark.internal.UDXRegistrationHandler.registerUDF(UDXRegistrationHandler.scala:99)
        at com.snowflake.snowpark.UDFRegistration.register(UDFRegistration.scala:2368)
        at com.snowflake.snowpark.functions$.registerUdf(functions.scala:2998)
        at com.snowflake.snowpark.functions$.udf(functions.scala:3110)
        at us.company.snowpark.etl.HashProcessor.<init>(HashProcessor.scala:228)
        at us.company.snowpark.app.SnowparkAppDriver$.main(SnowparkAppDriver.scala:24)
        at us.company.snowpark.app.SnowparkAppDriver.main(SnowparkAppDriver.scala)

Answer 1

我已经在我的pom中添加了下面提到的构建，并运行了"mvn干净安装“。可执行的fat jar是在项目的目标文件夹中生成的。有关maven jar插件的更多信息可以在https://maven.apache.org/plugins/maven-jar-plugin/上找到。

​

<build>
    <sourceDirectory>src/main/scala</sourceDirectory>
    <resources>
      <resource>
        <directory>src/main/resources</directory>
      </resource>
    </resources>
    <plugins>
      <plugin>
        <groupId>net.alchim31.maven</groupId>
        <artifactId>scala-maven-plugin</artifactId>
        <version>3.2.2</version>
        <executions>
          <execution>
            <goals>
              <goal>compile</goal>
              <goal>testCompile</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <args>
            <arg>-encoding</arg>
            <arg>${project.build.sourceEncoding}</arg>
          </args>
          <checkMultipleScalaVersions>false</checkMultipleScalaVersions>
        </configuration>
      </plugin>

      <plugin>
        <artifactId>maven-shade-plugin</artifactId>
        <version>3.2.1</version>
        <executions>
          <execution>
            <id>jar-with-dependencies</id>
            <phase>package</phase>
            <goals>
              <goal>shade</goal>
            </goals>
            <configuration>
              <transformers>
                <transformer
                        implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                  <mainClass>us.company.mainClass</mainClass>
                </transformer>
              </transformers>
              <shadedArtifactAttached>true</shadedArtifactAttached>
              <shadedClassifierName>FAT</shadedClassifierName>
              <filters>
                <filter>
                  <artifact>*:*</artifact>
                  <excludes>
                    <exclude>META-INF/*.SF</exclude>
                    <exclude>META-INF/*.DSA</exclude>
                    <exclude>META-INF/*.RSA</exclude>
                  </excludes>
                </filter>
              </filters>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.7</version>
        <configuration>
          <skipTests>true</skipTests>
        </configuration>
      </plugin>
    </plugins>
  </build>

​