我试图在Maven上使用鳞状和火花试验底座进行集成测试。Spark作业读取CSV文件,验证结果,并将数据插入数据库。我试图通过输入已知格式的文件来测试验证,并查看它们是否失败以及如何失败。这个特定的测试只是确保验证通过。不幸的是,最严重的人找不到我的测试。
相关pom插件:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>
<!-- enable scalatest -->
<plugin>
<groupId>org.scalatest</groupId>
<artifactId>scalatest-maven-plugin</artifactId>
<version>1.0</version>
<configuration>
<reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
<wildcardSuites>com.cainc.data.etl.schema.proficiency</wildcardSuites>
</configuration>
<executions>
<execution>
<id>test</id>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>这是考试课:
class ProficiencySchemaITest extends FlatSpec with Matchers with SharedSparkContext with BeforeAndAfter {
private var schemaStrategy: SchemaStrategy = _
private var dataReader: DataFrameReader = _
before {
val sqlContext = new SQLContext(sc)
import sqlContext._
import sqlContext.implicits._
val dataInReader = sqlContext.read.format("com.databricks.spark.csv")
.option("header", "true")
.option("nullValue", "")
schemaStrategy = SchemaStrategyChooser("dim_state_test_proficiency")
dataReader = schemaStrategy.applySchema(dataInReader)
}
"Proficiency Validation" should "pass with the CSV file proficiency-valid.csv" in {
val dataIn = dataReader.load("src/test/resources/proficiency-valid.csv")
val valid: Try[DataFrame] = Try(schemaStrategy.validateCsv(dataIn))
valid match {
case Success(v) => ()
case Failure(e) => fail("Validation failed on what should have been a clean file: ", e)
}
}
}当我运行mvn test时,它找不到任何测试并输出以下消息:
[INFO] --- scalatest-maven-plugin:1.0:test (test) @ load-csv-into-db ---
[36mDiscovery starting.[0m
[36mDiscovery completed in 54 milliseconds.[0m
[36mRun starting. Expected test count is: 0[0m
[32mDiscoverySuite:[0m
[36mRun completed in 133 milliseconds.[0m
[36mTotal number of tests run: 0[0m
[36mSuites: completed 1, aborted 0[0m
[36mTests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0[0m
[33mNo tests were executed.[0m更新
通过使用:
<suites>com.cainc.data.etl.schema.proficiency.ProficiencySchemaITest</suites>而不是:
<wildcardSuites>com.cainc.data.etl.schema.proficiency</wildcardSuites>我可以让那一项测试运行。显然,这并不理想。wildcardSuites可能坏了,我要在GitHub上开一张票,看看会发生什么。
发布于 2017-02-16 08:46:28
这可能是因为项目路径中有一些空格字符。删除项目路径中的空间,可以成功地发现测试。希望能帮上忙。
发布于 2017-02-22 17:38:01
尝试将junit排除为传递依赖关系。为我工作。下面的示例,但是请注意Scala和Spark版本是特定于我的环境的。
<dependency>
<groupId>com.holdenkarau</groupId>
<artifactId>spark-testing-base_2.10</artifactId>
<version>1.5.0_0.6.0</version>
<scope>test</scope>
<exclusions>
<!-- junit is not compatible with scalatest -->
<exclusion>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</exclusion>
</exclusion>
</dependency>发布于 2018-06-16 22:28:57
对我来说,这是因为我没有使用以下插件:
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<configuration>
<scalaVersion>${scala.version}</scalaVersion>
<args>
<arg>-target:jvm-1.8</arg>
</args>
</configuration>
</plugin>
https://stackoverflow.com/questions/38700319
复制相似问题