如何在Dataframe的列上找到比率

Question

我已经将CSV加载到数据框架中，并使用SPARK执行操作。

我有列名决定，其中有“对”或“错”值。

我想找出正确决策/总决策的比率。

Answer 1

下面的代码片段使用spark来获得正确决策的比率。

scala> val df = sc.parallelize(Seq((1,"right"),(2,"right"),(3,"right"),(4,"wrong"))).toDF("col1","col2")
df: org.apache.spark.sql.DataFrame = [col1: int, col2: string]

scala> df.registerTempTable("test_table")
warning: there was one deprecation warning; re-run with -deprecation for details

scala> spark.sql("""SELECT sum(CASE WHEN col2 = "right" THEN 1 ELSE 0 END)/count(*) as percentage FROM test_table""").show()
+----------+
|percentage|
+----------+
|      0.75|
+----------+