无法将apache-zeppelin连接到org.apache.spark.sql.DataFrame错误: value seriesCol不是highcharts的成员

Question

我在试着把齐柏林飞艇和Highchars.

%spark 
import com.knockdata.zeppelin.highcharts._ 
import com.knockdata.zeppelin.highcharts.model._ 
import sqlContext.implicits._

val Tokyo = Seq(7.0, 6.9, 9.5, 14.5, 18.2, 21.5, 25.2, 26.5, 23.3,
18.3, 13.9, 9.6).map(("Tokyo", _))

val df = (Tokyo).toDF("city", "temperature")

df.show()

highcharts(df.seriesCol("city").series("y" -> col("temperature"))).plot()

这给了我们

import com.knockdata.zeppelin.highcharts._
import com.knockdata.zeppelin.highcharts.model._
import sqlContext.implicits._
Tokyo: Seq[(String, Double)] = List((Tokyo,7.0), (Tokyo,6.9), (Tokyo,9.5), (Tokyo,14.5), (Tokyo,18.2), (Tokyo,21.5), (Tokyo,25.2), (Tokyo,26.5), (Tokyo,23.3), (Tokyo,18.3), (Tokyo,13.9), (Tokyo,9.6))
df: org.apache.spark.sql.DataFrame = [city: string, temperature: double]
+-----+-----------+
| city|temperature|
+-----+-----------+
|Tokyo|        7.0|
|Tokyo|        6.9|
|Tokyo|        9.5|
|Tokyo|       14.5|
|Tokyo|       18.2|
|Tokyo|       21.5|
|Tokyo|       25.2|
|Tokyo|       26.5|
|Tokyo|       23.3|
|Tokyo|       18.3|
|Tokyo|       13.9|
|Tokyo|        9.6|
+-----+-----------+
<console>:201: error: value seriesCol is not a member of org.apache.spark.sql.DataFrame
              highcharts(df.seriesCol("city").series("y" -> col("temperature"))).plot()

我已经在spark解释器中添加了依赖项工件作为com.knockdata:zeppelin-highcharts:0.2

使用Are there better interface to add Highcharts support to Zeppelin跟踪https://github.com/knockdata/zeppelin-highcharts/blob/master/docs/DemoLineChart.md并尝试银行数据，但获得

<console>:224: error: value series is not a member of org.apache.spark.rdd.RDD[Bank]
possible cause: maybe a semicolon is missing before `value series'?
                .series("x" -> "age", "y" -> avg(col("income")))

请帮帮我，我哪里出问题了？可能的问题是什么？提前感谢

Answer 1

DataFrame可以隐式地转换为具有函数seriesCol SeriesHolder。它是0.6.0版本中添加的。

df.seriesCol("city") 

错误应该与使用错误版本的spark-highcharts有关。示例代码(Doc)对应于版本0.6.0 (直接映射到zeppelin版本)。

使用docker可能是最简单的方法。或者使用类似于Dockerfile的方式

docker run -p 8080:8080 -d knockdata/zeppelin-highcharts

Answer 2

我将spark解释器com.knockdata:zeppelin-highcharts:0.2中的依赖项工件更改为com.knockdata:zeppelin-highcharts:0.6.0以解决此问题。但银行数据问题依然存在..对此有什么帮助吗？

%spark
import com.knockdata.zeppelin.highcharts._
import com.knockdata.zeppelin.highcharts.model._
import sqlContext.implicits._

val bankText = sc.textFile("/home/priyanka/Downloads/bank-data.csv")

case class Bank(age:Integer, region:String, income : Float, married : String, children : Integer, car:String, save_act:String, current_act : String, mortgage : String, pep : String)

// split each line, filter out header (starts with "age"), and map it into Bank case class  
val bank = bankText.map(s=>s.split(",")).filter(s=>s(0)!="age").map(
    s=>Bank(s(0).toInt, 
            s(1).replaceAll("\"", ""),
            s(2).replaceAll("\"", "").toFloat,
            s(3).replaceAll("\"", ""),
            s(4).replaceAll("\"", "").toInt,
            s(5).replaceAll("\"", ""),
            s(6).replaceAll("\"", ""),
            s(7).replaceAll("\"", ""),      
            s(8).replaceAll("\"", ""),
            s(9).replaceAll("\"", "")
        )
)

// convert to DataFrame and create temporal table
bank.toDF().registerTempTable("bank")

highcharts(bank.series("x" -> "age", "y" -> avg(col("income"))).orderBy(col("age"))).plot()

这给了我们

import com.knockdata.zeppelin.highcharts._
import com.knockdata.zeppelin.highcharts.model._
import sqlContext.implicits._
bankText: org.apache.spark.rdd.RDD[String] = MapPartitionsRDD[49] at textFile at <console>:62
defined class Bank
bank: org.apache.spark.rdd.RDD[Bank] = MapPartitionsRDD[52] at map at <console>:66
<console>:70: error: value series is not a member of org.apache.spark.rdd.RDD[Bank]
possible cause: maybe a semicolon is missing before `value series'?
                .series("x" -> "age", "y" -> avg(col("income")))
                 ^

谢谢