替换列并获取列值的ltrim

Question

我想替换数据帧中的列。需要获取此Controlling_Area = CC2 Hierarchy_Name = CC2HIDNE的scala语法代码

需要这样写: HIDENE ie:删除Hierarchy_Name中存在的Controlling_Area。

val dfPC = ReadLatest("/Full", "parquet")
.select(
LRTIM(   REPLACE(col("Hierarchy_Name"),col("Controlling_Area"),"") ),
Col(ColumnN),
Col(ColumnO)
)

notebook:3: error: not found: value REPLACE
   REPLACE(col("Hierarchy_Name"),col("Controlling_Area"),"")
   ^

期望获得LTRIM并替换scala中的代码

Answer 1

您可以使用withColumnRenamed来实现这一点：

import org.apache.spark.sql.functions    
val dfPC = ReadLatest("/Full", "parquet")
                     .withColumnRenamed("Hierarchy_Name","Controlling_Area")
                     .withColumn("Controlling_Area",ltrim(col("Controlling_Area")))