读写文件

1、从HDFS读取csv文件

 

val tagOriginDF = sparkSession.read.format("csv").option("header", "true").load("mdfs://cloudhdfs/pcgkg/user/yangren/origin/tag.csv")

//指定分隔符
val queryDF = sparkSession.read.format("csv").option("header", "true").option("seq", "\t").load("mdfs://cloudhdfs/pcgkg/user/yangren/origin/tag.csv")

 

2、将结果写入HDFS

     resultDF.repartition(1).write.option("header", "true").mode(SaveMode.Overwrite).csv("mdfs://cloudhdfs/pcgkg/user/yangren/result")

标签：文件,option,pcgkg,yangren,读写,header,mdfs,csv
来源： https://www.cnblogs.com/renyang/p/16487722.html