Sood Sood - 1 year ago 139
Scala Question

Convert a RDD[(Long,Long)] to RDD[Row]

How do I convert a RDD[(Long, Long)] to RDD[Row]?

I require the RDD[Row] to convert to a DataFrame. The closest I got to an answer is
How to convert RDD[Row] to RDD[Vector] but I want to do the opposite.

Answer Source

Just apply schema to your RDD (ref: spark-sql-programming-guide)

case class MyObjectType(col1:Long, col2:Long)

val myRDD:RDD[Long, Long] = .........
val myDF =>MyObjectType(r._1, r._2)).toDF

now if you want to run sql over this df, you can register it as temp table

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download