I want to convert Spark Dataframe each row as a String with a delimiter between each column value.
I have a input dataframe 'df' with 3 columns "firstname","lastname","age", with two records which look like below.
Row1: John Abhraham 21
Row2: Steve Austin 22
I want to create a new dataframe with just one column which has data like below.
Can anyone please help in doing this.
I don't have a Spark shell handy, but I think this one liner should do it:
def stringifyRows(df: DataFrame, sep: String): DataFrame = df.map(row => row.mkString(sep))
For your example, you would call this as
stringifyRows(myDf, "$"). Let me know what the error message is if this doesn't work.