João_testeSW João_testeSW - 3 months ago 14
Scala Question

Error when parsing a line from the data into the class. Spark Mllib

I've this code implemented:

scala> import org.apache.spark._

scala> import org.apache.spark.rdd.RDD
import org.apache.spark.rdd.RDD

scala> import org.apache.spark.util.IntParam
import org.apache.spark.util.IntParam

scala> import org.apache.spark.graphx._
import org.apache.spark.graphx._

scala> import org.apache.spark.graphx.util.GraphGenerators
import org.apache.spark.graphx.util.GraphGenerators

scala> case class Transactions(ID:Long,Chain:Int,Dept:Int,Category:Int,Company:Long,Brand:Long,Date:String,ProductSize:Int,ProductMeasure:String,PurchaseQuantity:Int,PurchaseAmount:Double)
defined class Transactions


When I try to run this:

def parseTransactions(str:String): Transactions = {
| val line = str.split(",")
| Transactions(line(0),line(1),line(2),line(3),line(4),line(5),line(6),line(7),line(8),line(9),line(10))
| }


I am obtaining this error: :38: error: type mismatch;
found : String
required: Long

Anyone knows why I'm getting this error? I am doing a social netowork analysis over the schema that I put above.

Many thanks!

Answer

You are creating array from "," separated values which returns String array. Cast it to appropriate type before assigning to case class arguments.

val line = str.split(",")
line(0).toLong