João_testeSW João_testeSW - 1 year ago 75
Scala Question

Error when parsing a line from the data into the class. Spark Mllib

I've this code implemented:

scala> import org.apache.spark._

scala> import org.apache.spark.rdd.RDD
import org.apache.spark.rdd.RDD

scala> import org.apache.spark.util.IntParam
import org.apache.spark.util.IntParam

scala> import org.apache.spark.graphx._
import org.apache.spark.graphx._

scala> import org.apache.spark.graphx.util.GraphGenerators
import org.apache.spark.graphx.util.GraphGenerators

scala> case class Transactions(ID:Long,Chain:Int,Dept:Int,Category:Int,Company:Long,Brand:Long,Date:String,ProductSize:Int,ProductMeasure:String,PurchaseQuantity:Int,PurchaseAmount:Double)
defined class Transactions

When I try to run this:

def parseTransactions(str:String): Transactions = {
| val line = str.split(",")
| Transactions(line(0),line(1),line(2),line(3),line(4),line(5),line(6),line(7),line(8),line(9),line(10))
| }

I am obtaining this error: :38: error: type mismatch;
found : String
required: Long

Anyone knows why I'm getting this error? I am doing a social netowork analysis over the schema that I put above.

Many thanks!

Answer Source

You are creating array from "," separated values which returns String array. Cast it to appropriate type before assigning to case class arguments.

val line = str.split(",")
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download