Error when parsing a line from the data into the class. Spark Mllib

Question

I've this code implemented:

scala> import org.apache.spark._



scala> import org.apache.spark.rdd.RDD

import org.apache.spark.rdd.RDD



scala> import org.apache.spark.util.IntParam

import org.apache.spark.util.IntParam



scala> import org.apache.spark.graphx._

import org.apache.spark.graphx._



scala> import org.apache.spark.graphx.util.GraphGenerators

import org.apache.spark.graphx.util.GraphGenerators



scala> case class Transactions(ID:Long,Chain:Int,Dept:Int,Category:Int,Company:Long,Brand:Long,Date:String,ProductSize:Int,ProductMeasure:String,PurchaseQuantity:Int,PurchaseAmount:Double)

defined class Transactions

When I try to run this:

def parseTransactions(str:String): Transactions = {

     | val line = str.split(",")

     | Transactions(line(0),line(1),line(2),line(3),line(4),line(5),line(6),line(7),line(8),line(9),line(10))

     | }

I am obtaining this error: :38: error: type mismatch;
found : String
required: Long

Anyone knows why I'm getting this error? I am doing a social netowork analysis over the schema that I put above.

Many thanks!

Answer 1

You are creating array from "," separated values which returns String array. Cast it to appropriate type before assigning to case class arguments.

val line = str.split(",")
line(0).toLong

Error when parsing a line from the data into the class. Spark Mllib

Sign up