Anthony Anthony - 3 months ago 10
Scala Question

Is it possible to create an RDD in sbt console?

I'm working in SBT console and have imported the following

scala> import org.apache.spark._
scala> val sc = new SparkContext("local", "shell")
scala> import org.apache.spark.rdd._

I would like to make an
in the console. Is that possible?

scala> case class MyClass(foobar: String, something: String)

The reason I want to do this is to play around with RDD and create sample data that I can test with. Rather than reading a textFile or reading huge data from HDFS or S3


You can use sc.parallelize and give that a List of your objects.