Anthony Anthony - 1 year ago 70
Scala Question

Is it possible to create an RDD in sbt console?

I'm working in SBT console and have imported the following

scala> import org.apache.spark._
scala> val sc = new SparkContext("local", "shell")
scala> import org.apache.spark.rdd._

I would like to make an
in the console. Is that possible?

scala> case class MyClass(foobar: String, something: String)

The reason I want to do this is to play around with RDD and create sample data that I can test with. Rather than reading a textFile or reading huge data from HDFS or S3

Answer Source

You can use sc.parallelize and give that a List of your objects.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download