shakedzy shakedzy - 1 year ago 70
Scala Question

Scala: how can I perform actions when test are over?

I'm using

version 2.2.1. I'm trying to write test that run on a SparkContext. How can I initiate a SparkContext when the tests begin, take this Sc thorugh all the tests and then stop it when they are all done?

(I know that it suppose to stop by itself, but I'sd still like to do it myself)

Answer Source

You can use BeforeAndAfterAll from ScalaTest. Define a base trait that starts and stops the SparkContext and use it with your other tests.

trait SparkTest extends BeforeAndAfterAll {
  self: Suite =>

  @transient var sc: SparkContext = _

  override def beforeAll {
    val conf = new SparkConf().
    sc = new SparkContext(conf)

  override def afterAll: Unit = {
    try {
    } finally {


// Mix-in the trait with your tests like below.
class MyTest extends FunSuite with SparkTest {
  test("my test") {
    // you can access "sc" here.
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download