LearningSlowly LearningSlowly - 11 months ago 47
JSON Question

Spark file load - `try` and `except` in scala

I wish to read a file that is in one of two locations. I wish to try the first location and if this fails, try the second location. In python I would use a

and then if a
file does not exist
is returned use
for the second location. I can read one location in scala like this:

val vertices_raw = sqlContext.read.json("location_a/file.json")

I have tried the following, using getOrElse:

val vertices_raw = sqlContext.read.json("location_a/file.json") getOrElse vertices_raw = sqlContext.read.json("location_b/file.json")

However this did not compile

Answer Source

You can do the same thing in Scala

var vertices_raw: DataFrame = // Beware of var
try {
  vertices_raw = sqlContext.read.json("location_a/file.json")
} catch {
  case e: Exception =>
    vertices_raw = sqlContext.read.json("location_b/file.json")

Or alternatively

val vertices_raw =