LearningSlowly LearningSlowly - 1 month ago 10
JSON Question

Spark file load - `try` and `except` in scala

I wish to read a file that is in one of two locations. I wish to try the first location and if this fails, try the second location. In python I would use a

try
and then if a
IOError
file does not exist
is returned use
except
for the second location. I can read one location in scala like this:

val vertices_raw = sqlContext.read.json("location_a/file.json")


I have tried the following, using getOrElse:

val vertices_raw = sqlContext.read.json("location_a/file.json") getOrElse vertices_raw = sqlContext.read.json("location_b/file.json")


However this did not compile

Answer

You can do the same thing in Scala

var vertices_raw: DataFrame = // Beware of var
try {
  vertices_raw = sqlContext.read.json("location_a/file.json")
} catch {
  case e: Exception =>
    vertices_raw = sqlContext.read.json("location_b/file.json")
}

Or alternatively

val vertices_raw = 
  Try(sqlContext.read.json("location_a/file.json"))
    .getOrElse(sqlContext.read.json("location_b/file.json"))