I want to read a bunch of text files from a hdfs location and perform mapping on it in an iteration using spark.
JavaRDD<String> records = ctx.textFile(args, 1);
You can specify whole directories, use wildcards and even CSV of directories and wildcards. E.g.:
As Nick Chammas points out this is an exposure of Hadoop's
FileInputFormat and therefore this also works with Hadoop (and Scalding).