How do I generate and load multiple s3 file path in scala so that I can use :
You can take a look at this answer, You can specify whole
wildcards and even
CSV of directories and wildcards. E.g.:
Or you can use
AWS API to get the
list of files locations and read those files using spark .
You can look into this answer to AWS S3 file search.