Mike Mike - 1 month ago 12
Scala Question

Iterating over accumulo table with scala

I have a table in accumulo named records with a few families and qualifiers per row_id, it looks like this in accumulo shell.

michaelp@accumulo records> scan
2016-10-17 16:27:55,359 [Shell.audit] INFO : michaelp@accumulo records> scan
E001 department:sales [] 0
E001 hire_date:20160101 [] 0
E001 name:bob [] 0
E001 name:jerry [] 0
E002 department:marketing [] 0
E002 hire_date:20160202 [] 0
E002 name:sarah [] 0
E003 department:engineering [] 0
E003 hire_date:20160303 [] 0
E003 name:joe [] 0


I want to be able to scan through these couple of rows with a scala connector. After the needed imports my code looks like this:

var opts = new ClientOnRequiredTable()
var bsOpts = new BatchScannerOpts()
opts.parseArgs("test", Array("-t", "records","-u", "michaelp", "-p", "****", "-z", "zookeeper:2181", "-i", "accumulo"), bsOpts)
var connector = opts.getConnector()
var batchReader = connector.createBatchScanner("records", opts.auths, bsOpts.scanThreads)
batchReader.setTimeout(bsOpts.scanTimeout, TimeUnit.MILLISECONDS)
var x = new Range()
var y = new LinkedList[Range]
y.add(x)
batchReader.setRanges(y)


I pass in an empty range to get every row in the table. The issue is when i try to iterate through the results. It sticks on the first row.

scala> while (batchReader.iterator.hasNext()) {println(batchReader.iterator.next.getKey().toString())}
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
...


So why does the iterator not move along?

Answer

Because when you call batchReader.iterator new iterator is created every time. instead do something like below

val iterator = batchReader.iterator

while(iterator.hasNext) {
 println(iterator.next.getKey().toString())
}