Mike Mike - 1 year ago 60
Scala Question

Iterating over accumulo table with scala

I have a table in accumulo named records with a few families and qualifiers per row_id, it looks like this in accumulo shell.

michaelp@accumulo records> scan
2016-10-17 16:27:55,359 [Shell.audit] INFO : michaelp@accumulo records> scan
E001 department:sales [] 0
E001 hire_date:20160101 [] 0
E001 name:bob [] 0
E001 name:jerry [] 0
E002 department:marketing [] 0
E002 hire_date:20160202 [] 0
E002 name:sarah [] 0
E003 department:engineering [] 0
E003 hire_date:20160303 [] 0
E003 name:joe [] 0


I want to be able to scan through these couple of rows with a scala connector. After the needed imports my code looks like this:

var opts = new ClientOnRequiredTable()
var bsOpts = new BatchScannerOpts()
opts.parseArgs("test", Array("-t", "records","-u", "michaelp", "-p", "****", "-z", "zookeeper:2181", "-i", "accumulo"), bsOpts)
var connector = opts.getConnector()
var batchReader = connector.createBatchScanner("records", opts.auths, bsOpts.scanThreads)
batchReader.setTimeout(bsOpts.scanTimeout, TimeUnit.MILLISECONDS)
var x = new Range()
var y = new LinkedList[Range]
y.add(x)
batchReader.setRanges(y)


I pass in an empty range to get every row in the table. The issue is when i try to iterate through the results. It sticks on the first row.

scala> while (batchReader.iterator.hasNext()) {println(batchReader.iterator.next.getKey().toString())}
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
E001 department:sales [] 1476720996135 false
...


So why does the iterator not move along?

Answer Source

Because when you call batchReader.iterator new iterator is created every time. instead do something like below

val iterator = batchReader.iterator

while(iterator.hasNext) {
 println(iterator.next.getKey().toString())
}