ArtOfWarfare ArtOfWarfare - 10 months ago 67
Groovy Question

Trying to run loop iterations in parallel in Groovy - getting errors

I have a list of hundreds of thousands of tasks that needs to be done (in any order). It takes several hours to run through them all. I figure a good way of speeding it up would be to try running it in parallel.

Here's a minimal reproduction of the code that's single threaded and works fine:

doTask = { more ->
sleep 10000 // sleep 10 seconds

(1..3).each{ more -> doTask(more) }

I have no prior experience with Groovy so this functional syntax is a bit alien to me... but I found a lot of questions on SO where people suggested that you check this page:

Linkrot has occured - that page no longer exists, but I did find an archive of it here:

I tried adapting the examples from that page and ended up with this:

import java.util.concurrent.*

pool = Executors.newFixedThreadPool(3)

defer = {c -> pool.submit(c as Callable) }
doTask = { more ->
sleep 10000 // sleep 10 seconds

(1..3).each{ more -> defer{doTask(more)} }

When I run it over here:

I get this error: access denied ("java.lang.RuntimePermission" "modifyThreadGroup")
at Script1$_run_closure1.doCall(Script1.groovy:5)
at Script1$_run_closure3.doCall(Script1.groovy:12)

What's going on? Have I somehow messed up the syntax? Am I misusing an API? Is this maybe just a limitation of that website?

Answer Source

No, this code runs fine for me... the Exception is probably because the appspot webconsole you're using does not allow multi-threaded applications to run.

Download Groovy and run it with the Groovy Console, it will work fine.

But here's a suggestion for simplifying your code a little bit:

import java.util.concurrent.Callable
import java.util.concurrent.Executors

pool = Executors.newFixedThreadPool( 3 )

doTask = { more ->
    println "Start"
    sleep 10000 // sleep 10 seconds
    println "Finish"

( 1..3 ).each { more -> pool.submit { doTask( more ) } }

In Groovy, anything between curly braces is a closure you can submit to the pool.

So pool.submit { /* any code here to run async */ } works.