dan dan - 11 months ago 50
Ruby Question

Does Ruby have any construct similar to Clojure's pmap for parallel processing?

I'm trying to decide whether to implement an application in Ruby or Clojure. Two of the requirements involve parallel processing:

  1. The app must make parallel calls to fetch XML feeds and other types of data over the internet. Many such calls are made, and serializing the calls is inefficient.

  2. The responses to those calls ought ideally to be processed in parallel. Processing mainly means transforming raw XML down to a much smaller piece of structured data (a Ruby hash or Clojure map) and inserting that into a MySQL database or CouchDB database.

I know Ruby a lot better than Clojure but if this is the right sort of project for Clojure I am all for using it.

function seems ideal for these two requirements. I'm wondering if some Ruby library or feature had a similarly clean and easy way of doing parallel processing tasks like the above.


Here's a simple little example of one way to do this. Note that there's nothing limiting the number of threads it creates at once, so you might want to create some sort of thread pool if you're running lots of threads.

[1,2,3].map{|x| Thread.start{x+1}}.map{|t| t.join.value}