Vlad Cenan Vlad Cenan - 6 months ago 20
Ruby Question

Add multithreads/concurency in script

I created a script which checks healthcheck and ports status from a .json file populated with microservices.
So for every microservice from the .json file the script will output the HTTP status and healthcheck body and other small details, and I want to add multithreading here in order to return all the output at once.Please see the script below:

#!/usr/bin/env ruby
... get the environment argument part...

file = File.read('./services.json')
data_hash = JSON.parse(file)

threads = []
service = data_hash.keys
service.each do |microservice|
threads << Thread.new do
begin
puts "Microservice: #{microservice}"
port = data_hash["#{microservice}"]['port']
puts "Port: #{port}"

nodes = "knife search 'chef_environment:#{env} AND recipe:#{microservice}' -i"
node = %x[ #{nodes} ].split
node.each do |n|
puts "Node: #{n}"
uri = URI("http://#{n}:#{port}/healthcheck?count=10")
res = Net::HTTP.get_response(uri)
status = Net::HTTP.get(uri)
puts res.code
puts status
puts res.message
end

rescue Net::ReadTimeout
puts "ReadTimeout Error"
next
end
end
end
threads.each do |thread|
thread.join
end


Anyway in this way the script return first the puts "Microservice: #{microservice}" and puts "Port: #{port}" and after this it will return the nodes and only after the STATUS.
How can I return all the data for each loop together?

Answer

Instead of puts write output to a variable (hash). If you wand to wait for all threads to finish their job before showing the output, use ThreadsWait class.

require 'thwait'
file = File.read('./services.json')
data_hash = JSON.parse(file)

h = {}
threads = []
service = data_hash.keys
service.each do |microservice|
threads << Thread.new do
  thread_id = Thread.current.object_id.to_s(36)
  begin
  h[thread_id] = "Microservice: #{microservice}"
  port = data_hash["#{microservice}"]['port']
  h[thread_id] << "Port: #{port}"

  nodes = "knife search 'chef_environment:#{env} AND recipe:#{microservice}' -i"
  node = %x[ #{nodes} ].split
    node.each do |n|
    h[thread_id]<< "Node: #{n}"
    uri = URI("http://#{n}:#{port}/healthcheck?count=10")
    res = Net::HTTP.get_response(uri)
    status = Net::HTTP.get(uri)
    h[thread_id] << res.code
    h[thread_id] << status
    h[thread_id] << res.message
  end

rescue Net::ReadTimeout
  h[thread_id] << "ReadTimeout Error"
  next
end
end
end
threads.each do |thread|
  thread.join
end

# wait untill all threads finish their job
ThreadsWait.all_waits(*threads)

p h
Comments