Oktav Oktav - 5 months ago 18
Node.js Question

Add jobs to a rails-sidekiq queue from node.js

I have 2 apps. 1 is a rails app and the other one is a node.js one. I'm using sidekiq in the Rails app. My node.js app will receive a huge amount of http posts (at least 20 req/sec) and I need those requests to be processed by the rails app.

The best way I found is to put those requests in a sidekiq queue and have Rails process them when it can. Is it possible to add a job to sidekiq from a different application? Is this done by talking directly to redis? The job will be very simple:

message_type
source
payload


These fields are present in the initial http post request.

I thought of using rails directly as the first entry-point but rails is not that good when it comes to loads of concurrent http requests.

Any ideas on how to add a job to a sidekiq queue from outside rails?

Answer

Apparently it's not that difficult. Here is the code I used:

npm install sidekiq --save

// Node.js
// app.js
var redis = require("redis")
var Sidekiq = require("sidekiq")

client = redis.createClient(); // Will create a connection to redis DB 0 on 127.0.0.1
sidekiq = new Sidekiq(client); // Will instantiate a sidekiq object which we use to add jobs

sidekiq.enqueue("MyProcessor", ["some-source", "TEST_MESSAGE_TYPE", "Some payload; More payload"], {
    retry: false,
    queue: "default"
}); // Will enqueue a job in the "default" queue with the 3 arguments


# Rails
# app/workers/my_processor.rb
class MyProcessor
  include Sidekiq::Worker
  sidekiq_options retry: false

  def perform(source, message_type, payload)
    logger.debug("Message is being processed: #{source} - #{message_type} - #{payload}")
  end
end