Sidekiq documentation says:
Don't set the concurrency higher than 50. I've seen stability issues
with concurrency of 100, for example
app[worker.1]: 3 TID-ow5z46exw WARN: /app/vendor/ruby-2.3.0/lib/ruby/2.3.0/monitor.rb:187:in `lock'
app[worker.1]: 3 TID-os9ulw8ps WARN: /app/vendor/ruby-2.3.0/lib/ruby/2.3.0/net/http.rb:880:in `initialize'
app[worker.1]: 3 TID-os9ulw8ps WARN: /app/vendor/ruby-2.3.0/lib/ruby/2.3.0/timeout.rb:95:in `join'
app[worker.1]: 3 TID-osjnd6zac WARN: /app/vendor/ruby-2.3.0/lib/ruby/2.3.0/net/protocol.rb:158:in `wait_readable'
Well, sidekiq stability issues in high concurrency are as follows.
When you are setting a concurrency that is higher than 80 (or 50) you may encounter in this error "can't create Thread: Resource temporarily unavailable:"
Some jobs will return back to queue, sometimes the entire process will be terminated and jobs will be lost, unless you use sidekiq pro reliability feature
The above will happen also when the memory consumption will stay low (< 240MB in my example)
Everything is updated in the github issue