Description
I've been using Sidekiq + Parallel (https://github.com/grosser/parallel) together for a couple of years. Everything worked fine in in_processes paralellization. I would get basically the same performance when I ran Job.perform_now in the console and Job.perform_later.
This year I've upgraded my application to Ruby 3.0.2p107, Rails 6.1.6, Sidekiq 6.4.2 and Parallel 1.22 and I now get 1/10-20 the performance when the job runs in Sidekiq with perform_later.
class TestJob < ApplicationJob
def perform
Parallel.map(Model.all, in_processes: 5, progress: "Map") do |obj|
heavy_task(obj)
end
end
end
The last time I could get an equal performance I was using Ruby 2.7.1, Rails 6.1.4,Sidekiq 6.2.1 and Parallel 1.20.
Extra info: peform_now in the console has the same performance in the new and old setup.
From my debug I think the 5 processes are running but I doubt they are running in parallel. I see that thing get slower and slower over the execution time.
Any ideia about what can be causing that?