Tomer Shemesh Tomer Shemesh - 1 year ago 91
Ruby Question

Rails Circular Dependency Errors in rake task only when multithreading without delete_all

I am trying to create a rake task to do some updating of data. the problem is it is kind of slow so i thought i would make it multi threaded to speed it up. the problem is when i run it in 1 thread its fine but when i run it in 5 threads i only sometimes get

rake aborted!
Circular dependency detected while autoloading constant ClassInfo
/home/Tomer/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:492:in `load_missing_constant'
/home/Tomer/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:184:in `const_missing'
/home/Tomer/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:526:in `load_missing_constant'
/home/Tomer/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:184:in `const_missing'
/home/Tomer/workspace/DrexelRegistrationBackend/app/helpers/html_parse_helper.rb:66:in `block in load_class_from_link'
/home/Tomer/.rvm/gems/ruby-2.2.2/gems/nokogiri- `block in each'
/home/Tomer/.rvm/gems/ruby-2.2.2/gems/nokogiri- `upto'
/home/Tomer/.rvm/gems/ruby-2.2.2/gems/nokogiri- `each'
/home/Tomer/workspace/DrexelRegistrationBackend/app/helpers/html_parse_helper.rb:59:in `load_class_from_link'
/home/Tomer/workspace/DrexelRegistrationBackend/app/helpers/html_parse_helper.rb:43:in `block (3 levels) in load_classes_from_catalog'
/home/Tomer/workspace/DrexelRegistrationBackend/app/helpers/html_parse_helper.rb:42:in `each'
/home/Tomer/workspace/DrexelRegistrationBackend/app/helpers/html_parse_helper.rb:42:in `block (2 levels) in load_classes_from_catalog'

I am using nokogiri and opening a url and html parsing it and then inserting it into my database. The weird thing is if i insert ClassInfo.delete_all in the beginning the error goes away, but i dont want to do that. I need to check if its in my database so i check if it exists in this line which is where i crashes

myclass = ClassInfo.find_by(class_id: class_id)

It has no joins or other classes dependent on it this is my migrate

create_table :class_infos do |t|
t.text :class_id, index: true
t.text :subject_code
t.text :subject
t.text :course_number
t.text :title
t.float :credits_lower_range
t.float :credits_upper_range
t.text :description
t.boolean :writing_intensive
t.text :college
t.text :repeat_status
t.text :prereqs
t.text :restrictions
t.text :coreqs
t.text :term_type
t.text :student_type
t.timestamps null: false

and this is the model

class ClassInfo < ActiveRecord::Base

the very weird thing is that after it crashes i can usually run the rake task again immediately and it works fine the second time. Very weird

Here is mostly my entire module called from my rake

require 'open-uri'

module HtmlParseHelper

def self.parse_catalog
@@threads =
load_classes_from_catalog(url_sections) #an array of 5 arrays contaning urls objects
@@threads.each { |thr| thr.join }

def self.load_classes_from_catalog(urlArr)
@@threads.push( do

def self.load_class_from_link(link, term_type, student_type, subject)

doc = Nokogiri::HTML(open(link))do |config|
class_id = doc.css('div.courseblock').children[0].text
drexel_class = ClassInfo.find_by(class_id: class_id) #where it crashes

Answer Source

int the end it was eager loading that was screwing me up.

i just added


to the beginning and it worked

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download