Akh Akh - 2 months ago 15
Ruby Question

Preloading of chain of complex/indirect ActiveRecord functions/'associations'

I am working on a plugin for Discourse, which means that I can modify classes with class_eval, but I cannot change the DB schema. To store extra data about the Topic model, I can perform joins with TopicCustomField, which is provided for this purpose.

I am able to store and retrieve all the data I need, but when many Topics are loaded at once, the DB performance is inefficient because my indirect data is loaded once for each Topic by itself. It would be much better if this data were loaded all at once for each Topic, like can happen when using preload or includes.

For example, each Topic has a topic_guid, and a set of parent_guids (stored in a single string with dashes because order is important). These parent_guids point to both other Topic's topic_guids as well as the name of other Groups.

I would love to be able write something like:


has_many :topic_custom_fields
has_many :parent_guids, -> { where(name: 'parent_guids').pluck(:value).first }, :through => :topic_custom_fields
has_many :parent_groups, class_name: 'Group', primary_key: :parent_guids, foreign_key: :name


But this :through complains about not being able to find an association ":parent_guids" in TopicCustomField, and primary_key won't actually take an association instead of a DB column.

I've also tried the following, but the :through clauses are not able to use the functions as associations.

has_many :topic_custom_fields do
def parent_guids
parent_guids_str = where(name: PARENT_GUIDS_FIELD_NAME).pluck(:value).first
return [] unless parent_guids_str
parent_guids_str.split('-').delete_if { |s| s.length == 0 }
end
def parent_groups
Group.where(name: parent_guids)
end
end

has_many :parent_guids, :through => :topic_custom_fields
has_many :parent_groups, :through => :topic_custom_fields


Using Rails 4.2.7.1

Akh Akh
Answer

I hope there is a more elegant solution, but this is what I have done in order to preload my data efficiently. This should be fairly easy to extend to other applications.

I modify Relation's exec_queries, which calls other preloading functions.

ActiveRecord::Relation.class_eval do
    attr_accessor :preload_funcs

    old_exec_queries = self.instance_method(:exec_queries)
    define_method(:exec_queries) do |&block|
        records = old_exec_queries.bind(self).call(&block)
        if preload_funcs
            preload_funcs.each do |func|
                func.call(self, records)
            end
        end
        records
    end
end

To Topic, I added:

has_many :topic_custom_fields
attr_accessor :parent_groups

def parent_guids
    parent_guids_str = topic_custom_fields.select { |a| a.name == PARENT_GUIDS_FIELD_NAME }.first
    return [] unless parent_guids_str
    parent_guids_str.value.split('-').delete_if { |s| s.length == 0 }
end

And then in order to preload the parent_groups, I do:

def preload_parent_groups(topics)
    topics.preload_funcs ||= []
    topics.preload_funcs <<= Proc.new do |association, records|
        parent_guidss = association.map {|t| t.parent_guids}.flatten
        parent_groupss = Group.where(name: parent_guidss).to_a

        records.each do |t|
            t.parent_groups = t.parent_guids.map {|guid| parent_groupss.select {|group| group.name == guid }.first}
        end
    end
    topics
end

And finally, I add the preloaders to my Relation query:

result = result.preload(:topic_custom_fields)
result = preload_parent_groups(result)
Comments