Akh Akh - 19 days ago 4x
Ruby Question

Preloading of chain of complex/indirect ActiveRecord functions/'associations'

I am working on a plugin for Discourse, which means that I can modify classes with class_eval, but I cannot change the DB schema. To store extra data about the Topic model, I can perform joins with TopicCustomField, which is provided for this purpose.

I am able to store and retrieve all the data I need, but when many Topics are loaded at once, the DB performance is inefficient because my indirect data is loaded once for each Topic by itself. It would be much better if this data were loaded all at once for each Topic, like can happen when using preload or includes.

For example, each Topic has a topic_guid, and a set of parent_guids (stored in a single string with dashes because order is important). These parent_guids point to both other Topic's topic_guids as well as the name of other Groups.

I would love to be able write something like:

has_many :topic_custom_fields
has_many :parent_guids, -> { where(name: 'parent_guids').pluck(:value).first }, :through => :topic_custom_fields
has_many :parent_groups, class_name: 'Group', primary_key: :parent_guids, foreign_key: :name

But this :through complains about not being able to find an association ":parent_guids" in TopicCustomField, and primary_key won't actually take an association instead of a DB column.

I've also tried the following, but the :through clauses are not able to use the functions as associations.

has_many :topic_custom_fields do
def parent_guids
parent_guids_str = where(name: PARENT_GUIDS_FIELD_NAME).pluck(:value).first
return [] unless parent_guids_str
parent_guids_str.split('-').delete_if { |s| s.length == 0 }
def parent_groups
Group.where(name: parent_guids)

has_many :parent_guids, :through => :topic_custom_fields
has_many :parent_groups, :through => :topic_custom_fields

Using Rails

Akh Akh

I hope there is a more elegant solution, but this is what I have done in order to preload my data efficiently. This should be fairly easy to extend to other applications.

I modify Relation's exec_queries, which calls other preloading functions.

ActiveRecord::Relation.class_eval do
    attr_accessor :preload_funcs

    old_exec_queries = self.instance_method(:exec_queries)
    define_method(:exec_queries) do |&block|
        records = old_exec_queries.bind(self).call(&block)
        if preload_funcs
            preload_funcs.each do |func|
                func.call(self, records)

To Topic, I added:

has_many :topic_custom_fields
attr_accessor :parent_groups

def parent_guids
    parent_guids_str = topic_custom_fields.select { |a| a.name == PARENT_GUIDS_FIELD_NAME }.first
    return [] unless parent_guids_str
    parent_guids_str.value.split('-').delete_if { |s| s.length == 0 }

And then in order to preload the parent_groups, I do:

def preload_parent_groups(topics)
    topics.preload_funcs ||= []
    topics.preload_funcs <<= Proc.new do |association, records|
        parent_guidss = association.map {|t| t.parent_guids}.flatten
        parent_groupss = Group.where(name: parent_guidss).to_a

        records.each do |t|
            t.parent_groups = t.parent_guids.map {|guid| parent_groupss.select {|group| group.name == guid }.first}

And finally, I add the preloaders to my Relation query:

result = result.preload(:topic_custom_fields)
result = preload_parent_groups(result)