Specific preloading with scoped associations

It is very common the need to preload an association then filter that association and then use the filtered collection.

For example, imagine that you need to render a list of posts with just its “popular” comments, where “popular” means “with a likes_count greater than or equals to X”.

Approach without scoped association

So you can define Comment#popular? like this…

class Post < ActiveRecord::Base
  has_many :comments
end

class Comment < ActiveRecord::Base
  POPULAR = 9

  belongs_to :post

  def popular?
    likes_count >= POPULAR
  end
end

And so you can sometimes, complete your task, avoiding n+1 queries, by preloading the comments and then selecting just the “popular” comments with ruby.

Something like this…

# Preload all comments for each post
posts = Post.preload(:comments).limit(5)
# => SELECT "posts".* FROM "posts" LIMIT $1
# => SELECT "comments".* FROM "comments" WHERE "comments"."post_id" IN ($1, $2, $3, $4, $5)

posts.each do |post|
  render_title(post)

  # Select just the popular comments
  post.comments.select(&:popular?).each do |comment|
    render_comment(comment)
  end
end

Possible memory problems

This can work on many situations, but sometimes it can give you some troubles, with memory, if the association that you are preloading has a lot of records, because when you preload an association rails will instantiate all the associated records.

Solution with scoped association

One way of solving this problem with rails is to create an association passing an scope, to filter the associated records.

For example, in this problem, you can add a popular_comments associations, to help you preload just the comments that you are going to need.

Like this…

class Post < ActiveRecord::Base
  has_many :comments

  # Here you are defining an association, that will use Comment.popular,
  # to filter the Comment records.
  has_many :popular_comments, -> { popular }, class_name: "Comment"
end

class Comment < ActiveRecord::Base
  POPULAR = 9

  belongs_to :post

  # Here we are definining what `popular` means.
  scope :popular, -> { where(likes_count: POPULAR..) }
end

And then use the popular_comments association to preload just the “popular” comments instead of all the comments for each post, and then use the popular_comments method to display just the “popular” comments.

# Preload directly just the popular comments
posts = Post.preload(:popular_comments).limit(5)
# => SELECT "posts".* FROM "posts" LIMIT $1  [["LIMIT", 5]]
# => SELECT "comments".* FROM "comments" WHERE "comments"."likes_count" >= $1 AND "comments"."post_id" IN ($1, $2, $3, $4, $5)

posts.each do |post|
  render_title(post)

  # You can call the association for each post without n+1 queries
  # because the association has already been preloaded
  post.popular_comments.each do |comment|
    render_comment(comment)
  end
end

In this way you can use your defined scopes, to build “scoped” associations to help you preload just the records that your are going to need and save some memory.

Run the examples

You can find a repo with the examples from this posts, on github.com/bhserna/specific_preloading_with_scoped_associations.

Do you feel bad for asking for help to fix an n+1 queries problem?

Making things work isn't enough for you any more? Now you need to consider performance and scalability?

... But you normally have troubles fixing n+1 queries and trying to find why active record is ignoring your "includes"?

Are you are worried because you feel unqualified to tackle tasks with complex data models?