-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Skip redundant queries when include_data is disabled #47
Changes from all commits
a87b378
8f115b0
2f3fe92
3289c9f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -64,7 +64,9 @@ class UserSerializer < BaseSerializer | |||||
# The previous one is a shorthand for the following lines: | ||||||
lazy_relationship :blog_posts, loader: AmsLazyRelationships::Loaders::Association.new("User", :blog_posts) | ||||||
has_many :blog_posts, serializer: BlogPostSerializer do |serializer| | ||||||
serializer.lazy_blog_posts | ||||||
# non-proc custom finder will work as well, but it can produce redundant sql | ||||||
# queries, please see [Example 2: Modifying the relationship before rendering](#example-2-modifying-the-relationship-before-rendering) | ||||||
-> { serializer.lazy_blog_posts } | ||||||
end | ||||||
|
||||||
lazy_has_one :poro_model, loader: AmsLazyRelationships::Loaders::Direct.new(:poro_model) { |object| PoroModel.new(object) } | ||||||
|
@@ -116,6 +118,16 @@ end | |||||
#### Example 2: Modifying the relationship before rendering | ||||||
Sometimes it may happen that you need to process the relationship before rendering, e.g. decorate the records. In this case the gem provides a special method (in our case `lazy_comments`) for each defined relationship. Check out the example - we'll decorate every comment before serializing: | ||||||
|
||||||
```ruby | ||||||
class BlogPostSerializer < BaseSerializer | ||||||
lazy_has_many :comments do |serializer| | ||||||
-> { serializer.lazy_comments.map(&:decorate) } | ||||||
end | ||||||
end | ||||||
``` | ||||||
|
||||||
Despite the fact that non-block custom finder such as | ||||||
|
||||||
```ruby | ||||||
class BlogPostSerializer < BaseSerializer | ||||||
lazy_has_many :comments do |serializer| | ||||||
|
@@ -124,6 +136,29 @@ class BlogPostSerializer < BaseSerializer | |||||
end | ||||||
``` | ||||||
|
||||||
will work still, it's better to implement it in a form of lambda, in order to avoid redundant SQL queries when `include_data` AMS setting appears to be `false`: | ||||||
|
||||||
```ruby | ||||||
class BlogPostSerializer < BaseSerializer | ||||||
lazy_has_many :comments do |serializer| | ||||||
include_data :if_sideloaded | ||||||
-> { serializer.lazy_comments.map(&:decorate) } | ||||||
end | ||||||
end | ||||||
``` | ||||||
|
||||||
Feel free to skip custom lazy finder for association if your goal is just to define `include_data` setting and/or to specify some links and metas: | ||||||
|
||||||
```ruby | ||||||
class BlogPostSerializer < BaseSerializer | ||||||
lazy_has_many :comments do | ||||||
include_data :if_sideloaded | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Wasn't it supposed to be:
Suggested change
? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
In second case we need to suppress any kind of finders but we need to keep the finder alive for the time when first case will happen. i.e. lazy_has_many :comments do
include_data false
nil
end is completely ok and There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok, thanks for explanation - so this will work correctly, without N+1s, both when the resource is included and when it isn't 👍 |
||||||
link :self, 'a link' | ||||||
meta name: 'Dan Brown' | ||||||
end | ||||||
end | ||||||
``` | ||||||
|
||||||
#### Example 3: Introducing loader classes | ||||||
Under the hood ams_lazy_relationships uses special loader classes to batch load the relationships. By default the gem uses serializer class names and relationship names to instantiate correct loaders, but it may happen that e.g. your serializer's class name doesn't match the model name (e.g. your model's name is `BlogPost` but the serializer's name is `PostSerializer`). | ||||||
|
||||||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
# frozen_string_literal: true | ||
|
||
require "ams_lazy_relationships/extensions/reflection" | ||
|
||
module AmsLazyRelationships | ||
module Extensions | ||
end | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,71 @@ | ||
# frozen_string_literal: true | ||
|
||
# There is a general problem inside AMS related to custom association finder | ||
# combined with `include_data` setting: | ||
# | ||
# class BlogPostSerializer < BaseSerializer | ||
# belongs_to :category do | ||
# include_data :if_sideloaded | ||
# object.categories.last | ||
# end | ||
# end | ||
# | ||
# The problem is that `belongs_to` block will be fully evaluated each time for | ||
# each object, and only after that AMS is able to take into account | ||
# `include_data` mode - | ||
# https://github.com/rails-api/active_model_serializers/blob/v0.10.10/lib/active_model/serializer/reflection.rb#L162-L163 | ||
# | ||
# def value(serializer, include_slice) | ||
# # ... | ||
# block_value = instance_exec(serializer, &block) if block | ||
# return unless include_data?(include_slice) | ||
# # ... | ||
# end | ||
# | ||
# That causing redundant (and so huge potentially!) SQL queries and AR objects | ||
# allocation when `include_data` appears to be `false` but `belongs_to` block | ||
# defines instant (not a kind of AR::Relation) custom association finder. | ||
# | ||
# Described problem is a very specific use case for pure AMS applications. | ||
# The bad news is that `ams_lazy_relationships` always utilizes the | ||
# association block - | ||
# https://github.com/Bajena/ams_lazy_relationships/blob/v0.2.0/lib/ams_lazy_relationships/core/relationship_wrapper_methods.rb#L32-L36 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'd avoid adding links to specific versions in the code. Let's get rid of lines 32-44 - they may quickly get out of date. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That is exactly why I pinned |
||
# | ||
# def define_lazy_association(type, name, options, block) | ||
# #... | ||
# block ||= lambda do |serializer| | ||
# serializer.public_send("lazy_#{name}") | ||
# end | ||
# | ||
# public_send(type, name.to_sym, real_relationship_options, &block) | ||
# #... | ||
# end | ||
# | ||
# This way we break `include_data` optimizations for the host application. | ||
# | ||
# In order to overcome that we are forced to monkey-patch | ||
# `AmsLazyRelationships::Extensions::Reflection#value` method and make it to be | ||
# ready for Proc returned by association block. This way we will use a kind of | ||
# | ||
# block ||= lambda do |serializer| | ||
# -> { serializer.public_send("lazy_#{name}") } | ||
# end | ||
# | ||
# as association block, then AMS will evaluate it, get the value of `include_data` | ||
# setting, make a decision do we need to continue with that association, if so - | ||
# will finally evaluate the proc with lazy relationship inside it. | ||
|
||
module AmsLazyRelationships | ||
module Extensions | ||
module Reflection | ||
def value(*) | ||
case (block_value = super) | ||
when Proc, Method then block_value.call | ||
else block_value | ||
end | ||
end | ||
end | ||
end | ||
end | ||
|
||
::ActiveModel::Serializer::Reflection.prepend AmsLazyRelationships::Extensions::Reflection | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'd consider applying this patch only to AMS versions >= 0.10.3. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I thought a lot about that from the beginning, but this way we need to apply the same condition in default Even worse, readme description will be more complex as well since for lower AMS versions the proc-like custom finder will instantly fail. I.e. we propose the snippet of code which will crush on AMS version still accepted by As an option, we can move all How you think? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Notice that this way There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok, you're right, doing it the way I suggested generates more problems than it solves. Let's keep it then. But in such case let's make sure that all AMS versions are tested for proc evaluation (specs are skipped for lower AMS versions in https://github.com/Bajena/ams_lazy_relationships/pull/47/files#diff-6e72b60a2fb8b930cc9741785114a948R871). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. They are tested already due to the new default |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It can still be called without using a proc, right?
Maybe we could keep the old version for simplicity and just mention the proc form in the
include_data
example you added in https://github.com/Bajena/ams_lazy_relationships/pull/47/files#diff-04c6e90faac2675aa89e2176d2eec7d8R134 ?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sure, Reflection monkey-patch will forward non-proc value as it is.
But I believe that in readme is better to propose the best possible sample by default.
Just imagine:
ActiveModel::Serializer.config.include_data_default = false
globallydata
, butI think we need to leave
-> { serializer.lazy_blog_posts }
as a very default sample, and add a small hint thatserializer.lazy_blog_posts
will work as well but can produce redundant queries.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I must admit that your explanation makes sense to me. Let's leave proc, but also let's mention the non-proc version as you suggested 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
roger that! )