feat: Oj Serializers 2.0 #9

ElMassimo · 2023-03-17T19:37:59Z

Description 📖

This pull request introduces a rework of the internals, after experimenting with ways to reduce the indirection to obtain the values for each attribute.

It has more features than v1, while increasing performance even further 🚀

Features ✨

Added render_as_hash to efficiently build a Hash from the serializer
render shortcut, unifying one and many
transform_keys :camelize: a built-in setting to convert keys, in a way that does not affect runtime performance
sort_keys_by :name: allows to sort the response alphabetically, without affecting runtime performance
serialize as an easier approach to define serializer attributes

Breaking Changes

Since returning a Hash is more convenient than returning a Oj::StringWriter, and performance is comparable, I'm considering making default_format :hash the default.

The previous APIs will still be available as one_as_json and many_as_json, as well as a default_format :json setting to make the library work exactly the same as in version 1.

Performance 🚀

Additional optimizations have made JSON rendering 20% to 40% faster than the previous version, so it's now 3x to 7x faster than active_model_serializers or blueprinter.

These are some preliminary results from running the benchmarks running on a MacBook Pro (16-inch, 2021, M1 Pro):

serializing options

        oj_serializers (v2):     9257.4 i/s
   oj_serializers (v2 hash):     9256.2 i/s
                      panko:     8902.1 i/s - 1.20x slower
        oj_serializers (v1):     6579.9 i/s - 1.40x slower
                       alba:     2330.7 i/s - 4.58x slower
                blueprinter:     1741.4 i/s - 5.32x slower
   active_model_serializers:     1308.1 i/s - 7.08x slower

serializing a model

   oj_serializers (v2 hash):    23819.3 i/s
        oj_serializers (v2):    22655.6 i/s - 1.05x slower
        oj_serializers (v1):    19870.2 i/s - 1.20x slower
                      panko:    13756.3 i/s - 1.72x slower
                blueprinter:     6635.5 i/s - 3.59x slower
   active_model_serializers:     6298.1 i/s - 3.78x slower
                       alba:     4197.5 i/s - 5.64x slower

serializing a collection

     oj_serializers as_hash:      243.6 i/s
             oj_serializers:      239.4 i/s - 1.02x slower
                      panko:      169.8 i/s - 1.43x slower
                blueprinter:       68.5 i/s - 3.56x slower
   active_model_serializers:       64.3 i/s - 3.79x slower
                       alba:       44.7 i/s - 5.44x slower

serializing a large collection

             oj_serializers:       23.1 i/s
     oj_serializers as_hash:       23.1 i/s - 1.00x slower
                      panko:       16.6 i/s - 1.40x slower
                blueprinter:        6.6 i/s - 3.53x slower
   active_model_serializers:        6.1 i/s - 3.79x slower
                       alba:        4.1 i/s - 5.64x slower

memory when serializing an object

                         oj:       5809 allocated
                    oj_hash:       6809 allocated - 1.17x more
                blueprinter:      17465 allocated - 3.01x more
                        ams:      26704 allocated - 4.60x more

memory when serializing a collection

                         oj:    5521290 allocated
                    oj_hash:    6777082 allocated - 1.23x more
                blueprinter:   17217298 allocated - 3.12x more
                        ams:   26672082 allocated - 4.83x more

…ing writer

usman-anwar-mindoula · 2023-03-17T20:14:21Z

lib/oj_serializers/serializer.rb

+    # options - list of external options to pass to the sub class (available in `item.options`)
+    #
+    # Returns a Hash, with the attributes specified in the serializer.
+    def one_as_hash(item, options = nil)


@ElMassimo Can you make options default to {}? Our serializer specs were raising undefined method [] for nil errors since @object.options was nil.

Sounds unusual, as options would never return nil, because it falls back to {}.

If you provide a small reproduction I can take a look.

Ok, did not realize there was an options method directly in the serializer. Our serializer is calling @object.options directly, which is where the nil error comes from.

usman-anwar-mindoula · 2023-03-17T20:17:32Z

@ElMassimo Will this include changes to allow ControllerSerializer to use the new render method?

ElMassimo · 2023-03-18T17:44:30Z

Will this include changes to allow ControllerSerializer to use the new render method?

Correct, the controller will use whatever format is specified in the serializer. For example, if you use default_format :hash in your base serializer, then the controller will use one_as_hash and many_as_hash internally.

usman-anwar-mindoula · 2023-03-20T13:34:20Z

Also, can we add the benchmark results in the Readme?

usman-anwar-mindoula · 2023-03-21T00:25:58Z

Looks good!

usman-anwar-mindoula · 2023-03-21T15:22:52Z

Also, might be a good idea to add Panko to the benchmarks?

usman-anwar-mindoula · 2023-03-21T18:19:36Z

Can you add the belongs_to alias again?

ElMassimo · 2023-03-27T12:42:04Z

Added Alba and Panko to the benchmarks.

Following results from running on MacBook Pro (16-inch, 2021, M1 Pro):

serializing a model

          oj_serializers 22.667k (± 0.7%) i/s - 113.730k in 5.017690s
  oj_serializers as_hash 23.684k (± 0.4%) i/s - 120.615k in 5.092785s
                   panko 13.756k (± 0.4%) i/s -  69.050k in 5.019598s
             blueprinter  6.613k (± 0.3%) i/s -  33.660k in 5.090191s
active_model_serializers  6.199k (± 0.7%) i/s -  31.000k in 5.000678s
                    alba  4.197k (± 0.5%) i/s -  21.100k in 5.026957s

Comparison:

  oj_serializers as_hash:    23683.9 i/s
          oj_serializers:    22667.0 i/s - 1.04x slower
                   panko:    13756.3 i/s - 1.72x slower
             blueprinter:     6612.8 i/s - 3.58x slower
active_model_serializers:     6199.5 i/s - 3.82x slower
                    alba:     4197.5 i/s - 5.64x slower

serializing a collection

          oj_serializers    239.447  (± 0.4%) i/s -   1.219k in 5.090952s
  oj_serializers as_hash    243.253  (± 0.4%) i/s -   1.224k in 5.031894s
                   panko    169.772  (± 0.6%) i/s - 864.000  in 5.089264s
             blueprinter     68.523  (± 0.0%) i/s - 348.000  in 5.078683s
active_model_serializers     63.380  (± 1.6%) i/s - 318.000  in 5.018310s
                alba         44.679  (± 0.0%) i/s - 224.000  in 5.013989s

Comparison:

  oj_serializers as_hash:      243.3 i/s
          oj_serializers:      239.4 i/s - 1.02x slower
                   panko:      169.8 i/s - 1.43x slower
             blueprinter:       68.5 i/s - 3.55x slower
active_model_serializers:       63.4 i/s - 3.84x slower
                    alba:       44.7 i/s - 5.44x slower

serializing a large collection

          oj_serializers  23.279 i/s - 118.000 in 5.069285s
  oj_serializers as_hash  23.222 i/s - 118.000 in 5.081735s
                   panko  16.570 i/s -  83.000 in 5.009535s
             blueprinter   6.553 i/s -  33.000 in 5.036259s
active_model_serializers   6.108 i/s -  31.000 in 5.075949s
                    alba   4.127 i/s -  21.000 in 5.114307s

Comparison:
          oj_serializers: 23.3 i/s
  oj_serializers as_hash: 23.2 i/s - 1.00x slower
                   panko: 16.6 i/s - 1.40x slower
             blueprinter:  6.6 i/s - 3.55x slower
active_model_serializers:  6.1 i/s - 3.81x slower
                    alba:  4.1 i/s - 5.64x slower

I think the only reason Panko performs worse in this benchmark is that it's specialized for ActiveRecord, but the benchmark uses a normal object, and the fact that the CPU is an M1 Pro instead of an Intel chip.

Otherwise I'd expect Panko to be 2x faster in simple cases (although it requires a C extension, it's less compatible with AMS, has less features, and is not as flexible when combining with other libraries as it requires using Panko::Response).

…ine aliases

usman-anwar-mindoula · 2023-03-27T19:16:52Z

When are you expecting to release to master?

usman-anwar-mindoula · 2023-03-29T18:04:17Z

Nice work, congrats!

ElMassimo added 3 commits March 17, 2023 13:26

feat: new hash-based zero indirection implementation, faster than str…

0dced46

…ing writer

feat: serialize shortcut, sort_attributes_by, and identifier

a18e6ac

feat: add built-in support to transform keys (camelize / camel_case)

0c7f719

ElMassimo added the enhancement New feature or request label Mar 17, 2023

ElMassimo mentioned this pull request Mar 17, 2023

Add one_as_hash and many_as_hash #7

Closed

ElMassimo added 4 commits March 17, 2023 16:40

chore: fix test

d25b46c

chore: run rubocop

b420c2b

chore: require support files

d37a8af

chore: fix gemfiles

257b511

usman-anwar-mindoula reviewed Mar 17, 2023

View reviewed changes

ElMassimo added 2 commits March 17, 2023 17:17

perf: optimize performance of string writer, now it's in the lead again

173645b

chore: avoid bind_object to squeeze more performance

94cc663

ElMassimo added 6 commits March 20, 2023 17:57

chore: avoid calling instance inside a loop

7321e64

chore: fix test gemfiles

33b1a12

chore: improve dev-time checks

a1d6389

chore: fix rubocop

02a0637

chore: fix edge gemfile

516c503

chore: fix memory usage benchmark

e94694c

ElMassimo force-pushed the perf/render-in-hash branch from 7351866 to e94694c Compare March 20, 2023 21:35

ElMassimo added 3 commits March 20, 2023 18:39

chore: rubocop [skip ci]

8543f23

chore: mention the purpose

71dc3b0

release: 2.0.0-alpha.1

6b81d5f

feat: allow attributes to take a hash of aliases

7e7cf5e

ElMassimo force-pushed the perf/render-in-hash branch from 6eb3d0b to 49ba50b Compare March 25, 2023 01:14

chore: add benchmarks for alba and panko_serializer

b366276

ElMassimo force-pushed the perf/render-in-hash branch from 49ba50b to b366276 Compare March 27, 2023 13:20

ElMassimo added 4 commits March 27, 2023 10:23

chore: fix rubocop

77223b0

chore: add alba and panko to memory usage benchmark

e6b2e16

chore: add belongs_to alias for has_one

6fd2c97

feat: enable caching for hash-based rendering

23edee8

ElMassimo force-pushed the perf/render-in-hash branch 2 times, most recently from 2faeccd to 7b1bfd1 Compare March 27, 2023 14:38

chore: document benchmarks

102d30e

ElMassimo force-pushed the perf/render-in-hash branch from 4416ed8 to 102d30e Compare March 27, 2023 14:40

feat: has_one and has_many can take a block, mongo_attributes can def…

59bb312

…ine aliases

ElMassimo force-pushed the perf/render-in-hash branch from 750cda7 to 59bb312 Compare March 27, 2023 17:48

feat: allow 'attribute' to take in a block

f726658

ElMassimo force-pushed the perf/render-in-hash branch from 4e9e254 to 8da9ef9 Compare March 27, 2023 18:34

chore: improve readme

8a481f7

ElMassimo force-pushed the perf/render-in-hash branch from 8da9ef9 to 8a481f7 Compare March 27, 2023 18:42

chore: fix rubocop

a50d922

chore: prepare release

90882b2

ElMassimo changed the title ~~Oj Serializers 2.0~~ feat: Oj Serializers 2.0 Mar 27, 2023

ElMassimo merged commit 5c38f8f into main Mar 27, 2023

ElMassimo deleted the perf/render-in-hash branch March 27, 2023 19:19

ElMassimo mentioned this pull request Apr 2, 2023

feat: add support for oj_serializers 2 ElMassimo/types_from_serializers#5

Merged

ElMassimo mentioned this pull request Apr 19, 2023

fix: allow using active_model_serializers in associations (compat) #12

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Oj Serializers 2.0 #9

feat: Oj Serializers 2.0 #9

ElMassimo commented Mar 17, 2023 •

edited

Loading

usman-anwar-mindoula Mar 17, 2023 •

edited

Loading

ElMassimo Mar 18, 2023

usman-anwar-mindoula Mar 20, 2023

usman-anwar-mindoula commented Mar 17, 2023

ElMassimo commented Mar 18, 2023 •

edited

Loading

usman-anwar-mindoula commented Mar 20, 2023

usman-anwar-mindoula commented Mar 21, 2023

usman-anwar-mindoula commented Mar 21, 2023

usman-anwar-mindoula commented Mar 21, 2023

ElMassimo commented Mar 27, 2023 •

edited

Loading

usman-anwar-mindoula commented Mar 27, 2023

usman-anwar-mindoula commented Mar 29, 2023

feat: Oj Serializers 2.0 #9

feat: Oj Serializers 2.0 #9

Conversation

ElMassimo commented Mar 17, 2023 • edited Loading

Description 📖

Features ✨

Breaking Changes

Performance 🚀

serializing options

serializing a model

serializing a collection

serializing a large collection

memory when serializing an object

memory when serializing a collection

usman-anwar-mindoula Mar 17, 2023 • edited Loading

Choose a reason for hiding this comment

ElMassimo Mar 18, 2023

Choose a reason for hiding this comment

usman-anwar-mindoula Mar 20, 2023

Choose a reason for hiding this comment

usman-anwar-mindoula commented Mar 17, 2023

ElMassimo commented Mar 18, 2023 • edited Loading

usman-anwar-mindoula commented Mar 20, 2023

usman-anwar-mindoula commented Mar 21, 2023

usman-anwar-mindoula commented Mar 21, 2023

usman-anwar-mindoula commented Mar 21, 2023

ElMassimo commented Mar 27, 2023 • edited Loading

serializing a model

serializing a collection

serializing a large collection

usman-anwar-mindoula commented Mar 27, 2023

usman-anwar-mindoula commented Mar 29, 2023

ElMassimo commented Mar 17, 2023 •

edited

Loading

usman-anwar-mindoula Mar 17, 2023 •

edited

Loading

ElMassimo commented Mar 18, 2023 •

edited

Loading

ElMassimo commented Mar 27, 2023 •

edited

Loading