Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make batch sizes configurable #4149

Merged
merged 22 commits into from
Mar 19, 2024
Merged

Make batch sizes configurable #4149

merged 22 commits into from
Mar 19, 2024

Conversation

kropidlowsky
Copy link
Contributor

@kropidlowsky kropidlowsky commented Mar 6, 2024 •
edited by AlekSi
Loading

Description

Replaced batch sizes hard-cored values with configuratable for: sqlite, postgrsql, hana.

Closes #3708.

Readiness checklist

  • I added/updated unit tests (and they pass).
  • I added/updated integration/compatibility tests (and they pass).
  • I added/updated comments and checked rendering.
  • I made spot refactorings.
  • I updated user documentation.
  • I ran task all, and it passed.
  • I ensured that PR title is good enough for the changelog.
  • (for maintainers only) I set Reviewers (@FerretDB/core), Milestone (Next), Labels, Project and project's Sprint fields.
  • I marked all done items in this checklist.

@kropidlowsky kropidlowsky requested review from AlekSi and a team as code owners March 6, 2024 00:14
@kropidlowsky kropidlowsky requested a review from rumyantseva March 6, 2024 00:14
@codecov Codecov
Copy link

codecov bot commented Mar 6, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 71.05263% with 11 lines in your changes are missing coverage. Please review.

Project coverage is 73.90%. Comparing base (aea56ea) to head (0a86ac7).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #4149      +/-   ##
==========================================
- Coverage   74.61%   73.90%   -0.72%     
==========================================
  Files         339      339              
  Lines       22862    22876      +14     
==========================================
- Hits        17059    16906     -153     
- Misses       4498     4684     +186     
+ Partials     1305     1286      -19
Files Coverage Δ
integration/setup/listener.go 70.94% <100.00%> (-3.89%) ⬇️
integration/setup/setup.go 84.74% <ø> (ø)
internal/backends/postgresql/backend.go 71.42% <100.00%> (ø)
internal/backends/postgresql/metadata/registry.go 71.54% <100.00%> (+0.05%) ⬆️
internal/backends/sqlite/backend.go 80.39% <100.00%> (ø)
internal/backends/sqlite/metadata/registry.go 70.58% <100.00%> (+0.10%) ⬆️
internal/handler/handler.go 50.86% <ø> (ø)
internal/handler/msg_insert.go 82.69% <100.00%> (-0.17%) ⬇️
internal/handler/registry/postgresql.go 85.18% <100.00%> (+1.18%) ⬆️
internal/handler/registry/registry.go 44.44% <ø> (ø)
... and 7 more

... and 11 files with indirect coverage changes

Flag Coverage Δ
filter-true 67.38% <71.05%> (-0.81%) ⬇️
hana-1 ?
integration 67.38% <71.05%> (-0.81%) ⬇️
mongodb-1 5.13% <0.00%> (-0.02%) ⬇️
postgresql-1 46.48% <39.47%> (+0.03%) ⬆️
postgresql-2 49.56% <39.47%> (-0.10%) ⬇️
postgresql-3 49.84% <39.47%> (+0.02%) ⬆️
sqlite-1 45.68% <42.10%> (+0.03%) ⬆️
sqlite-2 48.78% <42.10%> (+0.04%) ⬆️
sqlite-3 49.00% <42.10%> (-0.02%) ⬇️
unit 32.51% <48.64%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@AlekSi AlekSi requested review from a team, henvic, chilagrow and noisersup March 6, 2024 09:50
@AlekSi AlekSi changed the title Batch sizes configurable Make batch sizes configurable Mar 6, 2024
@AlekSi AlekSi added code/enhancement Some user-visible feature could work better community Issues and PRs assigned to community members labels Mar 6, 2024
@AlekSi AlekSi added this to the Next milestone Mar 6, 2024
@AlekSi AlekSi enabled auto-merge (squash) March 6, 2024 16:36
AlekSi
AlekSi reviewed Mar 7, 2024
View reviewed changes
Copy link
Member

@AlekSi AlekSi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests fail because we don't pass BatchSize somewhere, and invalid SQL is generated for INSERT

internal/backends/postgresql/collection.go Show resolved Hide resolved
internal/backends/sqlite/collection.go Show resolved Hide resolved
auto-merge was automatically disabled March 11, 2024 23:27

Head branch was pushed to by a user without write access

@kropidlowsky kropidlowsky requested a review from AlekSi March 13, 2024 23:20
chilagrow
chilagrow reviewed Mar 14, 2024
View reviewed changes
cmd/ferretdb/main.go Outdated Show resolved Hide resolved
@kropidlowsky kropidlowsky requested a review from chilagrow March 14, 2024 11:28
chilagrow
chilagrow previously approved these changes Mar 14, 2024
View reviewed changes
Copy link
Member

@chilagrow chilagrow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution!

@rumyantseva rumyantseva enabled auto-merge (squash) March 19, 2024 10:41
rumyantseva
rumyantseva previously approved these changes Mar 19, 2024
View reviewed changes
Copy link
Contributor

@rumyantseva rumyantseva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for contributing!

@AlekSi AlekSi removed this from the Next milestone Mar 19, 2024
@AlekSi AlekSi added this to the Next milestone Mar 19, 2024
@AlekSi AlekSi dismissed stale reviews from chilagrow and rumyantseva via 7aea53b March 19, 2024 13:56
@AlekSi AlekSi disabled auto-merge March 19, 2024 14:36
@AlekSi AlekSi merged commit 60fcd09 into FerretDB:main Mar 19, 2024
28 of 29 checks passed
@AlekSi
Copy link
Member

AlekSi commented Mar 19, 2024

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@AlekSi AlekSi AlekSi approved these changes

@chilagrow chilagrow chilagrow left review comments

@rumyantseva rumyantseva rumyantseva left review comments

@henvic henvic Awaiting requested review from henvic henvic was automatically assigned from FerretDB/core

@noisersup noisersup Awaiting requested review from noisersup noisersup was automatically assigned from FerretDB/core

Assignees

@kropidlowsky kropidlowsky

Labels
code/enhancement Some user-visible feature could work better community Issues and PRs assigned to community members
Projects
Current Engineering
Archived in project
Milestone
v1.21.0
Development

Successfully merging this pull request may close these issues.

Make batch sizes configurable
4 participants
@kropidlowsky @AlekSi @rumyantseva @chilagrow