Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make batch sizes configurable #4149

Merged
merged 22 commits into from
Mar 19, 2024
Merged

Make batch sizes configurable #4149

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
1bf9901
add batchsize support for Handler
kropidlowsky Feb 27, 2024
b228093
WIP
kropidlowsky Mar 4, 2024
309ccbf
WIP
kropidlowsky Mar 4, 2024
d43898b
fix postgres TestDatabaseStats
kropidlowsky Mar 4, 2024
b277e01
fix sqlite TestCappedCollectionInsertAllQueryExplain
kropidlowsky Mar 4, 2024
2a6219a
fix sqlite TestDatabaseStatsFreeStorage
kropidlowsky Mar 4, 2024
2c51cab
WIP
kropidlowsky Mar 5, 2024
a421052
wip
kropidlowsky Mar 5, 2024
d5f1052
Merge branch 'main' into batch_sizes
kropidlowsky Mar 6, 2024
4700ca4
Merge branch 'main' into batch_sizes
AlekSi Mar 6, 2024
d94737e
Merge branch 'main' into batch_sizes
AlekSi Mar 7, 2024
cae4c98
add panic for `batchSize` at `InsertAll`
kropidlowsky Mar 11, 2024
8913109
Merge branch 'main' into batch_sizes
kropidlowsky Mar 11, 2024
039ebeb
Merge branch 'main' into batch_sizes
kropidlowsky Mar 12, 2024
5092b59
add `batch-size` for integration
kropidlowsky Mar 13, 2024
8099fc2
improove `batch-size` panic message
kropidlowsky Mar 13, 2024
a6f8dfd
Merge branch 'main' into batch_sizes
kropidlowsky Mar 13, 2024
75b8ae6
correct flag help info
kropidlowsky Mar 14, 2024
a9cbcc4
Merge branch 'main' into batch_sizes
kropidlowsky Mar 14, 2024
fb1d058
Merge branch 'main' into batch_sizes
AlekSi Mar 19, 2024
7aea53b
Improve flag description
AlekSi Mar 19, 2024
0a86ac7
Simplify
AlekSi Mar 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add panic for batchSize at InsertAll
  • Loading branch information
@kropidlowsky
kropidlowsky committed Mar 11, 2024
commit cae4c98cb69084747821d6f9d77b4a246e22c76f
3 changes: 3 additions & 0 deletions internal/backends/postgresql/collection.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,9 @@

err = pool.InTransaction(ctx, p, func(tx pgx.Tx) error {
batchSize := c.r.BatchSize
kropidlowsky marked this conversation as resolved.
Show resolved Hide resolved
if batchSize < 1 {
panic("batch-size should be greater than 1")

Check warning on line 139 in internal/backends/postgresql/collection.go

View check run for this annotation

Codecov / codecov/patch

internal/backends/postgresql/collection.go#L139 

Added line #L139 was not covered by tests
}

var batch []*types.Document
docs := params.Docs
Expand Down
3 changes: 3 additions & 0 deletions internal/backends/sqlite/collection.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,9 @@

err := db.InTransaction(ctx, func(tx *fsql.Tx) error {
batchSize := c.r.BatchSize
kropidlowsky marked this conversation as resolved.
Show resolved Hide resolved
if batchSize < 1 {
panic("batch-size should be greater than 1")

Check warning on line 114 in internal/backends/sqlite/collection.go

View check run for this annotation

Codecov / codecov/patch

internal/backends/sqlite/collection.go#L114 

Added line #L114 was not covered by tests
}

var batch []*types.Document
docs := params.Docs
Expand Down
Loading