Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make batch sizes configurable #4149

Merged
merged 22 commits into from
Mar 19, 2024
Merged

Make batch sizes configurable #4149

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
1bf9901
add batchsize support for Handler
kropidlowsky Feb 27, 2024
b228093
WIP
kropidlowsky Mar 4, 2024
309ccbf
WIP
kropidlowsky Mar 4, 2024
d43898b
fix postgres TestDatabaseStats
kropidlowsky Mar 4, 2024
b277e01
fix sqlite TestCappedCollectionInsertAllQueryExplain
kropidlowsky Mar 4, 2024
2a6219a
fix sqlite TestDatabaseStatsFreeStorage
kropidlowsky Mar 4, 2024
2c51cab
WIP
kropidlowsky Mar 5, 2024
a421052
wip
kropidlowsky Mar 5, 2024
d5f1052
Merge branch 'main' into batch_sizes
kropidlowsky Mar 6, 2024
4700ca4
Merge branch 'main' into batch_sizes
AlekSi Mar 6, 2024
d94737e
Merge branch 'main' into batch_sizes
AlekSi Mar 7, 2024
cae4c98
add panic for `batchSize` at `InsertAll`
kropidlowsky Mar 11, 2024
8913109
Merge branch 'main' into batch_sizes
kropidlowsky Mar 11, 2024
039ebeb
Merge branch 'main' into batch_sizes
kropidlowsky Mar 12, 2024
5092b59
add `batch-size` for integration
kropidlowsky Mar 13, 2024
8099fc2
improove `batch-size` panic message
kropidlowsky Mar 13, 2024
a6f8dfd
Merge branch 'main' into batch_sizes
kropidlowsky Mar 13, 2024
75b8ae6
correct flag help info
kropidlowsky Mar 14, 2024
a9cbcc4
Merge branch 'main' into batch_sizes
kropidlowsky Mar 14, 2024
fb1d058
Merge branch 'main' into batch_sizes
AlekSi Mar 19, 2024
7aea53b
Improve flag description
AlekSi Mar 19, 2024
0a86ac7
Simplify
AlekSi Mar 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add batch-size for integration
  • Loading branch information
@kropidlowsky
kropidlowsky committed Mar 13, 2024
commit 5092b596578a60b5b9a8aa62c5a732c997f83a42
3 changes: 3 additions & 0 deletions Taskfile.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,7 @@ tasks:
-postgresql-url='postgres://username@127.0.0.1:5432/ferretdb?search_path='
-compat-url='mongodb://username:password@127.0.0.1:47018/?tls=true&tlsCertificateKeyFile=../build/certs/client.pem&tlsCaFile=../build/certs/rootCA-cert.pem&replicaSet=rs0'
-disable-pushdown={{.DISABLE_PUSHDOWN}}
-batch-size=100

test-integration-sqlite:
desc: "Run integration tests for `sqlite` backend"
Expand All @@ -244,6 +245,7 @@ tasks:
-target-tls
-compat-url='mongodb://username:password@127.0.0.1:47018/?tls=true&tlsCertificateKeyFile=../build/certs/client.pem&tlsCaFile=../build/certs/rootCA-cert.pem&replicaSet=rs0'
-disable-pushdown={{.DISABLE_PUSHDOWN}}
-batch-size=1000

test-integration-mysql:
desc: "Run integration tests for `mysql` handler"
Expand Down Expand Up @@ -364,6 +366,7 @@ tasks:
--handler=pg
--postgresql-url='postgres://username@127.0.0.1:5432/ferretdb?search_path='
--test-records-dir=tmp/records
--test-batch-size=100

run-sqlite:
desc: "Run FerretDB with `sqlite` backend"
Expand Down
1 change: 1 addition & 0 deletions integration/setup/listener.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,7 @@ func setupListener(tb testtb.TB, ctx context.Context, logger *zap.Logger) string
CappedCleanupPercentage: 20,
CappedCleanupInterval: 0,
EnableNewAuth: true,
BatchSize: *batchSizeF,
},
}
h, closeBackend, err := registry.NewHandler(handler, handlerOpts)
Expand Down
2 changes: 2 additions & 0 deletions integration/setup/setup.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ var (
mysqlURLF = flag.String("mysql-url", "", "in-process FerretDB: MySQL URL for 'mysql' handler.")
hanaURLF = flag.String("hana-url", "", "in-process FerretDB: Hana URL for 'hana' handler.")

batchSizeF = flag.Int("batch-size", 100, "number of maximum size of query parameters")

compatURLF = flag.String("compat-url", "", "compat system's (MongoDB) URL for compatibility tests; if empty, they are skipped")

benchDocsF = flag.Int("bench-docs", 1000, "benchmarks: number of documents to generate per iteration")
Expand Down