Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support InsertAll and FindAll for capped collections #3610

Merged
merged 42 commits into from
Oct 29, 2023
Merged

Support InsertAll and FindAll for capped collections #3610

merged 42 commits into from
Oct 29, 2023

Conversation

chilagrow
Copy link
Member

@chilagrow chilagrow commented Oct 19, 2023 •
edited
Loading

Description

Closes #3490.

Readiness checklist

  • I added/updated unit tests (and they pass).
  • I added/updated integration/compatibility tests (and they pass).
  • I added/updated comments and checked rendering.
  • I made spot refactorings.
  • I updated user documentation.
  • I ran task all, and it passed.
  • I ensured that PR title is good enough for the changelog.
  • (for maintainers only) I set Reviewers (@FerretDB/core), Milestone (Next), Labels, Project and project's Sprint fields.
  • I marked all done items in this checklist.

@chilagrow chilagrow added the code/feature Some user-visible feature is not implemented yet label Oct 19, 2023
@chilagrow chilagrow self-assigned this Oct 19, 2023
@codecov
Copy link

codecov bot commented Oct 19, 2023 •
edited
Loading

Codecov Report

Merging #3610 (7a1efe6) into main (e180e61) will increase coverage by 0.04%.
The diff coverage is 90.36%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3610      +/-   ##
==========================================
+ Coverage   73.93%   73.97%   +0.04%     
==========================================
  Files         369      370       +1     
  Lines       23462    23564     +102     
==========================================
+ Hits        17346    17431      +85     
- Misses       5087     5100      +13     
- Partials     1029     1033       +4
Files Coverage Δ
integration/pushdown.go 72.00% <100.00%> (+10.88%) ⬆️
internal/backends/collection.go 100.00% <ø> (ø)
internal/backends/postgresql/metadata/metadata.go 63.63% <100.00%> (+0.96%) ⬆️
internal/backends/postgresql/query.go 92.77% <100.00%> (+7.50%) ⬆️
internal/backends/sqlite/collection.go 80.46% <100.00%> (+1.01%) ⬆️
internal/backends/sqlite/metadata/metadata.go 100.00% <100.00%> (ø)
internal/backends/sqlite/query.go 100.00% <100.00%> (ø)
internal/util/fsql/rows.go 86.95% <100.00%> (+1.24%) ⬆️
internal/backends/postgresql/collection.go 71.91% <96.00%> (+5.00%) ⬆️
internal/backends/postgresql/query_iterator.go 87.80% <78.12%> (+1.13%) ⬆️
... and 1 more

... and 8 files with indirect coverage changes

Flag Coverage Δ
filter-true 70.06% <77.71%> (-0.03%) ⬇️
hana-1 ?
integration 70.06% <77.71%> (-0.03%) ⬇️
mongodb-1 5.29% <0.00%> (-0.03%) ⬇️
postgresql-1 50.84% <27.10%> (-0.37%) ⬇️
postgresql-2 50.21% <42.77%> (+2.18%) ⬆️
postgresql-3 49.10% <27.10%> (-1.73%) ⬇️
sort-false 70.06% <77.71%> (-0.03%) ⬇️
sqlite-1 50.12% <25.30%> (-0.39%) ⬇️
sqlite-2 49.49% <37.95%> (+2.13%) ⬆️
sqlite-3 48.36% <25.30%> (-1.71%) ⬇️
unit 28.89% <86.16%> (+1.53%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

@chilagrow chilagrow added this to the Next milestone Oct 20, 2023
@chilagrow chilagrow marked this pull request as ready for review October 20, 2023 08:55
@chilagrow chilagrow requested review from AlekSi and a team as code owners October 20, 2023 08:55
@chilagrow chilagrow requested a review from rumyantseva October 20, 2023 08:55
@chilagrow chilagrow enabled auto-merge (squash) October 20, 2023 08:55
@chilagrow chilagrow requested review from a team and noisersup October 20, 2023 08:55
AlekSi
AlekSi reviewed Oct 24, 2023
View reviewed changes
internal/backends/collection.go Outdated Show resolved Hide resolved
internal/backends/collection_test.go Outdated Show resolved Hide resolved
internal/backends/postgresql/query_iterator.go Outdated Show resolved Hide resolved
@chilagrow chilagrow requested a review from AlekSi October 25, 2023 03:09
@chilagrow chilagrow mentioned this pull request Oct 25, 2023
Merged
9 tasks
AlekSi
AlekSi reviewed Oct 25, 2023
View reviewed changes
integration/pushdown.go Show resolved Hide resolved
internal/backends/collection.go Outdated Show resolved Hide resolved
internal/backends/collection.go Outdated Show resolved Hide resolved
internal/backends/postgresql/collection.go Outdated Show resolved Hide resolved
internal/backends/postgresql/query.go Outdated Show resolved Hide resolved
internal/backends/postgresql/query.go Outdated Show resolved Hide resolved
internal/backends/postgresql/query_iterator.go Outdated Show resolved Hide resolved
internal/backends/collection_test.go Outdated Show resolved Hide resolved
internal/backends/collection_test.go Outdated Show resolved Hide resolved
@chilagrow chilagrow requested a review from AlekSi October 26, 2023 04:14
rumyantseva
rumyantseva previously approved these changes Oct 26, 2023
View reviewed changes
Copy link
Contributor

@rumyantseva rumyantseva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, to me, the code looks clear and easy to read. I added two questions, but I approve because these questions can be handled in a separate PR (if it's needed at all).

@chilagrow chilagrow requested a review from rumyantseva October 27, 2023 03:28
@chilagrow
Copy link
Member Author

integration test with ENABLE_SORT_PUSHDOWN

$ task test-integration-postgresql ENABLE_SORT_PUSHDOWN=true TEST_RUN=TestQueryCappedCollectionCompat
task: [gen-version] go generate -x ./build/version
go run ./generate.go
package.txt: build-host
commit.txt: 75b6fcdbcb49529b0c29d5a538d859f766e54d84
branch.txt: issue-3490-insert-find
version.txt: v1.12.1-102-g75b6fcdb
task: [gen-version] go build -v -o bin/ ./cmd/envtool/
task: [test-integration-postgresql] ../bin/envtool tests run --shard-index=1 --shard-total=1 --run='TestQueryCappedCollectionCompat' -- -count=1 -timeout=35m -race=true -tags=ferretdb_debug,ferretdb_hana -shuffle=on -coverpkg=../... -coverprofile=integration-postgresql.txt . -target-backend=ferretdb-postgresql -target-tls -postgresql-url='postgres://username@127.0.0.1:5432/ferretdb?search_path=' -compat-url='mongodb://username:password@127.0.0.1:47018/?tls=true&tlsCertificateKeyFile=../build/certs/client.pem&tlsCaFile=../build/certs/rootCA-cert.pem' -disable-filter-pushdown=false -enable-sort-pushdown=true

ok      github.com/FerretDB/FerretDB/integration        1.659s  coverage: 18.9% of statements in ../...
$ task test-integration-sqlite ENABLE_SORT_PUSHDOWN=true TEST_RUN=TestQueryCappedCollectionCompat
task: [gen-version] go generate -x ./build/version
go run ./generate.go
package.txt: build-host
branch.txt: issue-3490-insert-find
commit.txt: 75b6fcdbcb49529b0c29d5a538d859f766e54d84
version.txt: v1.12.1-102-g75b6fcdb
task: [gen-version] go build -v -o bin/ ./cmd/envtool/
task: [test-integration-sqlite] ../bin/envtool tests run --shard-index=1 --shard-total=1 --run='TestQueryCappedCollectionCompat' -- -count=1 -timeout=35m -race=true -tags=ferretdb_debug,ferretdb_hana -shuffle=on -coverpkg=../... -coverprofile=integration-sqlite.txt . -target-backend=ferretdb-sqlite -sqlite-url=file:../tmp/sqlite-tests/ -target-tls -compat-url='mongodb://username:password@127.0.0.1:47018/?tls=true&tlsCertificateKeyFile=../build/certs/client.pem&tlsCaFile=../build/certs/rootCA-cert.pem' -disable-filter-pushdown=false -enable-sort-pushdown=true

ok      github.com/FerretDB/FerretDB/integration        1.492s  coverage: 18.0% of statements in ../...

rumyantseva
rumyantseva approved these changes Oct 27, 2023
View reviewed changes
Copy link
Contributor

@rumyantseva rumyantseva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

noisersup
noisersup approved these changes Oct 27, 2023
View reviewed changes
Copy link
Member

@noisersup noisersup left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

integration/pushdown.go Show resolved Hide resolved
@noisersup
Copy link
Member

Just verified and resolved some older comments.

@chilagrow chilagrow merged commit ce26a01 into FerretDB:main Oct 29, 2023
34 of 40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@noisersup noisersup noisersup approved these changes

@rumyantseva rumyantseva rumyantseva approved these changes

@AlekSi AlekSi AlekSi approved these changes

Assignees

@chilagrow chilagrow

Labels
code/feature Some user-visible feature is not implemented yet
Projects
Current Engineering
Archived in project
Milestone
v1.14.0
Development

Successfully merging this pull request may close these issues.

Support InsertAll and FindAll for capped collections
4 participants
@chilagrow @noisersup @AlekSi @rumyantseva