Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve fuzzing with recorded seed data #2392

Merged
merged 174 commits into from
May 4, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
174 commits
Select commit Hold shift + click to select a range
487914d
WIP
fenogentov Apr 10, 2023
447f024
Update writing guide (#2373)
Fashander Apr 6, 2023
cb328c6
Fix version.txt generation for git tags (#2388)
AlekSi Apr 7, 2023
f5b5c0a
Add comments to our documentation workflow (#2390)
AlekSi Apr 7, 2023
b3b431d
Disable integration tests for Tigris for now
AlekSi Apr 10, 2023
e23198f
Fix types order linter (#2391)
AlekSi Apr 11, 2023
478912e
Add blogpost: Announcing FerretDB 1.0 GA - a truly Open Source MongoD…
AlekSi Apr 11, 2023
070e3bc
Deprecated error cleanup (#2411)
chilagrow Apr 12, 2023
646f5e5
Bump deps (#2408)
AlekSi Apr 12, 2023
63001bb
Fix `findAndModify` for `$exists` query operator (#2385)
chilagrow Apr 12, 2023
54c7a03
Update documentation for index options (#2417)
AlekSi Apr 12, 2023
e5cbda3
Parallel tests are consistently used in integration tests (#2409)
chilagrow Apr 12, 2023
f0d90cb
Update 2023-04-11-ferretdb-1-0-ga-opensource-mongodb-alternative.md
ptrfarkas Apr 12, 2023
298d6c8
Update 2023-04-11-ferretdb-1-0-ga-opensource-mongodb-alternative.md
ptrfarkas Apr 12, 2023
eb0cd70
Add query pushdown documentation (#2339)
noisersup Apr 12, 2023
d65804b
Update README.md to link to SSPL (#2420)
cooljeanius Apr 13, 2023
e5c49b4
Compress CI artifacts (#2424)
noisersup Apr 13, 2023
1d43c77
Update README.md
ptrfarkas Apr 13, 2023
be00196
Use exponential backoff with jitter (#2419)
j0holo Apr 14, 2023
2fd13be
Improve documentation for Docker (#2396)
AlekSi Apr 14, 2023
8da0258
Add Mergify rules for blog posts (#2434)
AlekSi Apr 14, 2023
14a3a8c
Add more detailed PR guides in CONTRIBUTING.md (#2435)
AuruTus Apr 16, 2023
0656ccf
Migrate to `pgx/v5` (#2439)
Apr 17, 2023
91ae7ae
Remove a few double spaces (#2431)
AlekSi Apr 17, 2023
ba793f5
Make it harder to misuse iterators (#2428)
AlekSi Apr 17, 2023
afc828e
Update PR template (#2441)
AlekSi Apr 17, 2023
131e418
Add image for a future blog post (#2453)
AlekSi Apr 17, 2023
0e16b61
Add blogpost - Using FerretDB with Studio 3T (#2454)
AlekSi Apr 17, 2023
be92f42
Fix YAML indentation (#2455)
AlekSi Apr 17, 2023
22545b5
Rename testing flag (#2437)
AlekSi Apr 17, 2023
9f961dd
Fix version.txt generation for git tags (#2388)
AlekSi Apr 7, 2023
749bd53
Add comments to our documentation workflow (#2390)
AlekSi Apr 7, 2023
f1206a8
Fix types order linter (#2391)
AlekSi Apr 11, 2023
f87f9b2
Add blogpost: Announcing FerretDB 1.0 GA - a truly Open Source MongoD…
AlekSi Apr 11, 2023
f8c9dcf
Deprecated error cleanup (#2411)
chilagrow Apr 12, 2023
f56033f
Bump deps (#2408)
AlekSi Apr 12, 2023
ae4da41
Fix `findAndModify` for `$exists` query operator (#2385)
chilagrow Apr 12, 2023
99b8218
Update documentation for index options (#2417)
AlekSi Apr 12, 2023
786c874
Update 2023-04-11-ferretdb-1-0-ga-opensource-mongodb-alternative.md
ptrfarkas Apr 12, 2023
665c51b
Update 2023-04-11-ferretdb-1-0-ga-opensource-mongodb-alternative.md
ptrfarkas Apr 12, 2023
89d01b8
Update README.md
ptrfarkas Apr 13, 2023
c593e9f
Use exponential backoff with jitter (#2419)
j0holo Apr 14, 2023
c73bcdb
Improve documentation for Docker (#2396)
AlekSi Apr 14, 2023
0e6b2e0
Add Mergify rules for blog posts (#2434)
AlekSi Apr 14, 2023
a9673ed
Migrate to `pgx/v5` (#2439)
Apr 17, 2023
3eceb1c
Remove a few double spaces (#2431)
AlekSi Apr 17, 2023
b212c4d
Add blogpost - Using FerretDB with Studio 3T (#2454)
AlekSi Apr 17, 2023
cc512fc
Fix YAML indentation (#2455)
AlekSi Apr 17, 2023
7601cb8
Rename testing flag (#2437)
AlekSi Apr 17, 2023
7796faf
`-race` is not supported on `riscv64` (#2456)
afiskon Apr 17, 2023
ca7c197
Update blog post on Using FerretDB with Studio3T (#2457)
Fashander Apr 17, 2023
b4dab52
Validate `scale` param for `dbStats` and `collStats` correctly (#2418)
Apr 18, 2023
9f14c0f
Minor cleanup of exponential backoff with jitter (#2443)
chilagrow Apr 18, 2023
ace05a3
Add workaround for CockroachDB issue (#2464)
AlekSi Apr 18, 2023
852c4c4
Fix `SchemaStats` to return correct data (#2426)
Apr 18, 2023
cdfae4e
Implement blog posts previews (#2433)
noisersup Apr 18, 2023
92475b9
Introduce integration benchmarks (#2381)
noisersup Apr 18, 2023
1d1c429
Add tests to findAndModify on `$exists` operator (#2422)
chilagrow Apr 19, 2023
b593980
Use separate environments for blog and docs
AlekSi Apr 19, 2023
5ad1b06
Bump deps (#2479)
AlekSi Apr 19, 2023
11b3f41
Refactor aggregation (#2463)
chilagrow Apr 19, 2023
e1013cc
Small documentation tweaks (#2452)
AlekSi Apr 19, 2023
11ddcf3
Fix query projection for top level fields (#2386)
Apr 19, 2023
6db24d6
Restore Go caches for packages building
AlekSi Apr 19, 2023
2a208ee
Fix version.txt generation for git tags (#2388)
AlekSi Apr 7, 2023
ab64b36
Fix types order linter (#2391)
AlekSi Apr 11, 2023
090753c
Add blogpost: Announcing FerretDB 1.0 GA - a truly Open Source MongoD…
AlekSi Apr 11, 2023
1f0db78
Deprecated error cleanup (#2411)
chilagrow Apr 12, 2023
1a088c4
Bump deps (#2408)
AlekSi Apr 12, 2023
020e9b1
Fix `findAndModify` for `$exists` query operator (#2385)
chilagrow Apr 12, 2023
7f9c858
Update documentation for index options (#2417)
AlekSi Apr 12, 2023
d1c0d0a
Update 2023-04-11-ferretdb-1-0-ga-opensource-mongodb-alternative.md
ptrfarkas Apr 12, 2023
0ff22b1
Update 2023-04-11-ferretdb-1-0-ga-opensource-mongodb-alternative.md
ptrfarkas Apr 12, 2023
25f46b7
Update README.md
ptrfarkas Apr 13, 2023
692a964
Use exponential backoff with jitter (#2419)
j0holo Apr 14, 2023
15e3003
Improve documentation for Docker (#2396)
AlekSi Apr 14, 2023
5d66b2a
Add Mergify rules for blog posts (#2434)
AlekSi Apr 14, 2023
7d11780
Add more detailed PR guides in CONTRIBUTING.md (#2435)
AuruTus Apr 16, 2023
efcc60d
Migrate to `pgx/v5` (#2439)
Apr 17, 2023
1ca7768
Remove a few double spaces (#2431)
AlekSi Apr 17, 2023
33c420f
Make it harder to misuse iterators (#2428)
AlekSi Apr 17, 2023
11aeee3
Add blogpost - Using FerretDB with Studio 3T (#2454)
AlekSi Apr 17, 2023
59379d4
Fix YAML indentation (#2455)
AlekSi Apr 17, 2023
51c7b93
Rename testing flag (#2437)
AlekSi Apr 17, 2023
5b347ad
Update blog post on Using FerretDB with Studio3T (#2457)
Fashander Apr 17, 2023
cb2dabe
Validate `scale` param for `dbStats` and `collStats` correctly (#2418)
Apr 18, 2023
798bfc9
Minor cleanup of exponential backoff with jitter (#2443)
chilagrow Apr 18, 2023
4e844b1
Fix `SchemaStats` to return correct data (#2426)
Apr 18, 2023
fe2359c
Implement blog posts previews (#2433)
noisersup Apr 18, 2023
b8eb21c
Introduce integration benchmarks (#2381)
noisersup Apr 18, 2023
fd9eda7
Add tests to findAndModify on `$exists` operator (#2422)
chilagrow Apr 19, 2023
3a2def7
Use separate environments for blog and docs
AlekSi Apr 19, 2023
6b685f4
Bump deps (#2479)
AlekSi Apr 19, 2023
1ad40bb
Small documentation tweaks (#2452)
AlekSi Apr 19, 2023
220ff8b
Fix query projection for top level fields (#2386)
Apr 19, 2023
9758cb8
Fix version.txt generation for git tags (#2388)
AlekSi Apr 7, 2023
4d4e6de
Add comments to our documentation workflow (#2390)
AlekSi Apr 7, 2023
39b39b9
Fix types order linter (#2391)
AlekSi Apr 11, 2023
4e108fa
Add blogpost: Announcing FerretDB 1.0 GA - a truly Open Source MongoD…
AlekSi Apr 11, 2023
fcdb2cb
Deprecated error cleanup (#2411)
chilagrow Apr 12, 2023
346092d
Bump deps (#2408)
AlekSi Apr 12, 2023
5134f75
Fix `findAndModify` for `$exists` query operator (#2385)
chilagrow Apr 12, 2023
75c30c9
Update documentation for index options (#2417)
AlekSi Apr 12, 2023
c6722d0
Update 2023-04-11-ferretdb-1-0-ga-opensource-mongodb-alternative.md
ptrfarkas Apr 12, 2023
57b3970
Update 2023-04-11-ferretdb-1-0-ga-opensource-mongodb-alternative.md
ptrfarkas Apr 12, 2023
eba34c6
Update README.md
ptrfarkas Apr 13, 2023
96efb33
Use exponential backoff with jitter (#2419)
j0holo Apr 14, 2023
8933f88
Improve documentation for Docker (#2396)
AlekSi Apr 14, 2023
7b3a8c7
Add Mergify rules for blog posts (#2434)
AlekSi Apr 14, 2023
8202f19
Add more detailed PR guides in CONTRIBUTING.md (#2435)
AuruTus Apr 16, 2023
c7336c7
Migrate to `pgx/v5` (#2439)
Apr 17, 2023
4bad3d4
Remove a few double spaces (#2431)
AlekSi Apr 17, 2023
0cf3991
Make it harder to misuse iterators (#2428)
AlekSi Apr 17, 2023
ffecfaf
Add blogpost - Using FerretDB with Studio 3T (#2454)
AlekSi Apr 17, 2023
8e155a7
Fix YAML indentation (#2455)
AlekSi Apr 17, 2023
1ab86f8
Rename testing flag (#2437)
AlekSi Apr 17, 2023
5eeeda2
Update blog post on Using FerretDB with Studio3T (#2457)
Fashander Apr 17, 2023
94f931c
Validate `scale` param for `dbStats` and `collStats` correctly (#2418)
Apr 18, 2023
2c34685
Minor cleanup of exponential backoff with jitter (#2443)
chilagrow Apr 18, 2023
6c09a0a
Fix `SchemaStats` to return correct data (#2426)
Apr 18, 2023
df5a335
Implement blog posts previews (#2433)
noisersup Apr 18, 2023
9aeeb71
Introduce integration benchmarks (#2381)
noisersup Apr 18, 2023
5a7d14f
Add tests to findAndModify on `$exists` operator (#2422)
chilagrow Apr 19, 2023
7f29b9a
Use separate environments for blog and docs
AlekSi Apr 19, 2023
611f9b2
Bump deps (#2479)
AlekSi Apr 19, 2023
1fd8c22
Small documentation tweaks (#2452)
AlekSi Apr 19, 2023
d7122a6
Fix query projection for top level fields (#2386)
Apr 19, 2023
7fb45f8
Fix version.txt generation for git tags (#2388)
AlekSi Apr 7, 2023
debf50b
Add comments to our documentation workflow (#2390)
AlekSi Apr 7, 2023
a241f41
Fix types order linter (#2391)
AlekSi Apr 11, 2023
1a1bd31
Add blogpost: Announcing FerretDB 1.0 GA - a truly Open Source MongoD…
AlekSi Apr 11, 2023
313269e
Deprecated error cleanup (#2411)
chilagrow Apr 12, 2023
4eeb0e3
Bump deps (#2408)
AlekSi Apr 12, 2023
c04a7bd
Fix `findAndModify` for `$exists` query operator (#2385)
chilagrow Apr 12, 2023
290e9f9
Update documentation for index options (#2417)
AlekSi Apr 12, 2023
49743ea
Update 2023-04-11-ferretdb-1-0-ga-opensource-mongodb-alternative.md
ptrfarkas Apr 12, 2023
c054136
Update 2023-04-11-ferretdb-1-0-ga-opensource-mongodb-alternative.md
ptrfarkas Apr 12, 2023
34d1b99
Update README.md
ptrfarkas Apr 13, 2023
235b9ea
Use exponential backoff with jitter (#2419)
j0holo Apr 14, 2023
d1c268d
Improve documentation for Docker (#2396)
AlekSi Apr 14, 2023
f9604b0
Add Mergify rules for blog posts (#2434)
AlekSi Apr 14, 2023
cd58816
Add more detailed PR guides in CONTRIBUTING.md (#2435)
AuruTus Apr 16, 2023
d503d04
Migrate to `pgx/v5` (#2439)
Apr 17, 2023
7a40a5d
Remove a few double spaces (#2431)
AlekSi Apr 17, 2023
43c7ff8
Make it harder to misuse iterators (#2428)
AlekSi Apr 17, 2023
611937b
Add blogpost - Using FerretDB with Studio 3T (#2454)
AlekSi Apr 17, 2023
dfe99ae
Fix YAML indentation (#2455)
AlekSi Apr 17, 2023
54455db
Rename testing flag (#2437)
AlekSi Apr 17, 2023
006cdbb
Update blog post on Using FerretDB with Studio3T (#2457)
Fashander Apr 17, 2023
ce60768
Validate `scale` param for `dbStats` and `collStats` correctly (#2418)
Apr 18, 2023
96db0b8
Minor cleanup of exponential backoff with jitter (#2443)
chilagrow Apr 18, 2023
d9b7ddd
Fix `SchemaStats` to return correct data (#2426)
Apr 18, 2023
0f8c74d
Implement blog posts previews (#2433)
noisersup Apr 18, 2023
64806bc
Introduce integration benchmarks (#2381)
noisersup Apr 18, 2023
04c62e8
Add tests to findAndModify on `$exists` operator (#2422)
chilagrow Apr 19, 2023
da2c820
Use separate environments for blog and docs
AlekSi Apr 19, 2023
d198625
Bump deps (#2479)
AlekSi Apr 19, 2023
57ad088
Small documentation tweaks (#2452)
AlekSi Apr 19, 2023
33f7805
Fix query projection for top level fields (#2386)
Apr 19, 2023
ef3574a
Merge branch 'main' into issues-2100-ImproveFuzzing
fenogentov Apr 20, 2023
d778922
WIP
fenogentov Apr 21, 2023
6e367ff
Merge branch 'main' into issues-2100-ImproveFuzzing
fenogentov Apr 21, 2023
dea668f
Merge branch 'main' into issues-2100-ImproveFuzzing
fenogentov Apr 22, 2023
9d31229
Merge branch 'main' into issues-2100-ImproveFuzzing
fenogentov Apr 25, 2023
31ca4f5
wip
fenogentov Apr 25, 2023
ec2779c
Merge branch 'main' into issues-2100-ImproveFuzzing
fenogentov Apr 27, 2023
db78dd4
added selection of random files
fenogentov Apr 27, 2023
cf26c2a
Merge branch 'main' into issues-2100-ImproveFuzzing
fenogentov May 3, 2023
f1a1d21
Merge branch 'main' into issues-2100-ImproveFuzzing
fenogentov May 4, 2023
ec2800f
fixed random file selection
fenogentov May 4, 2023
b1777ff
fix test-records-dir
fenogentov May 4, 2023
7db9382
fix
fenogentov May 4, 2023
acffbbf
Merge branch 'main' into issues-2100-ImproveFuzzing
fenogentov May 4, 2023
20eb8ac
применяется метод 'hex.EncodeToString'
fenogentov May 4, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add query pushdown documentation (#2339)
Closes #1956.
  • Loading branch information
noisersup authored and fenogentov committed Apr 17, 2023
commit eb0cd7093446d153595c59c43bf47a1211f36f06
2 changes: 1 addition & 1 deletion website/docs/configuration/_category_.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
label: Configuration
position: 6
position: 7
link:
type: generated-index
description: >
Expand Down
2 changes: 1 addition & 1 deletion website/docs/contributing/_category_.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
label: Contributing to FerretDB
position: 11
position: 12
link:
type: generated-index
description: >
Expand Down
2 changes: 1 addition & 1 deletion website/docs/diff.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 7
sidebar_position: 8
slug: /diff/ # referenced in README.md and beacon
---

Expand Down
52 changes: 52 additions & 0 deletions website/docs/pushdown.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
---
sidebar_position: 6
hide_table_of_contents: true
---

# Query pushdown

**Query pushdown** is the method of optimizing a query by reducing the amount of data read and processed.
It saves memory space, network bandwidth, and reduces the query execution time by moving some parts
of the query execution closer to the data source.

Initially FerretDB retrieved all data related to queried collection, and applies filters on its own, making
it possible to implement complex logic safely and quickly.
To make this process more efficient, we minimize the amount of incoming data, by applying WHERE clause on SQL queries.

:::info
You can learn more about query pushdown in our [blog post](https://blog.ferretdb.io/ferretdb-fetches-data-query-pushdown/).
:::

## Supported types and operators

The following table shows all operators and types that FerretDB pushdowns on PostgreSQL backend.
If filter uses type and operator, that's marked as pushdown-supported on this list,
FerretDB will prefetch less data, resulting with more performent query.

If your application requires better performance for specific operation,
feel free to share this with us in our [community](/#community)!

:::tip
As query pushdown allows developers to implement query optimizations separately from the features,
the table will be updated frequently.
:::

<!-- markdownlint-capture -->
<!-- markdownlint-disable MD001 MD033 MD051 -->
| | Object | Array | Double | String | Binary | ObjectID | Boolean | Date | Null | Regex | Integer | Timestamp | Long |
| ---------- | -------- | ------- | ---------------------- | -------- | -------- | ---------- | --------- | ------ | ------ | ------- | --------- | ----------- | ---------------------- |
| `=` | ✖️ | ✖️ | ⚠️ <sub>[[1]](#1)</sub> | ✅ | ✖️ | ✅ | ✅ | ✅ | ✖️ | ✖️ | ✅ | ✖️ | ⚠️ <sub>[[1]](#1)</sub> |
| `$eq` | ✖️ | ✖️ | ⚠️ <sub>[[1]](#1)</sub> | ✅ | ✖️ | ✅ | ✅ | ✅ | ✖️ | ✖️ | ✅ | ✖️ | ⚠️ <sub>[[1]](#1)</sub> |
| `$gt` | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ |
| `$gte` | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ |
| `$lt` | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ |
| `$lte` | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ |
| `$in` | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ |
| `$ne` | ✖️ | ✖️ | ⚠️ <sub>[[1]](#1)</sub> | ✅ | ✖️ | ✅ | ✅ | ✅ | ✖️ | ✖️ | ✅ | ✖️ | ⚠️ <sub>[[1]](#1)</sub> |
| `$nin` | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ |

###### [1] {#1}

Numbers outside the range of the safe IEEE 754 precision (`< -9007199254740991.0, 9007199254740991.0 >`),
will prefetch all numbers larger/smaller than max/min value of the range.
<!-- markdownlint-restore -->
2 changes: 1 addition & 1 deletion website/docs/reference/_category_.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
label: Reference
position: 10
position: 11
link:
type: generated-index
description: >
Expand Down
2 changes: 1 addition & 1 deletion website/docs/security.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 9
sidebar_position: 10
slug: /security/ # referenced in README.md
description: TLS and authentication
---
Expand Down
2 changes: 1 addition & 1 deletion website/docs/telemetry.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 8
sidebar_position: 9
slug: /telemetry/ # referenced in many places; must not change
---

Expand Down