Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(eqx query cosmos): Add -sn, -o, -P, -C, -m raw #444

Merged
merged 5 commits into from
Feb 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,9 @@ The `Unreleased` section name is replaced by the expected version of next releas
- `EventStore`: Revise test rig to target a Docker-hosted cluster [#317](https://github.com/jet/equinox/pull/317)
- `EventStoreDb`: As per `EventStore` module, but using the modern `EventStore.Client.Grpc.Streams` client [#196](https://github.com/jet/equinox/pull/196)
- `MessageDb`: Implements a [message-db](http://docs.eventide-project.org/user-guide/message-db/) storage backend [#339](https://github.com/jet/equinox/pull/339) with OpenTelemetry tracing and snapshotting support [#348](https://github.com/jet/equinox/pull/348) :pray: [@nordfjord](https://github.com/nordfjord)
- `eqx init`: `-U/--indexunfolds` enables querying uncompressed unfolds (see `shouldCompress`) [#434](https://github.com/jet/equinox/pull/434)
- `eqx query`: Queries based on uncompressed unfolds (see `eqx init -U`) [#434](https://github.com/jet/equinox/pull/434)
- `eqx init --indexUnfolds cosmos`: enables querying uncompressed unfolds (see `shouldCompress`) [#434](https://github.com/jet/equinox/pull/434)
- `eqx query cosmos`: Queries based on uncompressed unfolds (see `eqx init -U`) [#434](https://github.com/jet/equinox/pull/434)
- `eqx query -m raw -o FILEPATH cosmos`: Allows capture of raw JSON to a file [#444](https://github.com/jet/equinox/pull/444)
- `eqx dump`: `-s` flag is now optional
- `eqx stats`: `-A` flag to request all stats (equivalent to requesting `-ESD`) [#424](https://github.com/jet/equinox/pull/424)

Expand Down
32 changes: 28 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -390,22 +390,46 @@ While Equinox is implemented in F#, and F# is a great fit for writing event-sour

# use a wild card (LIKE) for the stream name
eqx query -cl '$Us%' -un Snapshotted cosmos -d db -c $EQUINOX_COSMOS_VIEWS -b 100000
# > Querying SELECT c.u, c.p, c._etag FROM c WHERE c.p LIKE "$Us%" AND EXISTS (SELECT VALUE u FROM u IN c.u WHERE u.c = "Snapshotted") {}
# > Querying Default: SELECT c.u, c.p, c._etag FROM c WHERE c.p LIKE "$Us%" AND EXISTS (SELECT VALUE u FROM u IN c.u WHERE u.c = "Snapshotted") {}
# > Page 7166s, 7166u, 0e 320.58RU 3.9s {}
# > Page 1608s, 1608u, 0e 68.59RU 0.9s {}
# > TOTALS 1c, 8774s, 389.17RU 4.7s {}

# Skip loading the _etag to simulate a query where you will only render the result (not `Transact` against it)
eqx query -cn '$User' -un Snapshotted cosmos -d db -c $EQUINOX_COSMOS_VIEWS -b 100000
# > Querying SELECT c.u FROM c WHERE c.p LIKE "$User-%" AND EXISTS (SELECT VALUE u FROM u IN c.u WHERE u.c = "Snapshotted") {}
eqx query -cn '$User' -m readOnly -un Snapshotted cosmos -d db -c $EQUINOX_COSMOS_VIEWS -b 100000
# > Querying ReadOnly: SELECT c.u FROM c WHERE c.p LIKE "$User-%" AND EXISTS (SELECT VALUE u FROM u IN c.u WHERE u.c = "Snapshotted") {}
# > Page 8774s, 8774u, 0e 342.33RU 3.8s {}
# > TOTALS 0c, 8774s, 342.33RU 3.8s {} # 👈 cheaper and only one batch as no .p or ._etag

# add criteria filtering based on an Uncompressed Unfold
eqx query -cn '$User' -un EmailIndex -uc 'u.d.email = "a@b.com"' cosmos -d db -c $EQUINOX_COSMOS_VIEWS -b 100000
# > Querying SELECT c.u, c.p, c._etag FROM c WHERE c.p LIKE "$User-%" AND EXISTS (SELECT VALUE u FROM u IN c.u WHERE u.c = "EmailIndex" AND u.d.email = "a@b.com") {}
# > Querying Default: SELECT c.u, c.p, c._etag FROM c WHERE c.p LIKE "$User-%" AND EXISTS (SELECT VALUE u FROM u IN c.u WHERE u.c = "EmailIndex" AND u.d.email = "a@b.com") {}
# > Page 0s, 0u, 0e 2.8RU 0.7s {}
# > TOTALS 0c, 0s, 2.80RU 0.7s {} # 👈 only 2.8RU if nothing is returned

# DUMP ONE STREAM TO A FILE (equivalent to queries performed by CosmosStore.AccessStrategy.Unoptimized)
# Can be imported into another store via https://github.com/jet/dotnet-templates/blob/master/propulsion-indexer
eqx query -m readOnly -sn 'user-f28fb6feea00550e93ca77b6f29899cd' -o dump-user.json cosmos -d db -c $EQUINOX_COSMOS_CONTAINER -b 9999
# > Dumping Raw content to ./dump-user.json {}
# > Querying Raw: SELECT * FROM c WHERE c.p = "user-f28fb6feea00550e93ca77b6f29899cd" AND 1=1 {}
# > Page 9s, 1u, 10e 3.23RU 0.5s 0.0MiB age 0002.10:04:13 {}
# > TOTALS 1c, 9s, 3.23RU R/W 0.0/0.0MiB 3.9s {}

# DUMP FULL CONTENT OF THE CONTAINER TO A FILE
# Can be imported into another store via https://github.com/jet/dotnet-templates/blob/master/propulsion-indexer
eqx query -m raw -o ../dump-240216.json -m raw cosmos -d db -c $EQUINOX_COSMOS_CONTAINER -b 9999
# > Dumping Raw content to ~/dumps/dump-240216.json {}
# > No StreamName or CategoryName/CategoryLike specified - Unfold Criteria better be unambiguous {}
# > Querying Raw: SELECT * FROM c WHERE 1=1 AND 1=1 {}
# > Page 2972s, 748u, 3112e 108.9RU 3.8s 4.0MiB age 0212.18:00:45 {}
# > Page 3211s, 777u, 3161e 112.29RU 3.3s 4.0MiB age 0212.09:06:02 {}
# > Page 3003s, 663u, 3172e 110.33RU 3.4s 4.0MiB age 0211.04:09:12 {}
# <chop>
# > Page 2768s, 498u, 3153e 107.46RU 3.0s 4.0MiB age 0016.13:09:02 {}
# > Page 2806s, 505u, 3198e 107.17RU 3.0s 4.0MiB age 0010.18:52:45 {}
# > Page 2903s, 601u, 3188e 107.53RU 3.1s 4.0MiB age 0004.05:24:51 {}
# > Page 2638s, 316u, 3019e 93.09RU 2.5s 3.4MiB age 0000.05:08:38 {}
# > TOTALS 11c, 206,356s, 7,886.75RU R/W 290.4/290.4MiB 225.3s {}
```

6. Use `propulsion` tool to run a CosmosDB ChangeFeedProcessor
Expand Down
2 changes: 1 addition & 1 deletion src/Equinox.Tests/CachingTests.fs
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,7 @@ type Tests() =
let! struct (_token, state) = allowStale 250 // Delay of 90 overlapped with delay of 50+10 should not have expired the entry (was 200)
test <@ (4, 1, 3) = (state, cat.Loads, cat.Reloads) @> // The newer cache entry won
cat.Delay <- TimeSpan.FromMilliseconds 10 // Reduce the delay, but we do want to overlap a write
let t4 = allowStale 200 // Delay of 75 in load2/load3 should not have aged the read result beyond 200
let t4 = allowStale 300 // Delay of 75 in load2/load3 should not have aged the read result beyond 200
do! Task.Delay 10 // ensure delay has been picked up, before...
cat.Delay <- TimeSpan.Zero // no further delays required for the rest of the tests

Expand Down
Loading