Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use one implementation for finding path values #3087

Merged
merged 58 commits into from
Aug 4, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
bce806c
create initial common implementation
chilagrow Jul 20, 2023
c4073da
rename
chilagrow Jul 20, 2023
d0c0334
distinct uses new impl
chilagrow Jul 20, 2023
33936ff
use new function in filter
chilagrow Jul 20, 2023
aea192d
some cleaning
chilagrow Jul 20, 2023
4594b67
update doc
chilagrow Jul 20, 2023
3a693bd
re-organise filter pair code
chilagrow Jul 20, 2023
1ef1ef9
handle empty key for filter
chilagrow Jul 20, 2023
bee2816
add tests, comments and refactor
chilagrow Jul 21, 2023
d1b8ff0
tidy up
chilagrow Jul 21, 2023
2e0eff7
Merge branch 'main' into issue-2348-get-path-value
chilagrow Jul 21, 2023
9e188b5
fix merge
chilagrow Jul 21, 2023
5ce3f80
fix merge
chilagrow Jul 21, 2023
7b65633
reorganise
chilagrow Jul 21, 2023
876744d
Merge branch 'main' into issue-2348-get-path-value
mergify[bot] Jul 24, 2023
292d897
Merge branch 'main' into issue-2348-get-path-value
mergify[bot] Jul 24, 2023
d0be65d
Merge branch 'main' into issue-2348-get-path-value
mergify[bot] Jul 24, 2023
cc4af16
Merge branch 'main' into issue-2348-get-path-value
mergify[bot] Jul 24, 2023
15aaf03
Merge branch 'main' into issue-2348-get-path-value
mergify[bot] Jul 24, 2023
faf7daa
Merge branch 'main' into issue-2348-get-path-value
mergify[bot] Jul 24, 2023
ae4f67d
Refactor `types.Path` a bit
AlekSi Jul 24, 2023
fc446a3
Small tweaks
AlekSi Jul 24, 2023
0313d17
update test
chilagrow Jul 26, 2023
51c106b
merge
chilagrow Jul 26, 2023
1d5051f
checkpoint
chilagrow Jul 26, 2023
20c464a
Tweak documentation a little bit
AlekSi Jul 26, 2023
aa74542
another checkpoint
chilagrow Jul 26, 2023
7bf0bdd
Merge branch 'issue-2348-get-path-value' of github.com:chilagrow/Ferr…
chilagrow Jul 26, 2023
789bae3
some update
chilagrow Jul 26, 2023
2885c85
more comment
chilagrow Jul 27, 2023
76f0724
projection returns correct error code
chilagrow Jul 27, 2023
b31332f
fmt
chilagrow Jul 27, 2023
85f509d
Merge branch 'main' into issue-2348-get-path-value
chilagrow Jul 27, 2023
74d4fc1
path does not return error on $ prefix
chilagrow Jul 27, 2023
ca376f4
Revert "projection returns correct error code"
chilagrow Jul 27, 2023
8147dfe
add todo for fixing projection positional operator
chilagrow Jul 27, 2023
e2fc0ff
aggregation projection checks valid path
chilagrow Jul 27, 2023
bac5eeb
add todo
chilagrow Jul 27, 2023
9f19a76
fmt
chilagrow Jul 27, 2023
8a41dc4
Merge branch 'main' into issue-2348-get-path-value
chilagrow Jul 27, 2023
d7c16a0
Revert "aggregation projection checks valid path"
chilagrow Jul 28, 2023
7e49bd4
review comment
chilagrow Jul 28, 2023
3811585
renaming
chilagrow Jul 28, 2023
bc74089
test empty and space keys for insert and find
chilagrow Jul 28, 2023
1316e0c
Revert "test empty and space keys for insert and find"
chilagrow Jul 28, 2023
ea14b20
clean up
chilagrow Jul 28, 2023
b047e14
Merge branch 'main' into issue-2348-get-path-value
AlekSi Aug 1, 2023
e00bd95
update expression documentation
chilagrow Aug 1, 2023
9c4899c
Merge branch 'issue-2348-get-path-value' of github.com:chilagrow/Ferr…
chilagrow Aug 1, 2023
b706f91
update comment
chilagrow Aug 1, 2023
aa3fecd
Merge branch 'main' into issue-2348-get-path-value
chilagrow Aug 2, 2023
f2f0106
update expression comments
chilagrow Aug 2, 2023
b93ea91
more comments
chilagrow Aug 2, 2023
8b205c2
from pairing session
chilagrow Aug 2, 2023
7c0c76f
add more comments
chilagrow Aug 3, 2023
fd19a7a
more update of inline comments
chilagrow Aug 3, 2023
76a63cf
minor comment update
chilagrow Aug 3, 2023
307792e
resolve merge conflict
chilagrow Aug 4, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
more comment
  • Loading branch information
chilagrow committed Jul 27, 2023
commit 2885c85a0ec5d2e2d77ca6fb1b8bbaff46fd3868
4 changes: 2 additions & 2 deletions internal/handlers/common/aggregations/expression.go
Original file line number Diff line number Diff line change
Expand Up @@ -79,8 +79,8 @@ type Expression struct {
func NewExpression(expression string, opts *commonpath.FindValuesOpts) (*Expression, error) {
if opts == nil {
opts = &commonpath.FindValuesOpts{
FindArrayIndex: false,
FindArrayValues: true,
FindArrayIndex: false,
FindArrayDocuments: true,
}
}

Expand Down
4 changes: 2 additions & 2 deletions internal/handlers/common/aggregations/stages/unwind.go
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,8 @@ func newUnwind(stage *types.Document) (aggregations.Stage, error) {
}

expr, err = aggregations.NewExpression(field, &commonpath.FindValuesOpts{
FindArrayIndex: false,
FindArrayValues: false,
FindArrayIndex: false,
FindArrayDocuments: false,
})

if err != nil {
Expand Down
4 changes: 2 additions & 2 deletions internal/handlers/common/distinct.go
Original file line number Diff line number Diff line change
Expand Up @@ -109,8 +109,8 @@ func FilterDistinctValues(iter types.DocumentsIterator, key string) (*types.Arra

// vals contains all values exist at the suffix of the path
vals, err := commonpath.FindValues(doc, path, &commonpath.FindValuesOpts{
FindArrayIndex: true,
FindArrayValues: true,
FindArrayIndex: true,
FindArrayDocuments: true,
})
if err != nil {
return nil, lazyerrors.Error(err)
Expand Down
4 changes: 2 additions & 2 deletions internal/handlers/common/filter.go
Original file line number Diff line number Diff line change
Expand Up @@ -109,8 +109,8 @@ func filterDocumentPair(doc *types.Document, filterKey string, filterValue any)
} else {
filterSuffix = path.Suffix()
vals, err = commonpath.FindValues(doc, path, &commonpath.FindValuesOpts{
FindArrayIndex: true,
FindArrayValues: true,
FindArrayIndex: true,
FindArrayDocuments: true,
})
if err != nil {
return false, lazyerrors.Error(err)
Expand Down
74 changes: 37 additions & 37 deletions internal/handlers/commonpath/commonpath.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,28 +26,31 @@ import (

// FindValuesOpts sets options for FindValues.
type FindValuesOpts struct {
// If FindArrayValues is true, it iterates the array to find documents containing the key.
// If FindArrayValues is false, it does not find any value from the array.
// If FindArrayDocuments is true, it iterates the array to find documents that have path.
// If FindArrayDocuments is false, it does not find documents from the array.
// Using path `v.foo` and `v` is an array:
// - with FindArrayValues true, it returns documents' values of key `foo`;
// - with FindArrayValues false, it returns an empty array.
// If `v` is not an array, FindArrayValues has no impact.
FindArrayValues bool
// If FindArrayIndex is true, it uses indexes to find a value of the array.
// If FindArrayIndex is false, it does use indexes on the path.
// - with FindArrayDocuments true, it finds values of `foo` of found documents;
// - with FindArrayDocuments false, it returns an empty array.
// If `v` is not an array, FindArrayDocuments has no impact.
FindArrayDocuments bool
// If FindArrayIndex is true, it finds value at index of an array.
// If FindArrayIndex is false, it does not find value at index of an array.
// Using path `v.0` and `v` is an array:
// - with FindArrayIndex true, it returns 0-th index value of the array;
// - with FindArrayIndex true, it finds 0-th index value of the array;
// - with FindArrayIndex false, it returns empty array.
// If `v` is not an array, FindArrayIndex has no impact.
FindArrayIndex bool
chilagrow marked this conversation as resolved.
Show resolved Hide resolved
}

// FindValues iterates path elements, at each path element it adds to next values to iterate:
// - if it is a document and has the key;
// - if it is an array, FindArrayIndex is true and finds value at index;
// - if it is an array, FindArrayValues is true and finds values.
// FindValues returns values found at path. Unlike document.GetByPath it may find multiple values.
//
// It returns values or an empty array.
// It iterates path elements, at each path element it adds to next values to iterate:
// - if it is a document and has path, it adds the document field value to next values;
// - if it is an array, FindArrayIndex is true and finds value at index, it adds value to next values;
// - if it is an array, FindArrayDocuments is true and documents in the array have path,
// it adds field value of all documents that have path to next values.
//
// It returns next values after iterating path elements.
func FindValues(doc *types.Document, path types.Path, opts *FindValuesOpts) ([]any, error) {
if opts == nil {
opts = new(FindValuesOpts)
Expand All @@ -71,15 +74,14 @@ func FindValues(doc *types.Document, path types.Path, opts *FindValuesOpts) ([]a
case *types.Array:
if opts.FindArrayIndex {
res, err := findArrayIndex(next, p)
if err != nil {
return nil, lazyerrors.Error(err)
if err == nil {
values = append(values, res)
continue
}

values = append(values, res...)
}

if opts.FindArrayValues {
res, err := findArrayValues(next, p)
if opts.FindArrayDocuments {
res, err := findArrayDocuments(next, p)
if err != nil {
return nil, lazyerrors.Error(err)
}
Expand All @@ -98,28 +100,27 @@ func FindValues(doc *types.Document, path types.Path, opts *FindValuesOpts) ([]a
return nextValues, nil
}

// findArrayIndex checks if key can be used as an index of array and returns
// the value found at that index.
// Empty array is returned if the key cannot be used as an index
// or key is not an existing index of array.
func findArrayIndex(array *types.Array, key string) ([]any, error) {
index, err := strconv.Atoi(key)
// findArrayIndex returns the value found at array index.
// Error is returned if index is not a number or index does not exist in array.
func findArrayIndex(array *types.Array, index string) (any, error) {
i, err := strconv.Atoi(index)
if err != nil {
return []any{}, nil
return nil, lazyerrors.Error(err)
}

// key is an integer, check if that integer is an index of the array
v, _ := array.Get(index)
if v != nil {
return []any{v}, nil
v, err := array.Get(i)
if err != nil {
return nil, lazyerrors.Error(err)
}

return []any{}, nil
return v, nil
}

// findArrayValues finds document fields containing the key and returns the document field value.
// Empty array is returned if no document containing the key was found.
func findArrayValues(array *types.Array, key string) ([]any, error) {
// findArrayDocuments iterates array to find documents that have documentKey.
// It returns field values of found documents at documentKey.
// Multiple documents with documentKey may exist in array, hence an array is returned.
// Empty array is returned if no document was found.
func findArrayDocuments(array *types.Array, documentKey string) ([]any, error) {
iter := array.Iterator()
defer iter.Close()

Expand All @@ -140,8 +141,7 @@ func findArrayValues(array *types.Array, key string) ([]any, error) {
continue
}

v, _ = doc.Get(key)
if v != nil {
if v, _ = doc.Get(documentKey); v != nil {
res = append(res, v)
}
}
Expand Down
64 changes: 32 additions & 32 deletions internal/handlers/commonpath/commonpath_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,107 +42,107 @@ func TestGetPathValue(t *testing.T) {
doc: array,
path: types.NewStaticPath("foo", "1", "bar"),
opts: &FindValuesOpts{
FindArrayIndex: true,
FindArrayValues: true,
FindArrayIndex: true,
FindArrayDocuments: true,
},
res: []any{1},
},
"PathNested2": {
doc: array,
path: types.NewStaticPath("foo", "1", "bar"),
opts: &FindValuesOpts{
FindArrayIndex: true,
FindArrayValues: false,
FindArrayIndex: true,
FindArrayDocuments: false,
},
res: []any{1},
},
"PathNested3": {
doc: array,
path: types.NewStaticPath("foo", "1", "bar"),
opts: &FindValuesOpts{
FindArrayIndex: false,
FindArrayValues: true,
FindArrayIndex: false,
FindArrayDocuments: true,
},
res: []any{},
},
"PathNested4": {
doc: array,
path: types.NewStaticPath("foo", "1", "bar"),
opts: &FindValuesOpts{
FindArrayIndex: false,
FindArrayValues: false,
FindArrayIndex: false,
FindArrayDocuments: false,
},
res: []any{},
},
"PathIndexDotNotation1": {
doc: array,
path: types.NewStaticPath("foo", "1"),
opts: &FindValuesOpts{
FindArrayIndex: true,
FindArrayValues: true,
FindArrayIndex: true,
FindArrayDocuments: true,
},
res: []any{must.NotFail(types.NewDocument("bar", 1))},
},
"PathIndexDotNotation2": {
doc: array,
path: types.NewStaticPath("foo", "1"),
opts: &FindValuesOpts{
FindArrayIndex: true,
FindArrayValues: false,
FindArrayIndex: true,
FindArrayDocuments: false,
},
res: []any{must.NotFail(types.NewDocument("bar", 1))},
},
"PathIndexDotNotation3": {
doc: array,
path: types.NewStaticPath("foo", "1"),
opts: &FindValuesOpts{
FindArrayIndex: false,
FindArrayValues: true,
FindArrayIndex: false,
FindArrayDocuments: true,
},
res: []any{},
},
"PathIndexDotNotation4": {
doc: array,
path: types.NewStaticPath("foo", "1"),
opts: &FindValuesOpts{
FindArrayIndex: false,
FindArrayValues: false,
FindArrayIndex: false,
FindArrayDocuments: false,
},
res: []any{},
},
"PathDotNotation1": {
doc: array,
path: types.NewStaticPath("foo", "bar"),
opts: &FindValuesOpts{
FindArrayIndex: true,
FindArrayValues: true,
FindArrayIndex: true,
FindArrayDocuments: true,
},
res: []any{0, 1},
},
"PathDotNotation2": {
doc: array,
path: types.NewStaticPath("foo", "bar"),
opts: &FindValuesOpts{
FindArrayIndex: true,
FindArrayValues: false,
FindArrayIndex: true,
FindArrayDocuments: false,
},
res: []any{},
},
"PathDotNotation3": {
doc: array,
path: types.NewStaticPath("foo", "bar"),
opts: &FindValuesOpts{
FindArrayIndex: false,
FindArrayValues: true,
FindArrayIndex: false,
FindArrayDocuments: true,
},
res: []any{0, 1},
},
"PathDotNotation4": {
doc: array,
path: types.NewStaticPath("foo", "bar"),
opts: &FindValuesOpts{
FindArrayIndex: false,
FindArrayValues: false,
FindArrayIndex: false,
FindArrayDocuments: false,
},
res: []any{},
},
Expand Down Expand Up @@ -176,35 +176,35 @@ func TestGetPathValue(t *testing.T) {
doc: doc,
path: types.NewStaticPath("foo", "bar"),
opts: &FindValuesOpts{
FindArrayIndex: true,
FindArrayValues: true,
FindArrayIndex: true,
FindArrayDocuments: true,
},
res: []any{0},
},
"PathDotNotation2": {
doc: doc,
path: types.NewStaticPath("foo", "bar"),
opts: &FindValuesOpts{
FindArrayIndex: true,
FindArrayValues: false,
FindArrayIndex: true,
FindArrayDocuments: false,
},
res: []any{0},
},
"PathDotNotation3": {
doc: doc,
path: types.NewStaticPath("foo", "bar"),
opts: &FindValuesOpts{
FindArrayIndex: false,
FindArrayValues: true,
FindArrayIndex: false,
FindArrayDocuments: true,
},
res: []any{0},
},
"PathDotNotation4": {
doc: doc,
path: types.NewStaticPath("foo", "bar"),
opts: &FindValuesOpts{
FindArrayIndex: false,
FindArrayValues: false,
FindArrayIndex: false,
FindArrayDocuments: false,
},
res: []any{0},
},
Expand Down