Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate database and collection names for SQLite handler #2868

Merged
merged 89 commits into from
Jul 26, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
6e3df55
wip
noisersup Jun 20, 2023
8a2af00
wip
noisersup Jun 21, 2023
ae491af
wip
noisersup Jun 21, 2023
3f84105
basic
noisersup Jun 22, 2023
a5ca6cb
remove CollectionToTable from struct
noisersup Jun 22, 2023
88182fb
Replace with collectionGet
noisersup Jun 22, 2023
2009ea8
it works???
noisersup Jun 22, 2023
07738c7
test case
noisersup Jun 23, 2023
004aa8e
wip
noisersup Jun 23, 2023
1178e65
prefix validation
noisersup Jun 23, 2023
019ceb7
Merge branch 'main' into validate-db-coll-2749
noisersup Jun 23, 2023
8c3714d
tweak
noisersup Jun 23, 2023
1012a42
wip
noisersup Jun 23, 2023
7230c38
fix dot for postgres
noisersup Jun 26, 2023
176cb05
fix dot for sqlite
noisersup Jun 26, 2023
f184458
Merge remote-tracking branch 'upstream/main' into validate-db-coll-2749
noisersup Jun 26, 2023
2fbbd64
Update sqlite comment
noisersup Jun 26, 2023
dac0fa9
wip
noisersup Jun 26, 2023
f97a473
add sqlite validation
noisersup Jun 26, 2023
c179a50
add comment + lint
noisersup Jun 27, 2023
4bfe05b
fmt
noisersup Jun 27, 2023
825b11b
Merge branch 'main' into validate-db-coll-2749
noisersup Jun 27, 2023
4d98281
wip
noisersup Jun 27, 2023
0332333
wip
noisersup Jun 27, 2023
257e866
make all table names lowercase
noisersup Jun 27, 2023
7505d9a
proper error handling
noisersup Jun 27, 2023
f05bf62
lot of changes
noisersup Jun 27, 2023
15d7c77
Merge remote-tracking branch 'upstream/main' into validate-db-coll-2749
noisersup Jun 27, 2023
1ef2680
wip
noisersup Jun 27, 2023
a16bb20
fix
noisersup Jun 27, 2023
12a80e4
wip
noisersup Jun 27, 2023
2db636d
wip
noisersup Jun 27, 2023
ef77cfe
wip
noisersup Jun 27, 2023
9463201
Merge branch 'main' into validate-db-coll-2749
Jun 28, 2023
494dcd7
pair programming
noisersup Jun 28, 2023
53518ae
improve prefix check
noisersup Jun 28, 2023
3b95fe1
wip
noisersup Jun 28, 2023
af07a45
Merge branch 'fix-after-bump' into validate-db-coll-2749
noisersup Jun 28, 2023
b3d66ab
Remove test
noisersup Jun 28, 2023
10dcbaa
Merge branch 'fix-after-bump' into validate-db-coll-2749
noisersup Jun 28, 2023
1c2f423
apply review comments
noisersup Jun 28, 2023
8f5667c
Merge branch 'main' into validate-db-coll-2749
Jun 28, 2023
c266454
Merge branch 'main' into validate-db-coll-2749
AlekSi Jun 29, 2023
c5fc06d
Update issue comment
AlekSi Jun 29, 2023
50694df
Simplify
AlekSi Jun 29, 2023
a8bb524
rename CollectionGet
noisersup Jun 29, 2023
c81bc84
remove comment
noisersup Jun 29, 2023
877d7d6
Merge branch 'main' into validate-db-coll-2749
Jun 30, 2023
cb80f54
wip
noisersup Jun 30, 2023
a3b9649
wip
noisersup Jun 30, 2023
7c075bc
wip
noisersup Jun 30, 2023
43c36ae
wip
noisersup Jun 30, 2023
22ff58d
lint
noisersup Jun 30, 2023
0d617a5
Merge branch 'main' into validate-db-coll-2749
Jun 30, 2023
6780ab3
remove unused errorcode
noisersup Jun 30, 2023
50397c1
remove empty collection error from update
noisersup Jun 30, 2023
af88550
Merge remote-tracking branch 'upstream/main' into validate-db-coll-2749
noisersup Jun 30, 2023
1b1a1f4
cleanup
noisersup Jun 30, 2023
122b3fe
Merge branch 'main' into validate-db-coll-2749
AlekSi Jul 2, 2023
6922636
Merge branch 'main' into pr/noisersup/2868-1
AlekSi Jul 14, 2023
88c3c77
Merge branch 'main' into validate-db-coll-2749
AlekSi Jul 18, 2023
5e519bd
Tweak comment
AlekSi Jul 19, 2023
81eb51d
Merge branch 'main' into pr/noisersup/2868
AlekSi Jul 19, 2023
75706fd
Update comments
AlekSi Jul 19, 2023
8346e0c
Merge branch 'main' into pr/noisersup/2868
AlekSi Jul 19, 2023
e4e77bf
Add linter configuration
AlekSi Jul 19, 2023
eaf50a9
Improve tests
AlekSi Jul 19, 2023
d95cca4
Tweaks
AlekSi Jul 19, 2023
5fef6b7
Do not import `commonerrors` in tests
AlekSi Jul 19, 2023
0b96a31
Merge branch 'commonerrors' into pr/noisersup/2868
AlekSi Jul 19, 2023
c875dd1
More work
AlekSi Jul 19, 2023
14e2bff
Add comment
AlekSi Jul 19, 2023
67f909b
Create unique index
AlekSi Jul 19, 2023
a500b14
Merge branch 'main' into validate-db-coll-2749
AlekSi Jul 20, 2023
b6d2961
Make SQLite URL configurable
AlekSi Jul 20, 2023
6a35453
Add validation
AlekSi Jul 20, 2023
a18b8f2
Merge branch 'main' into pr/noisersup/2868
AlekSi Jul 20, 2023
e070733
Merge branch 'main' into validate-db-coll-2749
AlekSi Jul 26, 2023
c5cc1d5
Update comments
AlekSi Jul 26, 2023
be93b2d
Tweak validation
AlekSi Jul 26, 2023
fe79866
Fix number
AlekSi Jul 26, 2023
1344019
Merge branch 'main' into validate-db-coll-2749
AlekSi Jul 26, 2023
1c72efc
Update comment
AlekSi Jul 26, 2023
cc083fa
Update comments
AlekSi Jul 26, 2023
296973f
Tiny tweak
AlekSi Jul 26, 2023
20f8200
Fix tests
AlekSi Jul 26, 2023
0cc1df7
Merge branch 'main' into validate-db-coll-2749
AlekSi Jul 26, 2023
c39f316
Lint
AlekSi Jul 26, 2023
e266e35
Add explicit panic
AlekSi Jul 26, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
lot of changes
  • Loading branch information
noisersup committed Jun 27, 2023
commit f05bf627e5b8b3fdda33ca1534a5d1dcb62c2612
1 change: 1 addition & 0 deletions internal/backends/error.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ const (
ErrorCodeCollectionDoesNotExist
ErrorCodeCollectionAlreadyExists
ErrorCodeCollectionNameIsInvalid
ErrorCodeCollectionStartsWithDot
AlekSi marked this conversation as resolved.
Show resolved Hide resolved
)

// Error represents a backend error returned by all Backend, Database and Collection methods.
Expand Down
3 changes: 2 additions & 1 deletion internal/backends/sqlite/collection.go
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,8 @@ func (c *collection) Query(ctx context.Context, params *backends.QueryParams) (*
// Insert implements backends.Collection interface.
func (c *collection) Insert(ctx context.Context, params *backends.InsertParams) (*backends.InsertResult, error) {
if _, err := c.r.CollectionCreate(ctx, c.dbName, c.name); err != nil {
if errors.Is(err, metadata.ErrInvalidCollectionName) {
if backends.ErrorCodeIs(err, backends.ErrorCodeCollectionNameIsInvalid) {
//TODO
return nil, err
}
return nil, lazyerrors.Error(err)
Expand Down
4 changes: 2 additions & 2 deletions internal/backends/sqlite/database.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ package sqlite

import (
"context"
"errors"

"github.com/FerretDB/FerretDB/internal/backends"
"github.com/FerretDB/FerretDB/internal/backends/sqlite/metadata"
Expand Down Expand Up @@ -72,7 +71,8 @@ func (db *database) ListCollections(ctx context.Context, params *backends.ListCo
func (db *database) CreateCollection(ctx context.Context, params *backends.CreateCollectionParams) error {
created, err := db.r.CollectionCreate(ctx, db.name, params.Name)
if err != nil {
if errors.Is(err, metadata.ErrInvalidCollectionName) {
if backends.ErrorCodeIs(err, backends.ErrorCodeCollectionNameIsInvalid) {
//TODO
return err
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this is need? 🤔

return lazyerrors.Error(err)
Expand Down
15 changes: 4 additions & 11 deletions internal/backends/sqlite/metadata/registry.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ import (
"modernc.org/sqlite"
sqlitelib "modernc.org/sqlite/lib"

"github.com/FerretDB/FerretDB/internal/backends"
"github.com/FerretDB/FerretDB/internal/backends/sqlite/metadata/pool"
"github.com/FerretDB/FerretDB/internal/util/lazyerrors"
"github.com/FerretDB/FerretDB/internal/util/must"
Expand All @@ -45,14 +46,6 @@ const (
metadataTableName = reservedPrefix + "collections"
)

var (
// ErrInvalidCollectionName indicates that a collection didn't pass name checks.
ErrInvalidCollectionName = fmt.Errorf("invalid FerretDB collection name")

// ErrCollectionDoesNotExist indicates that collection does not exist.
ErrCollectionDoesNotExist = fmt.Errorf("collection does not exist")
)

// Registry provides access to SQLite databases and collections information.
type Registry struct {
p *pool.Pool
Expand Down Expand Up @@ -81,7 +74,7 @@ func (r *Registry) Close() {
// It returns ErrInvalidCollectionName error if the name is incorrect for SQLite backend.
func collectionToTable(collectionName string) (string, error) {
w84thesun marked this conversation as resolved.
Show resolved Hide resolved
if strings.HasPrefix(collectionName, reservedPrefix) {
return "", ErrInvalidCollectionName
return "", backends.NewError(backends.ErrorCodeCollectionNameIsInvalid, nil)
}

hash32 := fnv.New32a()
Expand Down Expand Up @@ -223,7 +216,7 @@ func (r *Registry) CollectionGet(ctx context.Context, dbName string, collectionN
defer rows.Close()

if !rows.Next() {
return "", ErrCollectionDoesNotExist
return "", backends.NewError(backends.ErrorCodeCollectionDoesNotExist, nil)
}

var name string
Expand All @@ -245,7 +238,7 @@ func (r *Registry) CollectionDrop(ctx context.Context, dbName string, collection
}

tableName, err := r.CollectionGet(ctx, dbName, collectionName)
if errors.Is(err, ErrCollectionDoesNotExist) {
if backends.ErrorCodeIs(err, backends.ErrorCodeCollectionDoesNotExist) {
return false, nil
rumyantseva marked this conversation as resolved.
Show resolved Hide resolved
chilagrow marked this conversation as resolved.
Show resolved Hide resolved
}

Expand Down
37 changes: 16 additions & 21 deletions internal/handlers/sqlite/msg_create.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,12 @@ package sqlite

import (
"context"
"errors"
"fmt"
"regexp"
"strings"
"unicode/utf8"

"github.com/FerretDB/FerretDB/internal/backends"
"github.com/FerretDB/FerretDB/internal/backends/sqlite/metadata"
"github.com/FerretDB/FerretDB/internal/handlers/common"
"github.com/FerretDB/FerretDB/internal/handlers/commonerrors"
"github.com/FerretDB/FerretDB/internal/types"
Expand Down Expand Up @@ -99,52 +97,49 @@ func (h *Handler) MsgCreate(ctx context.Context, msg *wire.OpMsg) (*wire.OpMsg,

return &reply, nil

case errors.Is(err, ErrInvalidCollectionName):
case backends.ErrorCodeIs(err, backends.ErrorCodeCollectionNameIsInvalid):
msg := fmt.Sprintf("Invalid collection name: '%s.%s'", dbName, collectionName)
return nil, commonerrors.NewCommandErrorMsgWithArgument(commonerrors.ErrInvalidNamespace, msg, "create")

case errors.Is(err, ErrCollectionStartsWithDot):
case backends.ErrorCodeIs(err, backends.ErrorCodeCollectionStartsWithDot):
msg := fmt.Sprintf("Collection names cannot start with '.': %s", collectionName)
return nil, commonerrors.NewCommandErrorMsgWithArgument(commonerrors.ErrInvalidNamespace, msg, "create")

case backends.ErrorCodeIs(err, backends.ErrorCodeCollectionAlreadyExists):
msg := fmt.Sprintf("Collection %s.%s already exists.", dbName, collectionName)
return nil, commonerrors.NewCommandErrorMsgWithArgument(commonerrors.ErrNamespaceExists, msg, "create")

case backends.ErrorCodeIs(err, backends.ErrorCodeCollectionNameIsInvalid):
msg := fmt.Sprintf("Invalid namespace: %s.%s", dbName, collectionName)
return nil, commonerrors.NewCommandErrorMsgWithArgument(commonerrors.ErrInvalidNamespace, msg, "create")

default:
return nil, lazyerrors.Error(err)
}
}

// validateCollectionNameRe validates collection names.
// Empty collection name, names with `$` and `\x00`,
// or exceeding the 255 bytes limit are not allowed.
// Collection names that start with `.` are also not allowed.
var validateCollectionNameRe = regexp.MustCompile("^[^.$\x00][^$\x00]{0,234}$")

// createCollection validates the collection name and creates collection.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's document errors this function can return.

// It returns ErrInvalidCollectionName when the collection name is invalid.
// It returns ErrCollectionStartsWithDot if collection starts with a dot.
func createCollection(ctx context.Context, db backends.Database, collectionName string) error {
AlekSi marked this conversation as resolved.
Show resolved Hide resolved
// validateCollectionNameRe validates collection names.
// Empty collection name, names with `$` and `\x00`,
// or exceeding the 255 bytes limit are not allowed.
// Collection names that start with `.` are also not allowed.
validateCollectionNameRe := regexp.MustCompile("^[^.$\x00][^$\x00]{0,234}$")

if strings.HasPrefix(collectionName, ".") {
return ErrCollectionStartsWithDot
return backends.NewError(backends.ErrorCodeCollectionStartsWithDot, nil)
}

if !validateCollectionNameRe.MatchString(collectionName) ||
!utf8.ValidString(collectionName) {
return ErrInvalidCollectionName
return backends.NewError(backends.ErrorCodeCollectionNameIsInvalid, nil)
}

err := db.CreateCollection(ctx, &backends.CreateCollectionParams{
return db.CreateCollection(ctx, &backends.CreateCollectionParams{
Name: collectionName,
})
if errors.Is(err, metadata.ErrInvalidCollectionName) {
return ErrInvalidCollectionName
}

return err
//if errors.Is(err, metadata.ErrInvalidCollectionName) {
// return ErrInvalidCollectionName
//}

//return err
}
6 changes: 3 additions & 3 deletions internal/handlers/sqlite/msg_update.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ import (
"errors"

"github.com/FerretDB/FerretDB/internal/backends"
"github.com/FerretDB/FerretDB/internal/backends/sqlite/metadata"
"github.com/FerretDB/FerretDB/internal/handlers/common"
"github.com/FerretDB/FerretDB/internal/types"
"github.com/FerretDB/FerretDB/internal/util/iterator"
Expand Down Expand Up @@ -77,8 +76,9 @@ func (h *Handler) updateDocument(ctx context.Context, params *common.UpdatesPara
err = nil
}

if errors.Is(err, metadata.ErrInvalidCollectionName) {
return 0, 0, nil, ErrInvalidCollectionName
if backends.ErrorCodeIs(err, backends.ErrorCodeCollectionNameIsInvalid) {
//TODO: check if its needed
return 0, 0, nil, err
}

if err != nil {
Expand Down
10 changes: 0 additions & 10 deletions internal/handlers/sqlite/sqlite.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,6 @@
package sqlite

import (
"fmt"

"github.com/prometheus/client_golang/prometheus"
"go.uber.org/zap"

Expand All @@ -32,14 +30,6 @@ import (
"github.com/FerretDB/FerretDB/internal/util/state"
)

var (
// ErrInvalidCollectionName indicates that a collection didn't pass name checks.
ErrInvalidCollectionName = fmt.Errorf("invalid FerretDB collection name")

// ErrCollectionStartsWithDot indicates that collection name starts with dot, but it shouldn't.
ErrCollectionStartsWithDot = fmt.Errorf("Collection names cannot start with '.'")
)

// notImplemented returns error for stub command handlers.
func notImplemented(command string) error {
return commonerrors.NewCommandErrorMsg(
Expand Down