Skip to content

Commit

Permalink
Refactor integration tests (#1925)
Browse files Browse the repository at this point in the history 
Closes #1568.
  • Loading branch information
@AlekSi
AlekSi authored Feb 21, 2023
1 parent 32a7bc5 commit 102b27d
Show file tree
Hide file tree
Showing 21 changed files with 533 additions and 598 deletions.
32 changes: 24 additions & 8 deletions CONTRIBUTING.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -145,16 +145,17 @@ you can run those with `task test-unit` after starting the environment as descri

We also have a set of "integration" tests in the `integration` directory.
They use the Go MongoDB driver like a regular user application.
They could test target any MongoDB-compatible database (such as FerretDB or MongoDB itself) via a regular TCP port.
They also could test target in-process FerretDB instances
They could test any MongoDB-compatible database (such as FerretDB or MongoDB itself) via a regular TCP or TLS port or Unix socket.
They also could test in-process FerretDB instances
(meaning that integration tests start and stop them themselves) with a given handler.
Some tests (so-called compatibility or "compat" tests) connect to two systems ("target" and "compat") at the same time,
Finally, some tests (so-called compatibility or "compat" tests) connect to two systems
("target" for FerretDB and "compat" for MongoDB) at the same time,
send the same queries to both, and compare results.
You can run them with:

* `task test-integration-pg` for in-process FerretDB with `pg` handler and MongoDB on port 37017 (as in our development environment);
* `task test-integration-tigris` for in-process FerretDB with `tigris` handler and MongoDB on port 37017;
* `task test-integration-mongodb` for MongoDB running on port 37017 only;
* `task test-integration-pg` for in-process FerretDB with `pg` handler and MongoDB;
* `task test-integration-tigris` for in-process FerretDB with `tigris` handler and MongoDB;
* `task test-integration-mongodb` for MongoDB only, skipping compat tests;
* or `task test-integration` to run all in parallel.

You may run all tests in parallel with `task test`.
Expand All @@ -167,9 +168,11 @@ you may also use all standard `go` tool facilities,
including [`GOFLAGS` environment variable](https://pkg.go.dev/cmd/go#hdr-Environment_variables).
For example:

* to run a single test case for `pg` handler with all subtests running sequentially,
* to run a single test case for in-process FerretDB with `pg` handler
with all subtests running sequentially,
you may use `env GOFLAGS='-run=TestName/TestCaseName -parallel=1' task test-integration-pg`;
* to run all tests for `tigris` handler with [Go execution tracer](https://pkg.go.dev/runtime/trace) enabled,
* to run all tests for in-process FerretDB with `tigris` handler
with [Go execution tracer](https://pkg.go.dev/runtime/trace) enabled,
you may use `env GOFLAGS='-trace=trace.out' task test-integration-tigris`.

(It is not recommended to set `GOFLAGS` and other Go environment variables with `export GOFLAGS=...`
Expand Down Expand Up @@ -207,6 +210,19 @@ Some of our idiosyncrasies:
The order of `case`s follows this order: <https://pkg.go.dev/github.com/FerretDB/FerretDB/internal/types#hdr-Mapping>
It may seem random, but it is only pseudo-random and follows BSON spec: <https://bsonspec.org/spec.html>

#### Integration tests conventions

We prefer our integration tests to be straightforward,
branchless (with a few, if any, `if` and `switch` statements),
and backend-independent.
Ideally, the same test should work for both FerretDB with all handlers and MongoDB.
If that's impossible without some branching, use helpers exported from the `setup` package,
such us `IsTigris`, `SkipForTigrisWithReason`, `TigrisOnlyWithReason`.
The bar for using other ways of branching, such as checking error codes and messages, is very high.
Writing separate tests might be much better than making a single test that checks error text.

Also, we should use driver methods as much as possible instead of testing commands directly via `RunCommand`.

### Submitting code changes

Before submitting a pull request, please make sure that:
Expand Down
27 changes: 15 additions & 12 deletions Taskfile.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ vars:
FUZZTIME: 15s
FUZZCORPUS: ../fuzz-corpus
RACEFLAG: -race={{and (ne OS "windows") (ne ARCH "arm")}}
UNIXSOCKETFLAG: -target-unix-socket={{ne OS "windows"}}
BUILDTAGS: ferretdb_debug,ferretdb_tigris,ferretdb_hana
SERVICES: postgres postgres_secured tigris tigris1 tigris2 tigris3 tigris4 mongodb mongodb_secured jaeger

Expand Down Expand Up @@ -175,9 +176,11 @@ tasks:
- >
go test -count=1 {{.RACEFLAG}} -tags={{.BUILDTAGS}} -shuffle=on -coverpkg=../...
-coverprofile=integration-pg.txt .
-target-backend=ferretdb-pg
{{.UNIXSOCKETFLAG}}
-postgresql-url=postgres://username@127.0.0.1:5432/ferretdb?pool_min_conns=1
-compat-url=mongodb://127.0.0.1:47017/
-records-dir=../records
-target-unix-socket
test-integration-tigris:
desc: "Run integration tests for Tigris handler"
Expand All @@ -186,11 +189,11 @@ tasks:
- >
go test -count=1 {{.RACEFLAG}} -tags={{.BUILDTAGS}} -shuffle=on -coverpkg=../...
-coverprofile=integration-tigris.txt .
-target-backend=ferretdb-tigris
-target-tls
-tigris-urls=127.0.0.1:8081,127.0.0.1:8091,127.0.0.1:8092,127.0.0.1:8093,127.0.0.1:8094
-compat-url='mongodb://username:password@127.0.0.1:47018/?tls=true&tlsCertificateKeyFile=../build/certs/client.pem&tlsCaFile=../build/certs/rootCA-cert.pem'
-records-dir=../records
-target-tls
-compat-port=37018
-compat-tls
test-integration-mongodb:
desc: "Run integration tests for MongoDB"
Expand All @@ -199,9 +202,9 @@ tasks:
- >
go test -count=1 {{.RACEFLAG}} -tags={{.BUILDTAGS}} -shuffle=on -coverpkg=../...
-coverprofile=integration-mongodb.txt .
-compat-port=0
-target-url=mongodb://127.0.0.1:47017/
-target-backend=mongodb
-records-dir=../records
-target-port=37017
bench-short:
desc: "Benchmark for about 25 seconds (with default BENCHTIME)"
Expand Down Expand Up @@ -254,7 +257,7 @@ tasks:
--listen-addr=:27017
--mode=diff-normal
--postgresql-url="postgres://username@127.0.0.1:5432/ferretdb"
--proxy-addr=127.0.0.1:37017
--proxy-addr=127.0.0.1:47017
--test-records-dir=records
run-secured:
Expand All @@ -266,7 +269,7 @@ tasks:
--listen-addr=''
--listen-tls-cert-file=./build/certs/server-cert.pem
--listen-tls-key-file=./build/certs/server-key.pem
--listen-tls-ca-file=./build/certs/rootCA.pem
--listen-tls-ca-file=./build/certs/rootCA-cert.pem
--listen-tls=:27018
--mode=normal
--postgresql-url="postgres://127.0.0.1:5433/ferretdb"
Expand All @@ -283,7 +286,7 @@ tasks:
--listen-tls-key-file=./build/certs/server-key.pem
--listen-tls=:27018
--mode=diff-normal
--proxy-addr=127.0.0.1:37017
--proxy-addr=127.0.0.1:47017
--test-records-dir=records
run-proxy:
Expand All @@ -298,7 +301,7 @@ tasks:
--listen-tls=:27018
--mode=diff-proxy
--postgresql-url="postgres://username@127.0.0.1:5432/ferretdb"
--proxy-addr=127.0.0.1:37017
--proxy-addr=127.0.0.1:47017
--test-records-dir=records
lint:
Expand Down Expand Up @@ -360,7 +363,7 @@ tasks:
cmds:
- >
docker compose exec mongodb mongosh
--tlsCertificateKeyFile=/etc/certs/client.pem --tlsCAFile=/etc/certs/rootCA.pem
--tlsCertificateKeyFile=/etc/certs/client.pem --tlsCAFile=/etc/certs/rootCA-cert.pem
'mongodb://username:password@host.docker.internal:27018/?authMechanism=PLAIN&tls=true&heartbeatFrequencyMS=300000'
--verbose --eval 'disableTelemetry()' --shell
Expand All @@ -370,7 +373,7 @@ tasks:
- >
docker compose run --rm legacy-mongo-shell
--tlsCertificateKeyFile /etc/certs/server.pem
--tlsCAFile /etc/certs/rootCA.pem
--tlsCAFile /etc/certs/rootCA-cert.pem
--tls
'mongodb://username:password@host.docker.internal:27017/?authMechanism=PLAIN'
/legacy-mongo-shell/test.js
Expand Down
0 build/certs/rootCA.pem → build/certs/rootCA-cert.pem
View file
File renamed without changes.
2 changes: 1 addition & 1 deletion build/mongod_secured.conf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ net:
tls:
mode: requireTLS
certificateKeyFile: /etc/certs/server.pem
CAFile: /etc/certs/rootCA.pem
CAFile: /etc/certs/rootCA-cert.pem
security:
# that also enables authentication
authorization: enabled
4 changes: 2 additions & 2 deletions cmd/envtool/envtool.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -212,11 +212,11 @@ func setup(ctx context.Context, logger *zap.SugaredLogger) error {
return err
}

if err := waitForPort(ctx, logger.Named("mongodb"), 37017); err != nil {
if err := waitForPort(ctx, logger.Named("mongodb"), 47017); err != nil {
return err
}

if err := waitForPort(ctx, logger.Named("mongodb_secure"), 37018); err != nil {
if err := waitForPort(ctx, logger.Named("mongodb_secure"), 47018); err != nil {
return err
}

Expand Down
4 changes: 2 additions & 2 deletions docker-compose.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ services:
container_name: ferretdb_mongodb
command: --config /etc/mongod.conf
ports:
- 37017:27017
- 47017:27017
extra_hosts:
- "host.docker.internal:host-gateway"
environment:
Expand All @@ -111,7 +111,7 @@ services:
container_name: ferretdb_mongodb_secured
command: --config /etc/mongod.conf
ports:
- 37018:27017
- 47018:27017
extra_hosts:
- "host.docker.internal:host-gateway"
environment:
Expand Down
6 changes: 3 additions & 3 deletions ferretdb/ferretdb_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ func Example_tcp() {
func Example_unix() {
f, err := ferretdb.New(&ferretdb.Config{
Listener: ferretdb.ListenerConfig{
Unix: "/tmp/ferretdb-27017.sock",
Unix: "/tmp/ferretdb.sock",
},
Handler: "pg",
PostgreSQLURL: "postgres://127.0.0.1:5432/ferretdb",
Expand All @@ -90,13 +90,13 @@ func Example_unix() {
cancel()
<-done

// Output: mongodb://%2Ftmp%2Fferretdb-27017.sock/
// Output: mongodb://%2Ftmp%2Fferretdb.sock/
}

func Example_tls() {
certPath := filepath.Join("..", "build", "certs", "server-cert.pem")
keyPath := filepath.Join("..", "build", "certs", "server-key.pem")
caPath := filepath.Join("..", "build", "certs", "rootCA.pem")
caPath := filepath.Join("..", "build", "certs", "rootCA-cert.pem")

f, err := ferretdb.New(&ferretdb.Config{
Listener: ferretdb.ListenerConfig{
Expand Down
5 changes: 3 additions & 2 deletions integration/Taskfile.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ vars:
tasks:
env-data:
cmds:
# TODO https://github.com/FerretDB/FerretDB/issues/1568
- >
go test -count=1 {{.RACEFLAG}} -run=TestEnvData -v
-tags=ferretdb_testenvdata .
Expand All @@ -31,13 +32,13 @@ tasks:
# -tags=ferretdb_testenvdata .
# -records-dir=../records
# -compat-port=0
# -target-port=37017
# -target-port=47017
# - >
# go test -count=1 {{.RACEFLAG}} -run=TestEnvData -v
# -tags=ferretdb_testenvdata .
# -compat-port=0
# -records-dir=../records
# -target-port=37018
# -target-port=47018

integration-gen:
cmds:
Expand Down
21 changes: 8 additions & 13 deletions integration/embedded_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ package integration

import (
"context"
"crypto/tls"
"path/filepath"
"testing"

"github.com/stretchr/testify/assert"
Expand All @@ -36,17 +36,13 @@ func TestEmbedded(t *testing.T) {

t.Parallel()

serverTLSFiles := setup.GetTLSFilesPaths(t, setup.ServerSide)

for name, tc := range map[string]struct {
config *ferretdb.Config
tlsConfig *tls.Config
embedErr error
config *ferretdb.Config
}{
"TCP": {
config: &ferretdb.Config{
Listener: ferretdb.ListenerConfig{
TCP: "127.0.0.1:65432",
TCP: "127.0.0.1:37027",
},
Handler: "pg",
PostgreSQLURL: testutil.PostgreSQLURL(t, nil),
Expand All @@ -55,15 +51,14 @@ func TestEmbedded(t *testing.T) {
"TLS": {
config: &ferretdb.Config{
Listener: ferretdb.ListenerConfig{
TLS: "127.0.0.1:65433",
TLSCertFile: serverTLSFiles.Cert,
TLSKeyFile: serverTLSFiles.Key,
TLSCAFile: serverTLSFiles.CA,
TLS: "127.0.0.1:37028",
TLSCertFile: filepath.Join(setup.CertsRoot, "server-cert.pem"),
TLSKeyFile: filepath.Join(setup.CertsRoot, "server-key.pem"),
TLSCAFile: filepath.Join(setup.CertsRoot, "rootCA-cert.pem"),
},
Handler: "pg",
PostgreSQLURL: testutil.PostgreSQLURL(t, nil),
},
tlsConfig: setup.GetClientTLSConfig(t),
},
} {
name, tc := name, tc
Expand All @@ -85,7 +80,7 @@ func TestEmbedded(t *testing.T) {
close(done)
}()

client, err := mongo.Connect(ctx, options.Client().ApplyURI(f.MongoDBURI()).SetTLSConfig(tc.tlsConfig))
client, err := mongo.Connect(ctx, options.Client().ApplyURI(f.MongoDBURI()))
require.NoError(t, err)

filter := bson.D{{
Expand Down
Loading

0 comments on commit 102b27d

Please sign in to comment.