Use `slog` in envtool #4480

chilagrow · 2024-07-22T02:47:46Z

Description

Closes #4013.

Readiness checklist

I added/updated unit tests (and they pass).
I added/updated integration/compatibility tests (and they pass).
I added/updated comments and checked rendering.
I made spot refactorings.
I updated user documentation.
I ran task all, and it passed.
I ensured that PR title is good enough for the changelog.
(for maintainers only) I set Reviewers (@FerretDB/core), Milestone (Next), Labels, Project and project's Sprint fields.
I marked all done items in this checklist.

codecov · 2024-07-22T02:54:30Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.22%. Comparing base (f859bb1) to head (bae5a45).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4480      +/-   ##
==========================================
- Coverage   73.79%   71.22%   -2.58%     
==========================================
  Files         330      329       -1     
  Lines       22908    22904       -4     
==========================================
- Hits        16906    16313     -593     
- Misses       4765     5361     +596     
+ Partials     1237     1230       -7

see 32 files with indirect coverage changes

Flag	Coverage Δ
filter-true	`64.05% <ø> (-2.91%)`	⬇️
hana-1	`?`
integration	`64.05% <ø> (-2.91%)`	⬇️
mongodb-1	`5.42% <ø> (+<0.01%)`	⬆️
postgresql-1	`42.73% <ø> (+0.01%)`	⬆️
postgresql-2	`?`
postgresql-3	`42.47% <ø> (+<0.01%)`	⬆️
postgresql-4	`44.10% <ø> (+<0.01%)`	⬆️
postgresql-5	`45.44% <ø> (-0.03%)`	⬇️
sqlite-1	`41.64% <ø> (-0.01%)`	⬇️
sqlite-2	`?`
sqlite-3	`41.42% <ø> (-0.02%)`	⬇️
sqlite-4	`42.98% <ø> (+0.04%)`	⬆️
sqlite-5	`44.45% <ø> (+<0.01%)`	⬆️
unit	`33.56% <ø> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

chilagrow · 2024-07-22T03:37:11Z

Here's how integration test output looks like.

$ task test-integration-postgresql TEST_RUN=TestAuth                                                                                                [53/9635]
task: [gen-version] go generate -x ./build/version                                                                                                                                                          
go run ./generate.go                                                                                                                                                                                        
package.txt: build-host                                                                                                                                                                                     
commit.txt: bb04f67fdfa910798504f734151898f607eac7eb                                                                                                                                                        
branch.txt: slog-envtool                                                                                                                                                                                    version.txt: v1.21.0-104-gbb04f67fd                                                                                                                                                                         
task: [gen-version] go build -v -o bin/ -race=true -tags=ferretdb_debug,ferretdb_hana ./cmd/envtool/
github.com/FerretDB/FerretDB/build/version
github.com/FerretDB/FerretDB/internal/util/state
github.com/FerretDB/FerretDB/cmd/envtool                                                                                                                                                                    task: [test-integration-postgresql] ../bin/envtool tests run --shard-index=0 --shard-total=0 --run='TestAuth' --skip='' -- -count=1 -timeout=35m -race=true -tags=ferretdb_debug,ferretdb_hana -shuffle=on -
coverpkg=../... -coverprofile=integration-postgresql.txt ./... -target-backend=ferretdb-postgresql -target-tls -postgresql-url='postgres://username@127.0.0.1:5432/ferretdb?search_path=' -compat-url='mongo
db://username:password@127.0.0.1:47018/?tls=true&tlsCertificateKeyFile=client.pem&tlsCaFile=rootCA-cert.pem&replicaSet=rs0' -disable-pushdown=false
                                                                                                                                                                                                            
INFO    Running go test -list=TestAuth ./...                                                                                                                                                                
INFO    Starting OTLP tracer...                                                                                                                                                                             
INFO    Running go test -json -run=^(TestAuthentication|TestAuthenticationAuthSource|TestAuthenticationDifferentDatabase|TestAuthenticationOnAuthenticatedConnection|TestAuthenticationPLAIN)$ -count=1 -tim
eout=35m -race=true -tags=ferretdb_debug,ferretdb_hana -shuffle=on -coverpkg=../... -coverprofile=integration-postgresql.txt ./... -target-backend=ferretdb-postgresql -target-tls -postgresql-url=postgres:
//username@127.0.0.1:5432/ferretdb?search_path= -compat-url=mongodb://username:password@127.0.0.1:47018/?tls=true&tlsCertificateKeyFile=client.pem&tlsCaFile=rootCA-cert.pem&replicaSet=rs0 -disable-pushdow
n=false                                                                                                                                                                                                     
INFO            github.com/FerretDB/FerretDB/integration/shareddata             coverage: 0.0% of statements                              
INFO            github.com/FerretDB/FerretDB/integration/setup          coverage: 0.0% of statements
INFO    PASS github.com/FerretDB/FerretDB/integration/shareddata
INFO    PASS github.com/FerretDB/FerretDB/integration/setup                                                                                                                                                 INFO    2024-07-22T15:02:35.544+0900    INFO    observability/observability.go:89       Starting OTLP tracer... {"name":"otel"}                                                    
INFO    2024-07-22T15:02:35.544+0900    INFO    debug   debug/debug.go:221      Starting debug server on http://127.0.0.1:35083...
INFO    2024-07-22T15:02:35.544+0900    INFO    setup/startup.go:135    Target system: ferretdb-postgresql (built-in).
INFO    2024-07-22T15:02:35.544+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:35083/debug/graphs - Visualize metrics
INFO    2024-07-22T15:02:35.544+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:35083/debug/livez - Liveness probe
INFO    2024-07-22T15:02:35.544+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:35083/debug/metrics - Metrics in Prometheus format
INFO    2024-07-22T15:02:35.544+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:35083/debug/pprof - Runtime profiling data for pprof
INFO    2024-07-22T15:02:35.544+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:35083/debug/readyz - Readiness probe
INFO    2024-07-22T15:02:35.544+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:35083/debug/vars - Expvar package metrics
INFO    2024-07-22T15:02:35.545+0900    INFO    setup/startup.go:153    Compat system: MongoDB (mongodb://username:password@127.0.0.1:47018/?replicaSet=rs0&tls=true&tlsCaFile=%2Fhome%2Fchi%2Fgo%2Fsrc%2Fgi
thub.com%2Fchilagrow%2FFerretDB%2Fbuild%2Fcerts%2FrootCA-cert.pem&tlsCertificateKeyFile=%2Fhome%2Fchi%2Fgo%2Fsrc%2Fgithub.com%2Fchilagrow%2FFerretDB%2Fbuild%2Fcerts%2Fclient.pem).
INFO    -test.shuffle 1721628155545847668                                                             
INFO    2024-07-22T15:02:35.545+0900    INFO    observability/observability.go:89       Starting OTLP tracer... {"name":"otel"}
INFO    2024-07-22T15:02:35.546+0900    INFO    setup/startup.go:135    Target system: ferretdb-postgresql (built-in).
INFO    2024-07-22T15:02:35.546+0900    INFO    debug   debug/debug.go:221      Starting debug server on http://127.0.0.1:44597...
INFO    2024-07-22T15:02:35.546+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:44597/debug/graphs - Visualize metrics 
INFO    2024-07-22T15:02:35.546+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:44597/debug/livez - Liveness probe
INFO    2024-07-22T15:02:35.546+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:44597/debug/metrics - Metrics in Prometheus format
INFO    testing: warning: no tests to run                                                             
INFO    2024-07-22T15:02:35.546+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:44597/debug/pprof - Runtime profiling data for pprof
INFO    PASS                                                                                          
INFO    2024-07-22T15:02:35.546+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:44597/debug/readyz - Readiness probe
INFO    2024-07-22T15:02:35.546+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:44597/debug/vars - Expvar package metrics
INFO    2024-07-22T15:02:35.546+0900    INFO    observability/observability.go:89       Starting OTLP tracer... {"name":"otel"}
INFO    2024-07-22T15:02:35.547+0900    INFO    debug   debug/debug.go:221      Starting debug server on http://127.0.0.1:38589...
INFO    2024-07-22T15:02:35.546+0900    INFO    setup/startup.go:135    Target system: ferretdb-postgresql (built-in).         
INFO    2024-07-22T15:02:35.547+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:38589/debug/graphs - Visualize metrics
INFO    2024-07-22T15:02:35.547+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:38589/debug/livez - Liveness probe
INFO    2024-07-22T15:02:35.547+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:38589/debug/metrics - Metrics in Prometheus format
INFO    2024-07-22T15:02:35.547+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:38589/debug/pprof - Runtime profiling data for pprof
INFO    2024-07-22T15:02:35.547+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:38589/debug/readyz - Readiness probe
INFO    2024-07-22T15:02:35.547+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:38589/debug/vars - Expvar package metrics
INFO    2024-07-22T15:02:35.547+0900    INFO    setup/startup.go:153    Compat system: MongoDB (mongodb://username:password@127.0.0.1:47018/?replicaSet=rs0&tls=true&tlsCaFile=%2Fhome%2Fchi%2Fgo%2Fsrc%2Fgi
thub.com%2Fchilagrow%2FFerretDB%2Fbuild%2Fcerts%2FrootCA-cert.pem&tlsCertificateKeyFile=%2Fhome%2Fchi%2Fgo%2Fsrc%2Fgithub.com%2Fchilagrow%2FFerretDB%2Fbuild%2Fcerts%2Fclient.pem).
INFO    -test.shuffle 1721628155547784403
INFO    testing: warning: no tests to run
INFO    PASS
INFO    2024-07-22T15:02:35.549+0900    INFO    setup/startup.go:153    Compat system: MongoDB (mongodb://username:password@127.0.0.1:47018/?replicaSet=rs0&tls=true&tlsCaFile=%2Fhome%2Fchi%2Fgo%2Fsrc%2Fgi
thub.com%2Fchilagrow%2FFerretDB%2Fbuild%2Fcerts%2FrootCA-cert.pem&tlsCertificateKeyFile=%2Fhome%2Fchi%2Fgo%2Fsrc%2Fgithub.com%2Fchilagrow%2FFerretDB%2Fbuild%2Fcerts%2Fclient.pem).
INFO    -test.shuffle 1721628155549588490
INFO    2024-07-22T15:02:35.550+0900    INFO    observability/observability.go:89       Starting OTLP tracer... {"name":"otel"}
INFO    2024-07-22T15:02:35.550+0900    INFO    setup/startup.go:135    Target system: ferretdb-postgresql (built-in).
INFO    2024-07-22T15:02:35.550+0900    INFO    debug   debug/debug.go:221      Starting debug server on http://127.0.0.1:36895...
INFO    2024-07-22T15:02:35.550+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:36895/debug/graphs - Visualize metrics
INFO    2024-07-22T15:02:35.550+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:36895/debug/livez - Liveness probe
INFO    2024-07-22T15:02:35.551+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:36895/debug/metrics - Metrics in Prometheus format
INFO    2024-07-22T15:02:35.551+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:36895/debug/pprof - Runtime profiling data for pprof
INFO    2024-07-22T15:02:35.551+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:36895/debug/readyz - Readiness probe
INFO    2024-07-22T15:02:35.551+0900    INFO    debug   debug/debug.go:227      http://127.0.0.1:36895/debug/vars - Expvar package metrics
INFO    testing: warning: no tests to run
INFO    PASS
INFO    2024-07-22T15:02:35.553+0900    INFO    setup/startup.go:153    Compat system: MongoDB (mongodb://username:password@127.0.0.1:47018/?replicaSet=rs0&tls=true&tlsCaFile=%2Fhome%2Fchi%2Fgo%2Fsrc%2Fgi
thub.com%2Fchilagrow%2FFerretDB%2Fbuild%2Fcerts%2FrootCA-cert.pem&tlsCertificateKeyFile=%2Fhome%2Fchi%2Fgo%2Fsrc%2Fgithub.com%2Fchilagrow%2FFerretDB%2Fbuild%2Fcerts%2Fclient.pem).
INFO    -test.shuffle 1721628155553185089
INFO    coverage: 2.0% of statements in ../...
INFO    coverage: 2.0% of statements in ../...
INFO    coverage: 2.0% of statements in ../...
INFO    2024-07-22T15:02:35.635+0900    INFO    debug   debug/debug.go:250      Debug server stopped.
INFO    2024-07-22T15:02:35.635+0900    INFO    observability/observability.go:113      OTLP tracer stopped     {"name":"otel"}
INFO    2024-07-22T15:02:35.642+0900    INFO    debug   debug/debug.go:250      Debug server stopped.
INFO    2024-07-22T15:02:35.642+0900    INFO    observability/observability.go:113      OTLP tracer stopped     {"name":"otel"}
INFO    2024-07-22T15:02:35.645+0900    INFO    debug   debug/debug.go:250      Debug server stopped.
INFO    2024-07-22T15:02:35.646+0900    INFO    observability/observability.go:113      OTLP tracer stopped     {"name":"otel"}
INFO    PASS TestAuthenticationPLAIN (0.42s/0.42s/0.33s) 1/5
INFO    PASS TestAuthenticationOnAuthenticatedConnection (0.78s/0.78s/0.78s) 2/5
INFO    PASS TestAuthenticationAuthSource (0.85s/0.85s/0.43s) 3/5
INFO    PASS TestAuthenticationDifferentDatabase (0.92s/0.92s/0.73s) 4/5
INFO    ok      github.com/FerretDB/FerretDB/integration/cursors        1.120s  coverage: 2.0% of statements in ../... [no tests to run]
INFO    PASS github.com/FerretDB/FerretDB/integration/cursors
INFO    ok      github.com/FerretDB/FerretDB/integration        1.130s  coverage: 2.0% of statements in ../... [no tests to run]
INFO    PASS github.com/FerretDB/FerretDB/integration
INFO    ok      github.com/FerretDB/FerretDB/integration/oplog  1.131s  coverage: 2.0% of statements in ../... [no tests to run]
INFO    PASS github.com/FerretDB/FerretDB/integration/oplog
INFO    PASS TestAuthentication (3.98s/3.98s/3.47s) 5/5
INFO    PASS
INFO    coverage: 24.8% of statements in ../...
INFO    2024-07-22T15:02:39.616+0900    INFO    debug   debug/debug.go:250      Debug server stopped.
INFO    2024-07-22T15:02:39.617+0900    INFO    observability/observability.go:113      OTLP tracer stopped     {"name":"otel"}
INFO    ok      github.com/FerretDB/FerretDB/integration/users  5.110s  coverage: 24.8% of statements in ../...
INFO    PASS github.com/FerretDB/FerretDB/integration/users
INFO    OTLP tracer stopped

chilagrow · 2024-07-22T06:04:09Z

How envtool setup logs

$ task env-setup                                                                                                                                             
task: [gen-version] go generate -x ./build/version                                                                                                                                                          
go run ./generate.go                                                                                                                                                                                        
package.txt: build-host                                                                                                                                                                                     
branch.txt: slog-envtool                                                                                                                                                                                    
commit.txt: 0acd4ec75736d311a46ad3b72d68be6efc826ebf                                                                                                                                                        
version.txt: v1.21.0-103-g0acd4ec75                                                                                                                                                                         
task: [gen-version] go build -v -o bin/ -race=true -tags=ferretdb_debug,ferretdb_hana ./cmd/envtool/                                                                                                        
task: [env-setup] bin/envtool setup                                                                                                                                                                         
INFO    Waiting for 127.0.0.1:5432 to be up     {"name":"postgres"}                                                                                                                                         
INFO    Pool: connection succeed        {"username":"username"}                                                                                                                                             
INFO    Waiting for 127.0.0.1:5433 to be up     {"name":"postgres_secured"}                                                                                                                                 
INFO    Pool: connection succeed        {"username":"username"}                                                                                                                                             
INFO    Waiting for 127.0.0.1:3306 to be up     {"name":"mysql"}                                                                                                                                            
INFO    Pool: connection succeed        {"username":"root"}                                                                                                                                                 
INFO    Waiting for 127.0.0.1:47017 to be up    {"name":"mongodb"}

mergify · 2024-07-22T07:09:01Z

@chilagrow this pull request has merge conflicts.

rumyantseva

Looks good, just one question (I probably missed some context).

Taskfile.yml

mergify · 2024-07-23T07:22:53Z

@chilagrow this pull request has merge conflicts.

AlekSi

There are merge conflicts

rumyantseva

chilagrow added 3 commits July 19, 2024 18:10

use slog in envtool

6500b3e

Merge branch 'main' into slog-envtool

02f40fe

use slog in envtool

cd772fe

chilagrow self-assigned this Jul 22, 2024

chilagrow added the code/chore Code maintenance improvements label Jul 22, 2024

chilagrow added this to the Next milestone Jul 22, 2024

lint

fb187d6

shard is zero in integration test

0acd4ec

chilagrow marked this pull request as ready for review July 22, 2024 03:37

chilagrow requested review from a team and AlekSi as code owners July 22, 2024 03:37

chilagrow requested review from rumyantseva, a team and noisersup July 22, 2024 03:37

chilagrow enabled auto-merge (squash) July 22, 2024 03:37

chilagrow marked this pull request as draft July 22, 2024 05:24

auto-merge was automatically disabled July 22, 2024 05:24
Pull request was converted to draft

fix

bb04f67

chilagrow marked this pull request as ready for review July 22, 2024 06:04

chilagrow enabled auto-merge (squash) July 22, 2024 06:04

mergify bot added the conflict PRs that have merge conflicts label Jul 22, 2024

chilagrow added 2 commits July 22, 2024 16:58

use instantiated slog

3fc224e

merge main

63a444a

mergify bot removed the conflict PRs that have merge conflicts label Jul 22, 2024

rumyantseva previously approved these changes Jul 22, 2024

View reviewed changes

Taskfile.yml Show resolved Hide resolved

noisersup previously approved these changes Jul 22, 2024

View reviewed changes

mergify bot added the conflict PRs that have merge conflicts label Jul 23, 2024

AlekSi reviewed Jul 23, 2024

View reviewed changes

merge main

84a4fc9

chilagrow dismissed stale reviews from noisersup and rumyantseva via 84a4fc9 July 24, 2024 01:16

mergify bot removed the conflict PRs that have merge conflicts label Jul 24, 2024

chilagrow requested review from noisersup, rumyantseva and AlekSi July 24, 2024 01:33

AlekSi requested review from a team July 24, 2024 06:50

Merge branch 'main' into slog-envtool

7eb87f9

AlekSi approved these changes Jul 24, 2024

View reviewed changes

rumyantseva approved these changes Jul 24, 2024

View reviewed changes

Merge branch 'main' into slog-envtool

bae5a45

chilagrow merged commit 858324e into FerretDB:main Jul 24, 2024
30 of 31 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use `slog` in envtool #4480

Use `slog` in envtool #4480

chilagrow commented Jul 22, 2024 •

edited

Loading

codecov bot commented Jul 22, 2024 •

edited

Loading

chilagrow commented Jul 22, 2024 •

edited

Loading

chilagrow commented Jul 22, 2024

mergify bot commented Jul 22, 2024

rumyantseva left a comment

mergify bot commented Jul 23, 2024

AlekSi left a comment

rumyantseva left a comment

Use slog in envtool #4480

Use slog in envtool #4480

Conversation

chilagrow commented Jul 22, 2024 • edited Loading

Description

Readiness checklist

codecov bot commented Jul 22, 2024 • edited Loading

Codecov Report

chilagrow commented Jul 22, 2024 • edited Loading

chilagrow commented Jul 22, 2024

mergify bot commented Jul 22, 2024

rumyantseva left a comment

Choose a reason for hiding this comment

mergify bot commented Jul 23, 2024

AlekSi left a comment

Choose a reason for hiding this comment

rumyantseva left a comment

Choose a reason for hiding this comment

Use `slog` in envtool #4480

Use `slog` in envtool #4480

chilagrow commented Jul 22, 2024 •

edited

Loading

codecov bot commented Jul 22, 2024 •

edited

Loading

chilagrow commented Jul 22, 2024 •

edited

Loading