Skip to content

Commit

Permalink
implement module compatibility check
Browse files Browse the repository at this point in the history 
This package imports all "importable" packages, i.e., packages that:

- are not applications ("main")
- are not internal
- and that have non-test go-files

We do this to verify that our code can be consumed as a dependency
in "module mode". When using a dependency that does not have a go.mod
(i.e.; is not a "module"), go implicitly generates a go.mod. Lacking
information from the dependency itself, it assumes "go1.16" language
(see [DefaultGoModVersion]). Starting with Go1.21, go downgrades the
language version used for such dependencies, which means that any
language feature used that is not supported by go1.16 results in a
compile error;

    # github.com/docker/cli/cli/context/store
    /go/pkg/mod/github.com/docker/cli@v25.0.0-beta.2+incompatible/cli/context/store/storeconfig.go:6:24: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
    /go/pkg/mod/github.com/docker/cli@v25.0.0-beta.2+incompatible/cli/context/store/store.go:74:12: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)

These errors do NOT occur when using GOPATH mode, nor do they occur
when using "pseudo module mode" (the "-mod=mod -modfile=vendor.mod"
approach used in this repository).

As a workaround for this situation, we must include "//go:build" comments
in any file that uses newer go-language features (such as the "any" type
or the "min()", "max()" builtins).

From the go toolchain docs (https://go.dev/doc/toolchain):

> The go line for each module sets the language version the compiler enforces
> when compiling packages in that module. The language version can be changed
> on a per-file basis by using a build constraint.
>
> For example, a module containing code that uses the Go 1.21 language version
> should have a go.mod file with a go line such as go 1.21 or go 1.21.3.
> If a specific source file should be compiled only when using a newer Go
> toolchain, adding //go:build go1.22 to that source file both ensures that
> only Go 1.22 and newer toolchains will compile the file and also changes
> the language version in that file to Go 1.22.

This file is a generated module that imports all packages provided in
the repository, which replicates an external consumer using our code
as a dependency in go-module mode, and verifies all files in those
packages have the correct "//go:build <go language version>" set.

To test this package:

    make -C ./internal/gocompat/
    GO111MODULE=off go generate .
    go mod tidy
    go test -v
    # github.com/docker/docker/libnetwork/options
    ../../libnetwork/options/options.go:45:25: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
    # github.com/docker/docker/libnetwork/internal/setmatrix
    ../../libnetwork/internal/setmatrix/setmatrix.go:13:16: type parameter requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/internal/setmatrix/setmatrix.go:13:18: predeclared comparable requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/internal/setmatrix/setmatrix.go:14:20: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/internal/setmatrix/setmatrix.go:20:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/internal/setmatrix/setmatrix.go:31:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/internal/setmatrix/setmatrix.go:43:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/internal/setmatrix/setmatrix.go:59:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/internal/setmatrix/setmatrix.go:80:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/internal/setmatrix/setmatrix.go:93:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/internal/setmatrix/setmatrix.go:104:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/internal/setmatrix/setmatrix.go:104:10: too many errors
    # github.com/docker/docker/libnetwork/config
    ../../libnetwork/config/config.go:35:47: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/config/config.go:47:41: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/config/config.go:63:55: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../libnetwork/config/config.go:95:63: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
    # github.com/docker/docker/testutil
    ../../testutil/helpers.go:80:9: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
    # github.com/docker/docker/builder/builder-next/adapters/containerimage
    ../../builder/builder-next/adapters/containerimage/pull.go:72:4: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    ../../builder/builder-next/adapters/containerimage/pull.go:200:19: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
    FAIL	gocompat [build failed]
    make: *** [Makefile:5: verify] Error 1

[DefaultGoModVersion]: https://github.com/golang/go/blob/58c28ba286dd0e98fe4cca80f5d64bbcb824a685/src/cmd/go/internal/gover/version.go#L15-L24
[2]: https://go.dev/doc/toolchain

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
  • Loading branch information
@thaJeztah
thaJeztah committed Dec 15, 2023
1 parent bd70d66 commit bb0b6ab
Show file tree
Hide file tree
Showing 9 changed files with 3,301 additions and 0 deletions.
4 changes: 4 additions & 0 deletions internal/gocompat/.gitignore
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
go.mod
go.sum
main.go
main_test.go
13 changes: 13 additions & 0 deletions internal/gocompat/Makefile
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
.PHONY: verify
verify: generate
GO111MODULE=on go test -v

.PHONY: generate
generate: clean
GO111MODULE=off go generate .
GO111MODULE=on go mod tidy
GO111MODULE=on go test -v

.PHONY: clean
clean:
@rm -f go.mod go.sum main.go main_test.go
6 changes: 6 additions & 0 deletions internal/gocompat/generate.go
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
package main

//go:generate go run modulegenerator.go

// make sure the modfile package is vendored.
import _ "golang.org/x/mod/modfile"
187 changes: 187 additions & 0 deletions internal/gocompat/modulegenerator.go
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,187 @@
//go:build ignore

package main

import (
"bytes"
"fmt"
"log"
"os"
"os/exec"
"strings"
"text/template"

"golang.org/x/mod/modfile"
)

func main() {
if err := generateApp(); err != nil {
log.Fatal(err)
}
if err := generateModule(); err != nil {
log.Fatal(err)
}
}

func generateApp() error {
cmd := exec.Command("go", "list", "-find", "-f", `{{- if ne .Name "main"}}{{if .GoFiles}}{{.ImportPath}}{{end}}{{end -}}`, "../../...")
out, err := cmd.CombinedOutput()
if err != nil {
return err
}

var pkgs []string
for _, p := range strings.Split(string(out), "\n") {
if strings.TrimSpace(p) == "" || strings.Contains(p, "/internal") {
continue
}
pkgs = append(pkgs, p)
}
tmpl, err := template.New("main").Parse(appTemplate)
if err != nil {
return err
}

var buf bytes.Buffer
err = tmpl.Execute(&buf, appContext{Generator: cmd.String(), Packages: pkgs})
if err != nil {
return err
}

return os.WriteFile("main_test.go", buf.Bytes(), 0o644)
}

func generateModule() error {
content, err := os.ReadFile("../../go.mod")
if err != nil {
if !os.IsNotExist(err) {
return err
}
content = []byte("module github.com/docker/docker\n")
if err := os.WriteFile("../../go.mod", content, 0o644); err != nil {
return err
}
}
mod, err := modfile.Parse("../../go.mod", content, nil)
if err != nil {
return err
}
if mod.Go != nil && mod.Go.Version != "" {
return fmt.Errorf("main go.mod must not contain a go version")
}
content, err = os.ReadFile("../../vendor.mod")
if err != nil {
return err
}
mod, err = modfile.Parse("../../vendor.mod", content, nil)
if err != nil {
return err
}
if err := mod.AddModuleStmt("gocompat"); err != nil {
return err
}
if err := mod.AddReplace("github.com/docker/docker", "", "../../", ""); err != nil {
return err
}
if err := mod.AddGoStmt("1.21"); err != nil {
return err
}
out, err := mod.Format()
if err != nil {
return err
}
tmpl, err := template.New("mod").Parse(modTemplate)
if err != nil {
return err
}

gen, _ := os.Executable()

var buf bytes.Buffer
err = tmpl.Execute(&buf, appContext{Generator: gen, Dependencies: string(out)})
if err != nil {
return err
}

return os.WriteFile("go.mod", buf.Bytes(), 0o644)
}

type appContext struct {
Generator string
Packages []string
Dependencies string
}

const appTemplate = `// Code generated by "{{ .Generator }}". DO NOT EDIT.
package main_test
import (
"testing"
// Import all importable packages, i.e., packages that:
//
// - are not applications ("main")
// - are not internal
// - and that have non-test go-files
{{- range .Packages }}
_ "{{ . }}"
{{- end}}
)
// This file import all "importable" packages, i.e., packages that:
//
// - are not applications ("main")
// - are not internal
// - and that have non-test go-files
//
// We do this to verify that our code can be consumed as a dependency
// in "module mode". When using a dependency that does not have a go.mod
// (i.e.; is not a "module"), go implicitly generates a go.mod. Lacking
// information from the dependency itself, it assumes "go1.16" language
// (see [DefaultGoModVersion]). Starting with Go1.21, go downgrades the
// language version used for such dependencies, which means that any
// language feature used that is not supported by go1.16 results in a
// compile error;
//
// # github.com/docker/cli/cli/context/store
// /go/pkg/mod/github.com/docker/cli@v25.0.0-beta.2+incompatible/cli/context/store/storeconfig.go:6:24: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
// /go/pkg/mod/github.com/docker/cli@v25.0.0-beta.2+incompatible/cli/context/store/store.go:74:12: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
//
// These errors do NOT occur when using GOPATH mode, nor do they occur
// when using "pseudo module mode" (the "-mod=mod -modfile=vendor.mod"
// approach used in this repository).
//
// As a workaround for this situation, we must include "//go:build" comments
// in any file that uses newer go-language features (such as the "any" type
// or the "min()", "max()" builtins).
//
// From the go toolchain docs (https://go.dev/doc/toolchain):
//
// > The go line for each module sets the language version the compiler enforces
// > when compiling packages in that module. The language version can be changed
// > on a per-file basis by using a build constraint.
// >
// > For example, a module containing code that uses the Go 1.21 language version
// > should have a go.mod file with a go line such as go 1.21 or go 1.21.3.
// > If a specific source file should be compiled only when using a newer Go
// > toolchain, adding //go:build go1.22 to that source file both ensures that
// > only Go 1.22 and newer toolchains will compile the file and also changes
// > the language version in that file to Go 1.22.
//
// This file is a generated module that imports all packages provided in
// the repository, which replicates an external consumer using our code
// as a dependency in go-module mode, and verifies all files in those
// packages have the correct "//go:build <go language version>" set.
//
// [DefaultGoModVersion]: https://github.com/golang/go/blob/58c28ba286dd0e98fe4cca80f5d64bbcb824a685/src/cmd/go/internal/gover/version.go#L15-L24
// [2]: https://go.dev/doc/toolchain
func TestModuleCompatibllity(t *testing.T) {
t.Log("all packages have the correct go version specified through //go:build")
}
`

const modTemplate = `// Code generated by "{{ .Generator }}". DO NOT EDIT.
{{.Dependencies}}
`
184 changes: 184 additions & 0 deletions vendor/golang.org/x/mod/modfile/print.go
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit bb0b6ab

Please sign in to comment.