Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

integrate pkg/reexec #179

Merged
merged 47 commits into from
Dec 18, 2024
Merged

integrate pkg/reexec #179

merged 47 commits into from
Dec 18, 2024

Conversation

thaJeztah
Copy link
Member

integrate pkg/reexec

This integrates the reexec package from moby at commit;
cc1d50a63daa725dfb8d43bba7c259107fbb9f74 (1).

Migration was done using the following steps:

# install filter-repo (https://github.com/newren/git-filter-repo/blob/main/INSTALL.md)
brew install git-filter-repo

# create a temporary clone of docker
cd ~/Projects
git clone https://github.com/moby/moby.git moby_temp
cd moby_temp

# commit taken from
git rev-parse --verify HEAD
cc1d50a63daa725dfb8d43bba7c259107fbb9f74

git filter-repo --analyze

# remove all code, except for 'pkg/reexec', and rename to /reexec
git filter-repo \
  --path 'pkg/reexec' \
  --path-rename pkg/reexec:reexec

# go to the target github.com/moby/sys repository
cd ~/go/src/github.com/moby/sys

# create a branch to work with
git checkout -b integrate_reexec

# add the temporary repository as an upstream and make sure it's up-to-date
git remote add moby_temp ~/Projects/moby_temp
git fetch moby_temp

# merge the upstream code
git merge --allow-unrelated-histories --signoff -S moby_temp/master

unclejack and others added 30 commits October 30, 2014 14:48
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)

Conflicts:
	integration/runtime_test.go
		fixed imports
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)

Conflicts:
	builder/internals.go
	daemon/graphdriver/aufs/aufs.go
	daemon/volumes.go
		fixed conflicts in imports
Docker-DCO-1.1-Signed-off-by: Jessica Frazelle <jess@docker.com> (github: jfrazelle)
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
After the new libcontainer API, the reexec.Self() output of the daemon
binary is used as the libcontainer factory InitPath.  If it is relative,
it can't be found at container start time.  This patch solves the
problem by making sure that we return a rooted/absolute path if a
relative path is used.

Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
Signed-off-by: John Howard <jhoward@microsoft.com>
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
This keeps reexec working properly even if the on-disk binary was replaced.

Signed-off-by: Tibor Vass <tibor@docker.com>
Signed-off-by: Alexey Guskov <lexag@mail.ru>
Signed-off-by: Justas Brazauskas <brazauskasjustas@gmail.com>
Signed-off-by: allencloud <allen.sun@daocloud.io>
Signed-off-by: Amit Krishnan <krish.amit@gmail.com>
Signed-off-by: allencloud <allen.sun@daocloud.io>
Signed-off-by: yupeng <yu.peng36@zte.com.cn>
Signed-off-by: Naveed Jamil <naveed.jamil@tenpearls.com>
Changes most references of syscall to golang.org/x/sys/
Ones aren't changes include, Errno, Signal and SysProcAttr
as they haven't been implemented in /x/sys/.

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>

[s390x] switch utsname from unsigned to signed

per golang/sys@33267e0
char in s390x in the /x/sys/unix package is now signed, so
change the buildtags

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
[project] Switch most syscalls to golang.org/x/sys
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Remove solaris build tag and `contrib/mkimage/solaris`
Files that are suffixed with `_linux.go` or `_windows.go` are
already only built on Linux / Windows, so these build-tags
were redundant.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Remove dead code and redundant build tags
Signed-off-by: Daniel Nephin <dnephin@docker.com>
gty-migrate-from-testify --ignore-build-tags

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
thaJeztah and others added 15 commits August 24, 2021 23:33
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
On Linux, when (os/exec.Cmd).SysProcAttr.Pdeathsig is set, the signal
will be sent to the process when the OS thread on which cmd.Start() was
executed dies. The runtime terminates an OS thread when a goroutine
exits after being wired to the thread with runtime.LockOSThread(). If
other goroutines are allowed to be scheduled onto a thread which called
cmd.Start(), an unrelated goroutine could cause the thread to be
terminated and prematurely signal the command. See
golang/go#27505 for more information.

Prevent started subprocesses with Pdeathsig from getting signaled
prematurely by wiring the starting goroutine to the OS thread until the
subprocess has exited. No other goroutines can be scheduled onto a
locked thread so it will remain alive until unlocked or the daemon
process exits.

Signed-off-by: Cory Snider <csnider@mirantis.com>
Removed pre-go1.17 build-tags with go fix;

    go mod init
    go fix -mod=readonly ./...
    rm go.mod

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit e8e1ff4 changed most uses of
the syscall package to switch utsname from unsigned to signed (see
e8e1ff4). Those don't seem to be
impacting the code used here, so either stdlib or golang.org/x/sys/unix
should work for this case.

I chose stdlib's syscall package for this case, in case we'd decide to
move this package to a separate module (and want to limit its dependencies).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This combines the implementations of the Self function, to allow having
a single GoDoc to document the behavior. The naiveSelf function is kept,
because it's used in unit-tests.

There is a minor change in behavior, as this patch removes the stub for
unsupported platforms (non-linux, windows, freebsd or darwin), which will
now use `os.Args[0]`. The stub was added in 69ab52c
to fix compilation of https://github.com/ethereum/go-ethereum on OpenBSD,
which had docker/docker as dependency. It looks like that repository no
longer has this dependency, and as this was only to make the code
compilable, is unlikely to be a problem.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The Windows, Darwin, and FreeBSD implementations were identical, other
than their GoDoc to be different. Unify them so that we don't have to
maintain separate GoDoc for each.

It's worth noting that FreeBSD also supports Pdeathsig, so could be
using the same implementation as Linux. However, we don't test/maintain
the FreeBSD implementation, and it would require updating to GoDoc to
be more specific about the use of `/proc/self/exe`, so keeping the
status quo for now.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This package is used in BuildKit, and could be a potential candidate
for moving to a separate module. While it's not too problematic to have
this dependency, the tests only used basic assertions from gotest.tools,
which could be easily re-implemented without the dependency.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Touch-up some GoDoc in the package, and remove "import" comments.

This package is used in BuildKit, and could be a potential candidate
for moving to a separate module. The "import" comments are ignored when
used in go module mode so have little benefit. Let's remove them.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Also use a slightly different name, because "reexec" is used so
widely as term in this package, making it somewhat confusing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The reexec package originally was platform-agnostic, but gained some
Linux-specific handling in 3223844.

When Windows support was implemented in Docker, the pkg/reexec package
was adjusted accordingly in 1d39072,
which now made the package with with either Linux or Windows, with various
other platforms (freebsd, solaris, darwin) being added back in separate
changes.

Based on the history above, this package should be platform-agnostic, except
for Linux-specific changes introduced in 3223844
and 33dc5c6.

This patch:

- removes the stub-implementation to make it functional on other platforms.
- renames the files for consistency

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Move the exported `Command` to a platform-agnostic file, and un-export
the platform-specific implementations. This allows us to maintain the
GoDoc in a single place, describing platform-specific differences where
needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This integrates the reexec package from moby at commit;
cc1d50a63daa725dfb8d43bba7c259107fbb9f74 ([1]).

Migration was done using the following steps:

    # install filter-repo (https://github.com/newren/git-filter-repo/blob/main/INSTALL.md)
    brew install git-filter-repo

    # create a temporary clone of docker
    cd ~/Projects
    git clone https://github.com/moby/moby.git moby_temp
    cd moby_temp

    # commit taken from
    git rev-parse --verify HEAD
    cc1d50a63daa725dfb8d43bba7c259107fbb9f74

    git filter-repo --analyze

    # remove all code, except for 'pkg/reexec', and rename to /reexec
    git filter-repo \
      --path 'pkg/reexec' \
      --path-rename pkg/reexec:reexec

    # go to the target github.com/moby/sys repository
    cd ~/go/src/github.com/moby/sys

    # create a branch to work with
    git checkout -b integrate_reexec

    # add the temporary repository as an upstream and make sure it's up-to-date
    git remote add moby_temp ~/Projects/moby_temp
    git fetch moby_temp

    # merge the upstream code
    git merge --allow-unrelated-histories --signoff -S moby_temp/master

[1]: moby/moby@cc1d50a

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah
Copy link
Member Author

As expected, some old commits missing a DCO sign-off; I marked them as "pass"

Commit sha: 77f98aa, Author: Amit Krishnan, Committer: Amit Krishnan; Expected "Amit Krishnan amit.krishnan@oracle.com", but got "Amit Krishnan krish.amit@gmail.com".
Commit sha: 55f55ca, Author: Phil Estes, Committer: Phil Estes; The sign-off is missing.
Commit sha: 9a9b50f, Author: unclejack, Committer: unclejack; The sign-off is missing.
Commit sha: 27509ff, Author: unclejack, Committer: unclejack; The sign-off is missing.
Commit sha: c4bd5c1, Author: unclejack, Committer: unclejack; The sign-off is missing.
Commit sha: 3223844, Author: unclejack, Committer: unclejack; The sign-off is missing.

@thaJeztah thaJeztah mentioned this pull request Dec 17, 2024
74 tasks
@thaJeztah thaJeztah self-assigned this Dec 17, 2024
@thaJeztah thaJeztah marked this pull request as ready for review December 17, 2024 18:08
@thaJeztah
Copy link
Member Author

@@ -0,0 +1,3 @@
module github.com/moby/sys/reexec

go 1.18
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably will work with any version of Go, but go1.18 is the oldest version we test against in this repository

@crazy-max
Copy link
Member

Not directly related but would be good to have some coverage on modules in this repo. We had this on moby repo for this package: https://app.codecov.io/gh/moby/moby/tree/master/pkg%2Freexec

@thaJeztah
Copy link
Member Author

Not directly related but would be good to have some coverage on modules in this repo. We had this on moby repo for this package: https://app.codecov.io/gh/moby/moby/tree/master/pkg%2Freexec

Yeah, we should probably add CodeCov action to this repository. Haven't looked yet what it would require (and if it plays well with multi-module repositories).

@thaJeztah
Copy link
Member Author

Thanks for reviewing! I'll bring this one in, and open a draft PR in moby, buildkit to replace it. I won't tag/release yet; will do that later (perhaps before that we can address the CodeCov as well)

@thaJeztah thaJeztah merged commit 8698290 into moby:main Dec 18, 2024
20 checks passed
@thaJeztah thaJeztah deleted the integrate_reexec branch December 18, 2024 11:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.