Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI: switch to unprivileged Kaniko to build pipeline images #11292

Merged
merged 1 commit into from
Jun 11, 2024
Merged

CI: switch to unprivileged Kaniko to build pipeline images #11292

merged 1 commit into from
Jun 11, 2024

Conversation

ant31
Copy link
Contributor

@ant31 ant31 commented Jun 11, 2024

What type of PR is this?

/kind cleanup

What this PR does / why we need it:
The CI is using privileged containers, and one of the jobs requiring it is the pipeline-image-build.
This PR replaces the container build tool with Kaniko.

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

NONE

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jun 11, 2024
@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 11, 2024
@MrFreezeex
Copy link
Member

The CI is using privileged containers, and one of the jobs requiring it is the pipeline-image-build. This PR replaces the container build tool with Kaniko.

Hi! Afaik if the container running the current CI is privileged it doesn't need to be, buildkit is running in rootless mode so 🤷‍♂️

@ant31
Copy link
Contributor Author

ant31 commented Jun 11, 2024

/ok-to-test

@k8s-ci-robot k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Jun 11, 2024
@ant31
Copy link
Contributor Author

ant31 commented Jun 11, 2024 •
edited
Loading

@MrFreezeex when I tried to desactivate it, the job failed.
I'll try to find the job logs

@ant31
Copy link
Contributor Author

ant31 commented Jun 11, 2024 •
edited
Loading

the failed job: https://gitlab.com/kargo-ci/kubernetes-sigs-kubespray/-/jobs/7046293555

$ buildctl-daemonless.sh build \ # collapsed multi-line command
could not connect to unix:///run/user/1000/buildkit/buildkitd.sock after 10 trials
========== log ==========
[rootlesskit:child ] error: failed to share mount point: /: permission denied
[rootlesskit:parent] error: child exited: exit status 1
sh: can't kill pid 38: No such process

it worked after enabling privileged again 🤷‍♂️

MrFreezeex
MrFreezeex approved these changes Jun 11, 2024
View reviewed changes
Copy link
Member

@MrFreezeex MrFreezeex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the failed job: https://gitlab.com/kargo-ci/kubernetes-sigs-kubespray/-/jobs/7046293555

$ buildctl-daemonless.sh build \ # collapsed multi-line command
could not connect to unix:///run/user/1000/buildkit/buildkitd.sock after 10 trials
========== log ==========
[rootlesskit:child ] error: failed to share mount point: /: permission denied
[rootlesskit:parent] error: child exited: exit status 1
sh: can't kill pid 38: No such process

it worked after enabling privileged again 🤷‍♂️

Indeed, found this: moby/buildkit#2441 (comment) but not sure we can easily switch. So kaniko might be a good fit indeed 👍. It seems to support the mount type cache as well so looks good, thanks! 👍

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ant31, MrFreezeex

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ant31
Copy link
Contributor Author

ant31 commented Jun 11, 2024

Thanks for searching the issue.
Yes the kaniko works similarly with cache too.

missing the /lgtm 🙇‍♂️

@MrFreezeex
Copy link
Member

MrFreezeex commented Jun 11, 2024 •
edited
Loading

Thanks for searching the issue. Yes the kaniko works similarly with cache too.

missing the /lgtm 🙇‍♂️

AFAIK usually we wait for two reviewers even if submitted by an approver but this should be fine let's go with it.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 11, 2024
@k8s-ci-robot k8s-ci-robot merged commit 3e72be2 into kubernetes-sigs:master Jun 11, 2024
79 checks passed
Rickkwa pushed a commit to Rickkwa/kubespray that referenced this pull request Jun 26, 2024
@ant31 @Rickkwa
CI: switch to unprivileged Kaniko to build pipeline images (kubernete…
f77aa4c 
…s-sigs#11292)
@tico88612 tico88612 mentioned this pull request Jul 7, 2024
Merged
@tico88612
Copy link
Member

/cherrypick release-2.25

@k8s-infra-cherrypick-robot k8s-infra-cherrypick-robot mentioned this pull request Jul 12, 2024
Merged
@k8s-infra-cherrypick-robot
Copy link

@tico88612: new pull request created: #11375

In response to this:

/cherrypick release-2.25

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

davidumea pushed a commit to elastisys/kubespray that referenced this pull request Oct 25, 2024
@ant31 @davidumea
CI: switch to unprivileged Kaniko to build pipeline images (kubernete…
d50c6ad 
…s-sigs#11292)
kpoxo6op pushed a commit to kpoxo6op/kubespray that referenced this pull request Dec 27, 2024
@ant31 @kpoxo6op
CI: switch to unprivileged Kaniko to build pipeline images (kubernete…
bc30982 
…s-sigs#11292)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@MrFreezeex MrFreezeex MrFreezeex approved these changes

@yankay yankay Awaiting requested review from yankay

Assignees

@MrFreezeex MrFreezeex

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@ant31 @MrFreezeex @k8s-ci-robot @tico88612 @k8s-infra-cherrypick-robot