Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KEP-4427 : AllowRelaxedDNSSearchValidation #127167

Merged

Conversation

adrianmoisey
Copy link
Member

What type of PR is this?

/kind feature

What this PR does / why we need it:

Relates to https://kep.k8s.io/4427
Adds relaxed DNS search string validation behind an alpha feature flag.

Which issue(s) this PR fixes:

Fixes # N/A (I think)

Special notes for your reviewer:

  1. This is my first real code change to k/k, so please remember that in your review.
  2. I'm not very happy with the changes in staging/src/k8s.io/apimachinery/pkg/util/validation/validation.go, any suggestions will be welcome.
  3. The KEP called for e2e tests, but after writing the integration tests, I felt like the coverage was enough. I'm happy to add e2e tests if anyone thinks that they are needed.

Does this PR introduce a user-facing change?

Allow for Pod search domains to be a single dot "." or contain an underscore "_"

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Sep 5, 2024
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Sep 5, 2024
@k8s-ci-robot
Copy link
Contributor

Hi @adrianmoisey. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot requested review from deads2k and dims September 5, 2024 18:55
@k8s-ci-robot k8s-ci-robot added area/test kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/testing Categorizes an issue or PR as relevant to SIG Testing. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Sep 5, 2024
@k8s-triage-robot
Copy link

This PR may require API review.

If so, when the changes are ready, complete the pre-review checklist and request an API review.

Status of requested reviews is tracked in the API Review project.

@thockin thockin added the sig/network Categorizes an issue or PR as relevant to SIG Network. label Sep 5, 2024
@aojea
Copy link
Member

aojea commented Sep 5, 2024

/ok-to-test
@adrianmoisey you should request Kubernetes membership if you are interested https://github.com/kubernetes/community/blob/master/community-membership.md, I think you already meet the requirements.

I'm happy to sponsor you if you want, so you just need to get a sponsor that is not from Google

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 5, 2024
@aojea
Copy link
Member

aojea commented Sep 5, 2024

@adrianmoisey can you add an e2e test , you can duplicate one of those

ginkgo.It("should work with the pod containing more than 6 DNS search paths and longer than 256 search list characters", func(ctx context.Context) {
// All the names we need to be able to resolve.
namesToResolve := []string{
"kubernetes.default",
"kubernetes.default.svc",
}
hostFQDN := fmt.Sprintf("%s.%s.%s.svc.%s", dnsTestPodHostName, dnsTestServiceName, f.Namespace.Name, framework.TestContext.ClusterDNSDomain)
hostEntries := []string{hostFQDN, dnsTestPodHostName}
// TODO: Validate both IPv4 and IPv6 families for dual-stack
wheezyProbeCmd, wheezyFileNames := createProbeCommand(namesToResolve, hostEntries, "", "wheezy", f.Namespace.Name, framework.TestContext.ClusterDNSDomain, framework.TestContext.ClusterIsIPv6())
jessieProbeCmd, jessieFileNames := createProbeCommand(namesToResolve, hostEntries, "", "jessie", f.Namespace.Name, framework.TestContext.ClusterDNSDomain, framework.TestContext.ClusterIsIPv6())
ginkgo.By("Running these commands on wheezy: " + wheezyProbeCmd + "\n")
ginkgo.By("Running these commands on jessie: " + jessieProbeCmd + "\n")
ginkgo.By("Creating a pod with expanded DNS configuration to probe DNS")
testNdotsValue := "5"
testSearchPaths := []string{
fmt.Sprintf("%038d.k8s.io", 1),
fmt.Sprintf("%038d.k8s.io", 2),
fmt.Sprintf("%038d.k8s.io", 3),
fmt.Sprintf("%038d.k8s.io", 4),
fmt.Sprintf("%038d.k8s.io", 5),
fmt.Sprintf("%038d.k8s.io", 6), // 260 characters
}
pod := createDNSPod(f.Namespace.Name, wheezyProbeCmd, jessieProbeCmd, dnsTestPodHostName, dnsTestServiceName)
pod.Spec.DNSPolicy = v1.DNSClusterFirst
pod.Spec.DNSConfig = &v1.PodDNSConfig{
Searches: testSearchPaths,
Options: []v1.PodDNSConfigOption{
{
Name: "ndots",
Value: &testNdotsValue,
},
},
}
validateDNSResults(ctx, f, pod, append(wheezyFileNames, jessieFileNames...))
})
})
, we need to be sure that despite the API enables it , we don't hit issues with the container runtimes or the OS libraries (muslc, libc, per example)

Copy link
Member

@thockin thockin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM! Small nits.

@@ -386,11 +386,13 @@ func GetValidationOptionsFromPodSpecAndMeta(podSpec, oldPodSpec *api.PodSpec, po
AllowNamespacedSysctlsForHostNetAndHostIPC: false,
AllowNonLocalProjectedTokenPath: false,
AllowImageVolumeSource: utilfeature.DefaultFeatureGate.Enabled(features.ImageVolume),
AllowRelaxedDNSSearchValidation: utilfeature.DefaultFeatureGate.Enabled(features.RelaxedDNSSearchValidation),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kind of a nit, but why set this if you just override it below?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoops, good catch, thanks.
Fixed in 1290574

dnsConfig *core.PodDNSConfig
}{
{
name: "beings with underscore, contains underscore, featuregate enabled",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/beings/begins ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, thanks!
Fixed in 1b7aa45

dnsConfig: &core.PodDNSConfig{Searches: []string{"."}},
},
{
name: "beings with underscore, contains underscore, featuregate disabled",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

beings

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 1b7aa45

@thockin thockin self-assigned this Sep 6, 2024
@thockin thockin added the tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges. label Sep 6, 2024
@thockin
Copy link
Member

thockin commented Sep 6, 2024

Thanks!

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 6, 2024
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: ac624577e3d42e5a51c86056989e07597928961d

@adrianmoisey adrianmoisey force-pushed the KEP-4427-relaxed-dns-search branch 2 times, most recently from 7236d2e to e31cc2e Compare September 8, 2024 14:50
@adrianmoisey
Copy link
Member Author

/test pull-kubernetes-node-e2e-containerd

@aojea
Copy link
Member

aojea commented Sep 8, 2024

@adrianmoisey we have to modify the nftables job to not pick your change, lgtm kubernetes/test-infra#33440 and once it merges retest and report back

@adrianmoisey
Copy link
Member Author

/test pull-kubernetes-e2e-kind-nftables

Comment on lines +286 to +289
// Owner: sig-network
// Marks tests of KEP-4427 that require the `RelaxedDNSSearchValidation` feature gate
RelaxedDNSSearchValidation = framework.WithFeature(framework.ValidFeatures.Add("RelaxedDNSSearchValidation"))

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was my mistake, we don't need this here, Feature means you need something special on the cluster to use this fcuntionality, per example, IPv6DualStack needs to configure the cluster in a specific way, or Loadbalancer means you need to have a cluster that implements load balancer

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clarified in another PR by Patrick Ohly, we need this meanwhile we don't fully migrate to labels

@@ -597,7 +598,38 @@ var _ = common.SIGDescribe("DNS", func() {
}
validateDNSResults(ctx, f, pod, append(wheezyFileNames, jessieFileNames...))
})
})

var _ = common.SIGDescribe("DNS", feature.RelaxedDNSSearchValidation, func() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
var _ = common.SIGDescribe("DNS", feature.RelaxedDNSSearchValidation, func() {
var _ = common.SIGDescribe("DNS", framework.WithFeatureGate(features.RelaxedDNSSearchValidation), func() {

@aojea aojea mentioned this pull request Sep 9, 2024
@aojea
Copy link
Member

aojea commented Sep 9, 2024

@adrianmoisey the e2e error will be fixed by #126977

@@ -597,7 +599,38 @@ var _ = common.SIGDescribe("DNS", func() {
}
validateDNSResults(ctx, f, pod, append(wheezyFileNames, jessieFileNames...))
})
})

var _ = common.SIGDescribe(feature.RelaxedDNSSearchValidation, framework.WithFeatureGate(features.RelaxedDNSSearchValidation), func() {
Copy link
Member

@aojea aojea Sep 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can try to add a TODO and try to label directly

Suggested change
var _ = common.SIGDescribe(feature.RelaxedDNSSearchValidation, framework.WithFeatureGate(features.RelaxedDNSSearchValidation), func() {
// TODO replace WithLabel by framework.WithFeatureGate(features.RelaxedDNSSearchValidation) once https://github.com/kubernetes/kubernetes/pull/126977 is solved
var _ = common.SIGDescribe(feature.RelaxedDNSSearchValidation, framework.WithLabel("Feature:Alpha"), func() {

@adrianmoisey
Copy link
Member Author

/retest

@aojea
Copy link
Member

aojea commented Sep 10, 2024

@adrianmoisey squash for final review, that job failing is not required and besides that, it is independent of this PR content

@adrianmoisey adrianmoisey force-pushed the KEP-4427-relaxed-dns-search branch from edd5168 to 014f07c Compare September 10, 2024 15:07
@@ -24193,6 +24193,64 @@ func TestValidateSleepAction(t *testing.T) {
}
}

// TODO: merge these test to TestValidatePodSpec after AllowRelaxedDNSSearchValidation feature graduates to Beta
func TestValidatePodDNSConfigWithRelaxedSearchDomain(t *testing.T) {
Copy link
Member

@aojea aojea Sep 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thockin @adrianmoisey I'm worried about this combinations of underscore and dots

can we extend the coverage here?

diff --git a/pkg/apis/core/validation/validation_test.go b/pkg/apis/core/validation/validation_test.go
index d3ed0288685..d1fdf4e2eba 100644
--- a/pkg/apis/core/validation/validation_test.go
+++ b/pkg/apis/core/validation/validation_test.go
@@ -24219,6 +24219,42 @@ func TestValidatePodDNSConfigWithRelaxedSearchDomain(t *testing.T) {
                        featureEnabled: true,
                        dnsConfig:      &core.PodDNSConfig{Searches: []string{"."}},
                },
+               {
+                       name:           "two dots, featuregate enabled",
+                       expectError:    true,
+                       featureEnabled: true,
+                       dnsConfig:      &core.PodDNSConfig{Searches: []string{".."}},
+               },
+               {
+                       name:           "underscore and dot, featuregate enabled",
+                       expectError:    true,
+                       featureEnabled: true,
+                       dnsConfig:      &core.PodDNSConfig{Searches: []string{"_."}},
+               },
+                               {
+                       name:           "two underscore and dot, featuregate enabled",
+                       expectError:    true,
+                       featureEnabled: true,
+                       dnsConfig:      &core.PodDNSConfig{Searches: []string{"__."}},
+               },
+                               {
+                       name:           "dot and two underscore, featuregate enabled",
+                       expectError:    true,
+                       featureEnabled: true,
+                       dnsConfig:      &core.PodDNSConfig{Searches: []string{".__"}},
+               },
+               {
+                       name:           "dot and underscore, featuregate enabled",
+                       expectError:    true,
+                       featureEnabled: true,
+                       dnsConfig:      &core.PodDNSConfig{Searches: []string{"._"}},
+               },
+               {
+                       name:           "lot of underscores, featuregate enabled",
+                       expectError:    true,
+                       featureEnabled: true,
+                       dnsConfig:      &core.PodDNSConfig{Searches: []string{"____________"}},
+               },
                {
                        name:           "begins with underscore, contains underscore, featuregate disabled",
                        expectError:    true,
@@ -24237,6 +24273,42 @@ func TestValidatePodDNSConfigWithRelaxedSearchDomain(t *testing.T) {
                        featureEnabled: false,
                        dnsConfig:      &core.PodDNSConfig{Searches: []string{"."}},
                },
+               {
+                       name:           "two dots, featuregate disabled",
+                       expectError:    true,
+                       featureEnabled: false,
+                       dnsConfig:      &core.PodDNSConfig{Searches: []string{".."}},
+               },
+               {
+                       name:           "underscore and dot, featuregate disabled",
+                       expectError:    true,
+                       featureEnabled: false,
+                       dnsConfig:      &core.PodDNSConfig{Searches: []string{"_."}},
+               },
+               {
+                       name:           "dot and underscore, featuregate disabled",
+                       expectError:    true,
+                       featureEnabled: false,
+                       dnsConfig:      &core.PodDNSConfig{Searches: []string{"._"}},
+               },
+               {
+                       name:           "lot of underscores, featuregate disabled",
+                       expectError:    true,
+                       featureEnabled: false,
+                       dnsConfig:      &core.PodDNSConfig{Searches: []string{"____________"}},
+               },
+                                               {
+                       name:           "two underscore and dot, featuregate disabled",
+                       expectError:    true,
+                       featureEnabled: false,
+                       dnsConfig:      &core.PodDNSConfig{Searches: []string{"__."}},
+               },
+                       {
+                       name:           "dot and two underscore, featuregate dsiable",
+                       expectError:    true,
+                       featureEnabled: false,
+                       dnsConfig:      &core.PodDNSConfig{Searches: []string{".__"}},
+               },
        }
        for _, testCase := range testCases {
                t.Run(testCase.name, func(t *testing.T) {

and validate those testcases that fail are legit or no?

go test -timeout 30s -run ^TestValidatePodDNSConfigWithRelaxedSearchDomain$ k8s.io/kubernetes/pkg/apis/core/validation
--- FAIL: TestValidatePodDNSConfigWithRelaxedSearchDomain (0.00s)
    --- FAIL: TestValidatePodDNSConfigWithRelaxedSearchDomain/underscore_and_dot,_featuregate_enabled (0.00s)
        validation_test.go:24317: Unexpected success
FAIL
FAIL    k8s.io/kubernetes/pkg/apis/core/validation      0.244s
FAIL

Copy link
Member

@thockin thockin Sep 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.. should fail, I think - can't start with a dot unless the whole token is "."

_. should fail, I think - RFC952 and RFC1035 say it must start with alphabetic, RFC1123 allows that to be a letter or digit, RFC2782 allows an underscore to be prepended to the service and proto fields, which cannot be empty (RFC2052). We are choosing to allow '_' anywhere we would allow '-', and we do not allow a leading hyphen, so we must have at least one alphanumeric. Does that analysis make sense?

._ should fail for the same reason as .. -- we do not allow leading dots (BTW, your "dot and underscore" case is broken, it says "_."

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🍿 looking for the regex 😄

Copy link
Member Author

@adrianmoisey adrianmoisey Sep 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_. should fail, I think - RFC952 and RFC1035 say it must start with alphabetic, RFC1123 allows that to be a letter or digit, RFC2782 allows an underscore to be prepended to the service and proto fields, which cannot be empty (RFC2052). We are choosing to allow '_' anywhere we would allow '-', and we do not allow a leading hyphen, so we must have at least one alphanumeric. Does that analysis make sense?

Just to be clear:
_. should not be allowed
_tcp.exampe.com should be allowed

That right?

🍿 looking for the regex 😄

🥳

Copy link
Member

@thockin thockin Sep 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const dns1123LabelFmtWithUnderscore string = "[_a-z0-9]([-_a-z0-9]*[a-z0-9])?"

That looks like it will allow _ which is wrong. https://go.dev/play/p/S0rjL6SGJrO

`_?[a-z0-9]([-_a-z0-9]*[a-z0-9])?`

That seems more correct?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More test cases added in 1c2f8b8 (thanks Antonio)
Regex updated in the same commit (thanks Tim)

I'm happy to add more test cases if needed.

Also update Regex to match the case we want.

Thanks Tim and Antonio!
@k8s-ci-robot
Copy link
Contributor

k8s-ci-robot commented Sep 11, 2024

@adrianmoisey: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-kubernetes-e2e-gce-network-policies 1c2f8b8 link false /test pull-kubernetes-e2e-gce-network-policies

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@aojea
Copy link
Member

aojea commented Sep 12, 2024

/lgtm
/approve

@adrianmoisey we (most probable me) broke the pull-kubernetes-e2e-gce-network-policies with the new label, but we don't need to wait for it to merge this, do you mind sending a PR for fixing that job, most probable something related to bash expansion or quotes or likes

2024/09/11 20:19:43 main.go:326: Something went wrong: encountered 1 errors: [error during ./hack/ginkgo-e2e.sh --ginkgo.skip=GCE|Disruptive|Serial|SNAT --ginkgo.label-filter='(sig-network || Conformance || Feature: containsAny {NetworkPolicy, NetworkPolicyEndPort} ) && !Feature: containsAny {Alpha, Beta, Example, Federation, IPv6DualStack, LoadBalancer, Networking-IPv6, PerformanceDNS, SCTPConnectivity, KubeProxyDaemonSetMigration}' --report-dir=/logs/artifacts --disable-log-dump=true --cluster-ip-range=10.64.0.0/14: exit status 1]

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 12, 2024
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: 17d26d63e000c0bc3622edd55fd0e45ac506ad14

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: adrianmoisey, aojea, thockin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 8e3adc4 into kubernetes:master Sep 12, 2024
19 of 20 checks passed
@k8s-ci-robot k8s-ci-robot added this to the v1.32 milestone Sep 12, 2024
@adrianmoisey adrianmoisey deleted the KEP-4427-relaxed-dns-search branch September 12, 2024 14:20
aroradaman pushed a commit to aroradaman/kubernetes that referenced this pull request Oct 8, 2024
* KEP-4427 : AllowRelaxedDNSSearchValidation

* Add e2e test with feature gate to test KEP-4427 RelaxedDNSSearchValidation

* Add more validatePodDNSConfig test cases

Also update Regex to match the case we want.

Thanks Tim and Antonio!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/test cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/network Categorizes an issue or PR as relevant to SIG Network. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges.
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

5 participants