Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HOSTEDCP-1716: When running the HO locally it should no required a running pod #4268

Conversation

marcbrown2024
Copy link
Contributor

@marcbrown2024 marcbrown2024 commented Jun 24, 2024

What this PR does / why we need it:
This PR covers the steps to run the HyperShift Operator locally without needing access to a full production environment.

Which issue(s) this PR fixes (optional, use fixes #<issue_number>(, fixes #<issue_number>, ...) format, where issue_number might be a GitHub issue, or a Jira story:
Fixes #HOSTEDCP-1716

Checklist

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jun 24, 2024

@marcbrown2024: This pull request references HOSTEDCP-1716 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.17.0" version, but no target version was set.

In response to this:

What this PR does / why we need it:
This PR covers the steps to run the HyperShift Operator locally without needing access to a full production environment.

Which issue(s) this PR fixes (optional, use fixes #<issue_number>(, fixes #<issue_number>, ...) format, where issue_number might be a GitHub issue, or a Jira story:
Fixes #HOSTEDCP-1716

Checklist

Subject and description added to both, commit and PR.
Relevant issues have been referenced.
This change includes docs.
This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jun 24, 2024
@openshift-ci openshift-ci bot requested review from csrwng and hasueki June 24, 2024 21:26
@openshift-ci openshift-ci bot added area/documentation Indicates the PR includes changes for documentation needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. and removed do-not-merge/needs-area labels Jun 24, 2024
Copy link
Contributor

openshift-ci bot commented Jun 24, 2024

Hi @marcbrown2024. Thanks for your PR.

I'm waiting for a openshift member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jun 24, 2024

@marcbrown2024: This pull request references HOSTEDCP-1716 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.17.0" version, but no target version was set.

In response to this:

What this PR does / why we need it:
This PR covers the steps to run the HyperShift Operator locally without needing access to a full production environment.

Which issue(s) this PR fixes (optional, use fixes #<issue_number>(, fixes #<issue_number>, ...) format, where issue_number might be a GitHub issue, or a Jira story:
Fixes #HOSTEDCP-1716

Checklist

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jun 24, 2024

@marcbrown2024: This pull request references HOSTEDCP-1716 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.17.0" version, but no target version was set.

In response to this:

What this PR does / why we need it:
This PR covers the steps to run the HyperShift Operator locally without needing access to a full production environment.

Which issue(s) this PR fixes (optional, use fixes #<issue_number>(, fixes #<issue_number>, ...) format, where issue_number might be a GitHub issue, or a Jira story:
Fixes #HOSTEDCP-1716

Checklist

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@csrwng
Copy link
Contributor

csrwng commented Jun 24, 2024

/ok-to-test

@openshift-ci openshift-ci bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jun 24, 2024
@stevekuznetsov
Copy link
Contributor

@marcbrown2024 thanks for the contributions! I would suggest a slightly different track:

  • running a binary locally gives some useful input to the validity of the operator, but does not give signal on the associated requirements for the deployment
  • helpful guides in Markdown go stale faster than we can keep up with them

In test/integration an automated process already exists for spinning up a kind cluster on your machine and using your local content to run the HyperShift components in the management plane. I'd love to collaborate with you to make sure that the approach for the integration tests works for your use-cases. I think such an approach will prove to be more holistic in what it helps us verify locally and more durable to boot.

@csrwng
Copy link
Contributor

csrwng commented Jun 25, 2024

@stevekuznetsov I think we need both, since they're for different scenarios. Running the hypershift operator locally while pointing to a fully deployed management cluster (on AWS or Azure) allows you to test/debug things that you would only see in an environment like that (say nodepool controller AWS specifics, or even the scheduler/scaler for ROSA). The integration environment lets you quickly validate other things, and is definitely useful, so we should also document it.

@stevekuznetsov
Copy link
Contributor

@csrwng the integration scripts have no requirements on what they are pointed at - why not use them to hot-replace the hypershift-operator binary running in the cluster?

@stevekuznetsov
Copy link
Contributor

I guess I'd just say - let's put some value on executable, re-usable functionality here. I think everyone doing development on HyperShift has their own assemblage of shell scripts, hack docs, whatever. It's a lot. Find some semi-reusable thing and eat the cost of it not being perfectly optimal for your specific/special use-case, and reap the benefits to collaboration and onboarding of having a common platform.

@marcbrown2024 marcbrown2024 force-pushed the hypershift-operator-locally branch from ec521be to 8d588a8 Compare June 28, 2024 16:47
Copy link

netlify bot commented Jun 28, 2024

Deploy Preview for hypershift-docs ready!

Name Link
🔨 Latest commit cc1aa3c
🔍 Latest deploy log https://app.netlify.com/sites/hypershift-docs/deploys/66880f2c1216f700084ed288
😎 Deploy Preview https://deploy-preview-4268--hypershift-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@openshift-ci openshift-ci bot added the area/ci-tooling Indicates the PR includes changes for CI or tooling label Jun 28, 2024
@marcbrown2024 marcbrown2024 force-pushed the hypershift-operator-locally branch from 8d588a8 to d461491 Compare June 28, 2024 21:08
@jparrill
Copy link
Contributor

jparrill commented Jul 1, 2024

/approve

Copy link
Contributor

openshift-ci bot commented Jul 1, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jparrill, marcbrown2024

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 1, 2024
BUCKET_NAME="${HYPERSHIFT_BUCKET_NAME:-hypershift-ci-oidc}"
AWSCREDS="${HYPERSHIFT_AWS_CREDS:-$HOME/.aws/credentials}"

# Install hypershift
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to setup external dns https://hypershift-docs.netlify.app/how-to/aws/external-dns/
hypershift install --external-dns-provider=aws --external-dns-credentials=route53-aws-creds --external-dns-domain-filter=service-provider-domain.com ...?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that would definitely be useful for AWS, but not for AKS. Should we have different scripts for the different environments where we may want to install? (we could start by renaming this one with -aws) and install external-dns, wdyt?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One entrypoint that everyone uses, which knows how to validate the expected input based on the target you're deploying to? :D

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fwiw we do use external DNS atm in aks as well. But yes let's have a way to install the opinions per platform.

@marcbrown2024 marcbrown2024 force-pushed the hypershift-operator-locally branch from d461491 to 4c9ee48 Compare July 2, 2024 20:31
Makefile Outdated
@@ -286,3 +290,11 @@ ci-install-hypershift-private:
regenerate-pki:
REGENERATE_PKI=1 $(GO) test ./control-plane-pki-operator/...
REGENERATE_PKI=1 $(GO) test ./test/e2e/... -run TestRegeneratePKI

.PHONY: hypershift-install-aws
hypershift-install-aws:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hypershift-install-aws-dev?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you saying I should rename it to hypershift-install-aws-dev?

4. Install HyperShift in development mode which causes the operator deployment to be deployment scaled to zero so that it doesn't conflict with your local operator process (see [Prerequisites](../getting-started.md#prerequisites)):

```shell linenums="1"
HYPERSHIFT_REGION=us-east-2 HYPERSHIFT_BUCKET_NAME=my-bucket make hypershift-install-aws
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do I need to export the variables and then pass them again?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also can we mention you can set them but there's defaults and make whatever should just work without exporting or setting anything

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are exporting the vars because in the script the default is us-east-1 and bucket is hypershift-ci-oidc. So you'll have to export the correct var if default is not what is needed

should no required a running pod

This PR covers the steps to run the HyperShift Operator locally without
needing access to a full production environment.

Fixes #HOSTEDCP-1716
@marcbrown2024 marcbrown2024 force-pushed the hypershift-operator-locally branch from 4c9ee48 to cc1aa3c Compare July 5, 2024 15:20
@marcbrown2024
Copy link
Contributor Author

/retest

--oidc-storage-provider-s3-credentials "${AWSCREDS}" \
--oidc-storage-provider-s3-region "${REGION}" \
--oidc-storage-provider-s3-bucket-name "${BUCKET_NAME}" \
--control-plane-operator-image=quay.io/hypershift/hypershift:latest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably worth making --control-plane-operator-image a variable

@enxebre
Copy link
Member

enxebre commented Jul 9, 2024

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jul 9, 2024
Copy link
Contributor

openshift-ci bot commented Jul 9, 2024

@marcbrown2024: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot openshift-merge-bot bot merged commit eccb1cb into openshift:main Jul 9, 2024
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/ci-tooling Indicates the PR includes changes for CI or tooling area/documentation Indicates the PR includes changes for documentation jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants