-
Notifications
You must be signed in to change notification settings - Fork 7.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
installation: rewrite the code #52541
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
howardjohn
added
the
release-notes-none
Indicates a PR that does not require release notes.
label
Aug 6, 2024
Skipping CI for Draft Pull Request. |
istio-testing
added
do-not-merge/work-in-progress
Block merging of a PR because it isn't ready yet.
size/XXL
Denotes a PR that changes 1000+ lines, ignoring generated files.
labels
Aug 6, 2024
/test all |
howardjohn
force-pushed
the
operator/2.0-attempt
branch
from
August 6, 2024 23:05
8be5fa8
to
2d828d6
Compare
/test all |
2 similar comments
/test all |
/test all |
howardjohn
force-pushed
the
operator/2.0-attempt
branch
from
August 7, 2024 17:33
945fac5
to
36cc6d0
Compare
/test all |
2 similar comments
/test all |
/test all |
howardjohn
force-pushed
the
operator/2.0-attempt
branch
from
August 7, 2024 20:24
006fe1d
to
4d333fd
Compare
/test all |
1 similar comment
/test all |
howardjohn
force-pushed
the
operator/2.0-attempt
branch
from
August 7, 2024 23:00
6628f47
to
052565f
Compare
/test all |
/retest |
howardjohn
force-pushed
the
operator/2.0-attempt
branch
from
August 8, 2024 16:26
052565f
to
e5d5aa5
Compare
/test all |
howardjohn
force-pushed
the
operator/2.0-attempt
branch
from
August 8, 2024 17:04
e5d5aa5
to
e205dc5
Compare
/test all |
howardjohn
force-pushed
the
operator/2.0-attempt
branch
from
August 15, 2024 14:59
8563ffb
to
33e0f2e
Compare
istio-testing
removed
the
needs-rebase
Indicates a PR needs to be rebased before being merged
label
Aug 15, 2024
howardjohn
added a commit
to howardjohn/istio
that referenced
this pull request
Aug 15, 2024
I have no clue how this happened. istio#52668 fixed these, istio#52541 accidentally reverted in a rebase. But the tests didn't fail... but now it randomly is again.
Merged
istio-testing
pushed a commit
that referenced
this pull request
Aug 15, 2024
luksa
pushed a commit
to luksa/istio
that referenced
this pull request
Oct 14, 2024
* upstream/master: installer: fix missing labels on uninstall (istio#52717) Update various dependencies (istio#52683) Workaround calico issue in integration test (istio#52720) Auto detect cilium incompatibility during install (istio#52624) Fix lint issues (istio#52721) Update Go dependencies to 1.31 (istio#52661) Automator: update proxy@master in istio/istio@master (istio#52712) Automator: update common-files@master in istio/istio@master (istio#52685) installation: rewrite the code (istio#52541) Automator: update ztunnel@master in istio/istio@master (istio#52711) adding ds support for autoallocated IP in getAllAddressesForProxy (istio#52680)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
release-notes-none
Indicates a PR that does not require release notes.
size/XXL
Denotes a PR that changes 1000+ lines, ignoring generated files.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR effectively rewrites the
operator/
codebase. Over the past few weeks, I have been looking into refactoring the codebase to both simplify it dramatically, given its new reduced requirements, and allow it to be structured in a way that allows us to evolve our installation strategy. Namely, an increasingly converging behavior with Helm, migration paths to helm, etc. I have spent considerable time attempting to refactor the codebase to do so, and what I have ultimately concluded is that the way the codebase is currently structured is (somewhat subtly, at first glance) extremely problematic.Aside from usual issues around tech debt and an organically evolving codebase, both of which impact the codebase heavily, there is a systemic issues in how data flows through the system. We have a variety of critical objects:
Values
, MeshConfig translated to aMeshConfig
map[string]any
structpb.Struct
Through a simple install, we transition between these states an incredible amount of times. In one case, we wanted to translate to an IstioOperatorSpec into an IstioOperator; to do so, we converted to yaml, to a map, back to yaml, then to a Go template, then unmarshalled back into an IstioOperator. While that case is clearly an easy fix, there are many other cases where the structure of the codebase requires us to do constantly move back and forth.
My original attempts at cleaning up the codebase were to simply cleanup the discrepancies with
values
andmeshConfig
not being typed. I did two different prototypes of this, one keeping IstioOperator as a proto and another one just using standard Go structs. Both of these were not able to be completed. The root of the problem is we are really inherintly dealing with unstructured data. Most of the codebase is doing things like "merge these two maps", "set values.foo.bar=1", etc -- these are extremely hard to do with structured data. We currently do this with a custom-made library,tpath
, which is incredibly complicated to handle all the edge cases of the structured data. Many times we avoid this by translating between various type formats (as discussed above), but this introduces even more complexity, as we have all these alternative paths to manipulate data.This PR instead treats a
map[string]any
as the core data structure, wrapped in avalues.Map
type. Not-coincidentally, this is also how the Helm codebase is structured (originally, I had directly used most of Helm's libraries. However, for backwards compatibility we have different--set
semantics unfortunately).Beyond that, we now have clearly devised installation phases:
manifest generate
: print and exit.install
: apply to the cluster.This structure should be more conducive to future feature work. For instance, if we want to perform a helm migration tooling, we can easily insert logic after the 'merge all inputs'.
Overall, the intent of this PR is to have no user facing behavioral changes, with two minor exceptions:
istioctl manifest generate --component
- filters to one component. Can already be done by just only enabling that component, does not exist on istioctl installistioctl manifest generate -o <dir>
- output files to a dir instead of a file. It hasn't worked since at least 1.18, so I am pretty confident there is no usageThese were excluded just to clean up the code mostly
Fixes #52469