-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
.github: generate feature summary report from CI #36933
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ebbb71e
to
67be187
Compare
67be187
to
d04477a
Compare
/test |
170ee6d
to
a9c3db4
Compare
/test |
a9c3db4
to
65df60e
Compare
/test |
brlbil
approved these changes
Jan 13, 2025
65df60e
to
4a6de69
Compare
/test |
tklauser
approved these changes
Jan 14, 2025
Generate a summary report for all CI jobs associated with a specific commit. With this commit, cilium-cli has the capabilities of generating a markdown report which gives a summary of all features tested across all CI jobs. This summary give us the information with regarding which features, and where a feature, is being tested on a specific GitHub workflow. Also, with a GitHub workflow to generate this report, we will be able to execute this against any commit SHA from Cilium repository and put it as a status for that commit. This can be useful in case we want to generate a report for a particular PR and see if a new feature is covered by existing tests or if a new test is testing existing features. Signed-off-by: André Martins <andre@cilium.io>
4a6de69
to
d63c58c
Compare
/test |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
cilium-cli
This PR contains changes related with cilium-cli
ready-to-merge
This PR has passed all tests and received consensus from code owners to merge.
release-note/misc
This PR makes changes that have no direct user impact.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Generate a summary report for all CI jobs associated with a specific
commit.
With this commit, cilium-cli has the capabilities of generating a
markdown report which gives a summary of all features tested across all
CI jobs. This summary give us the information with regarding which
features, and where a feature, is being tested on a specific GitHub
workflow.
Also, with a GitHub workflow to generate this report, we will be able to
execute this against any commit SHA from Cilium repository and put it as
a status for that commit. This can be useful in case we want to generate
a report for a particular PR and see if a new feature is covered by
existing tests or if a new test is testing existing features
Note For Reviewers: Small Preview of the table that's going to be generated by the GH workflow:
Summary Table
Metric:
cilium_feature_adv_connect_and_lb_bgp_enabled
Click here to see
Cilium E2E Upgrade (ci-e2e-upgrade)
Metric:
cilium_feature_adv_connect_and_lb_big_tcp_enabled
address_family=ipv4-ipv6-dual-stack
Click here to see
Cilium E2E Upgrade (ci-e2e-upgrade)
Metric:
cilium_feature_adv_connect_and_lb_cilium_envoy_config_enabled
Click here to see
Cilium E2E Upgrade (ci-e2e-upgrade)
Cilium IPsec upgrade (ci-ipsec-upgrade)
Conformance Cluster Mesh (ci-clustermesh)
Conformance Gateway API (ci-gateway-api)
Conformance IPsec E2E (ci-ipsec-e2e)
Conformance Ingress (ci-ingress)
Conformance Kind Envoy Embedded
Smoke Test
Smoke Test with IPv6
Metric:
cilium_feature_adv_connect_and_lb_clustermesh_enabled
max_connected_clusters=255;mode=clustermesh-apiserver
Click here to see
Cilium Cluster Mesh upgrade (ci-clustermesh)
Conformance Cluster Mesh (ci-clustermesh)
Metric:
cilium_feature_adv_connect_and_lb_clustermesh_enabled
max_connected_clusters=255;mode=etcd
Click here to see
Cilium Cluster Mesh upgrade (ci-clustermesh)
Conformance Cluster Mesh (ci-clustermesh)
Metric:
cilium_feature_adv_connect_and_lb_clustermesh_enabled
max_connected_clusters=255;mode=kvstoremesh
Click here to see
Cilium Cluster Mesh upgrade (ci-clustermesh)
Cluster Mesh Scale Test (scale-clustermesh)
Conformance Cluster Mesh (ci-clustermesh)
Metric:
cilium_feature_adv_connect_and_lb_clustermesh_enabled
max_connected_clusters=511;mode=clustermesh-apiserver
Click here to see
Cilium Cluster Mesh upgrade (ci-clustermesh)
Conformance Cluster Mesh (ci-clustermesh)
Metric:
cilium_feature_adv_connect_and_lb_clustermesh_enabled
max_connected_clusters=511;mode=etcd
Click here to see
Conformance Cluster Mesh (ci-clustermesh)
Metric:
cilium_feature_adv_connect_and_lb_clustermesh_enabled
max_connected_clusters=511;mode=kvstoremesh
Click here to see
Cilium Cluster Mesh upgrade (ci-clustermesh)
Conformance Cluster Mesh (ci-clustermesh)
Metric:
cilium_feature_adv_connect_and_lb_egress_gateway_enabled
Click here to see
Cilium E2E Upgrade (ci-e2e-upgrade)
Cilium IPsec upgrade (ci-ipsec-upgrade)
Conformance IPsec E2E (ci-ipsec-e2e)
Metric:
cilium_feature_adv_connect_and_lb_envoy_proxy_enabled
mode=embedded
Click here to see
Conformance Kind Envoy Embedded
Conformance Runtime (ci-runtime)
Metric:
cilium_feature_adv_connect_and_lb_envoy_proxy_enabled
mode=standalone
Click here to see
Cilium Cluster Mesh upgrade (ci-clustermesh)
Cilium E2E Upgrade (ci-e2e-upgrade)
Cilium IPsec upgrade (ci-ipsec-upgrade)
CiliumEndpointSlice migration (ci-ces-migrate)
Cluster Mesh Scale Test (scale-clustermesh)
Conformance Cluster Mesh (ci-clustermesh)
Conformance Delegated IPAM (ci-delegated-ipam)
Conformance External Workloads (ci-external-workloads)
Conformance Gateway API (ci-gateway-api)
Conformance IPsec E2E (ci-ipsec-e2e)
Conformance Ingress (ci-ingress)
Conformance K8s Kind
Conformance K8s Upstream Network
Conformance Multi Pool IPAM (ci-multi-pool)
Cyclonus Network Policy Test
Hubble CLI integration tests
Smoke Test
Smoke Test with IPv6
Metric:
cilium_feature_adv_connect_and_lb_k8s_internal_traffic_policy_enabled
Click here to see
Cilium Cluster Mesh upgrade (ci-clustermesh)
Cilium E2E Upgrade (ci-e2e-upgrade)
Cilium IPsec upgrade (ci-ipsec-upgrade)
CiliumEndpointSlice migration (ci-ces-migrate)
Cluster Mesh Scale Test (scale-clustermesh)
Conformance Cluster Mesh (ci-clustermesh)
Conformance Delegated IPAM (ci-delegated-ipam)
Conformance External Workloads (ci-external-workloads)
Conformance Gateway API (ci-gateway-api)
Conformance IPsec E2E (ci-ipsec-e2e)
Conformance Ingress (ci-ingress)
Conformance K8s Kind
Conformance K8s Upstream Network
Conformance Kind Envoy Embedded
Conformance Multi Pool IPAM (ci-multi-pool)
Conformance Runtime (ci-runtime)
Cyclonus Network Policy Test
Hubble CLI integration tests
Smoke Test
Smoke Test with IPv6
Metric:
cilium_feature_adv_connect_and_lb_kube_proxy_replacement_enabled
Click here to see
Cilium Cluster Mesh upgrade (ci-clustermesh)
Cilium E2E Upgrade (ci-e2e-upgrade)
Cilium IPsec upgrade (ci-ipsec-upgrade)
Cluster Mesh Scale Test (scale-clustermesh)
Conformance Cluster Mesh (ci-clustermesh)
Conformance External Workloads (ci-external-workloads)
Conformance Gateway API (ci-gateway-api)
Conformance IPsec E2E (ci-ipsec-e2e)
Conformance Ingress (ci-ingress)
Conformance K8s Kind
Conformance K8s Upstream Network
Conformance Kind Envoy Embedded
Conformance Multi Pool IPAM (ci-multi-pool)
Smoke Test
Smoke Test with IPv6
Metric:
cilium_feature_adv_connect_and_lb_l2_lb_enabled
Click here to see
Conformance Gateway API (ci-gateway-api)
Conformance Ingress (ci-ingress)
Metric:
cilium_feature_adv_connect_and_lb_node_port_configuration
acceleration=disabled;algorithm=random;mode=dsr
Click here to see
Cilium E2E Upgrade (ci-e2e-upgrade)
Metric:
cilium_feature_adv_connect_and_lb_node_port_configuration
acceleration=disabled;algorithm=random;mode=snat
Click here to see
Cilium Cluster Mesh upgrade (ci-clustermesh)
Cilium E2E Upgrade (ci-e2e-upgrade)
Cilium IPsec upgrade (ci-ipsec-upgrade)
CiliumEndpointSlice migration (ci-ces-migrate)
Cluster Mesh Scale Test (scale-clustermesh)
Conformance Cluster Mesh (ci-clustermesh)
Conformance Delegated IPAM (ci-delegated-ipam)
Conformance External Workloads (ci-external-workloads)
Conformance Gateway API (ci-gateway-api)
Conformance IPsec E2E (ci-ipsec-e2e)
Conformance Ingress (ci-ingress)
Conformance K8s Kind
Conformance K8s Upstream Network
Conformance Kind Envoy Embedded
Conformance Multi Pool IPAM (ci-multi-pool)
Conformance Runtime (ci-runtime)
Cyclonus Network Policy Test
Hubble CLI integration tests
Smoke Test
Smoke Test with IPv6
Metric:
cilium_feature_adv_connect_and_lb_node_port_configuration
acceleration=native;algorithm=random;mode=snat
Click here to see
Conformance Ingress (ci-ingress)
Metric:
cilium_feature_adv_connect_and_lb_node_port_configuration
acceleration=testing-only;algorithm=random;mode=snat
Click here to see
Cilium E2E Upgrade (ci-e2e-upgrade)
Metric:
cilium_feature_adv_connect_and_lb_transparent_encryption
mode=ipsec;node2node_enabled=false
Click here to see
Cilium Cluster Mesh upgrade (ci-clustermesh)
Cilium IPsec upgrade (ci-ipsec-upgrade)
Conformance Cluster Mesh (ci-clustermesh)
Conformance Gateway API (ci-gateway-api)
Conformance IPsec E2E (ci-ipsec-e2e)
Metric:
cilium_feature_adv_connect_and_lb_transparent_encryption
mode=wireguard;node2node_enabled=false
Click here to see
Cilium Cluster Mesh upgrade (ci-clustermesh)
Cilium E2E Upgrade (ci-e2e-upgrade)
Conformance Cluster Mesh (ci-clustermesh)
Metric:
cilium_feature_adv_connect_and_lb_transparent_encryption
mode=wireguard;node2node_enabled=true
Click here to see
Cilium E2E Upgrade (ci-e2e-upgrade)
Metric:
cilium_feature_controlplane_cilium_endpoint_slices_enabled
Click here to see
Cilium E2E Upgrade (ci-e2e-upgrade)
CiliumEndpointSlice migration (ci-ces-migrate)
Conformance Cluster Mesh (ci-clustermesh)
Metric:
cilium_feature_controlplane_identity_allocation
mode=crd
Click here to see
Cilium Cluster Mesh upgrade (ci-clustermesh)
Cilium E2E Upgrade (ci-e2e-upgrade)
Cilium IPsec upgrade (ci-ipsec-upgrade)
CiliumEndpointSlice migration (ci-ces-migrate)
Cluster Mesh Scale Test (scale-clustermesh)
Conformance Cluster Mesh (ci-clustermesh)
Conformance Delegated IPAM (ci-delegated-ipam)
Conformance External Workloads (ci-external-workloads)
Conformance Gateway API (ci-gateway-api)
Conformance IPsec E2E (ci-ipsec-e2e)
Conformance Ingress (ci-ingress)
Conformance K8s Kind
Conformance K8s Upstream Network
Conformance Kind Envoy Embedded
Conformance Multi Pool IPAM (ci-multi-pool)
Cyclonus Network Policy Test
Hubble CLI integration tests
Smoke Test
Smoke Test with IPv6
Metric:
cilium_feature_controlplane_identity_allocation
mode=kvstore
Click here to see
Cilium Cluster Mesh upgrade (ci-clustermesh)
Cilium E2E Upgrade (ci-e2e-upgrade)
Cilium IPsec upgrade (ci-ipsec-upgrade)
Conformance Cluster Mesh (ci-clustermesh)
Conformance IPsec E2E (ci-ipsec-e2e)
Conformance Runtime (ci-runtime)
Metric:
cilium_feature_controlplane_ipam
mode=cluster-pool
Click here to see
Cilium IPsec upgrade (ci-ipsec-upgrade)
Conformance External Workloads (ci-external-workloads)
Conformance Runtime (ci-runtime)
Metric:
cilium_feature_controlplane_ipam
mode=delegated-plugin
Click here to see
Conformance Delegated IPAM (ci-delegated-ipam)
Metric:
cilium_feature_controlplane_ipam
mode=kubernetes
Click here to see
Cilium Cluster Mesh upgrade (ci-clustermesh)
Cilium E2E Upgrade (ci-e2e-upgrade)
CiliumEndpointSlice migration (ci-ces-migrate)
Cluster Mesh Scale Test (scale-clustermesh)
Conformance Cluster Mesh (ci-clustermesh)
Conformance Gateway API (ci-gateway-api)
Conformance IPsec E2E (ci-ipsec-e2e)
Conformance Ingress (ci-ingress)
Conformance K8s Kind
Conformance K8s Upstream Network
Conformance Kind Envoy Embedded
Cyclonus Network Policy Test
Hubble CLI integration tests
Smoke Test
Smoke Test with IPv6
Metric:
cilium_feature_controlplane_ipam
mode=multi-pool
Click here to see
Conformance Multi Pool IPAM (ci-multi-pool)
Metric:
cilium_feature_datapath_chaining_enabled
mode=none
Click here to see
Cilium Cluster Mesh upgrade (ci-clustermesh)
Cilium E2E Upgrade (ci-e2e-upgrade)
Cilium IPsec upgrade (ci-ipsec-upgrade)
CiliumEndpointSlice migration (ci-ces-migrate)
Cluster Mesh Scale Test (scale-clustermesh)
Conformance Cluster Mesh (ci-clustermesh)
Conformance Delegated IPAM (ci-delegated-ipam)
Conformance External Workloads (ci-external-workloads)
Conformance Gateway API (ci-gateway-api)
Conformance IPsec E2E (ci-ipsec-e2e)
Conformance Ingress (ci-ingress)
Conformance Multi Pool IPAM (ci-multi-pool)
Conformance Runtime (ci-runtime)
Cyclonus Network Policy Test
Hubble CLI integration tests
Smoke Test
Smoke Test with IPv6
Metric:
cilium_feature_datapath_chaining_enabled
mode=portmap
Click here to see
Conformance K8s Kind
Conformance K8s Upstream Network
Conformance Kind Envoy Embedded
Metric:
cilium_feature_datapath_config
mode=netkit
Click here to see
Cilium E2E Upgrade (ci-e2e-upgrade)
Metric:
cilium_feature_datapath_config
mode=netkit-l2
Click here to see
Cilium E2E Upgrade (ci-e2e-upgrade)
Metric:
cilium_feature_datapath_config
mode=veth
Click here to see
Cilium Cluster Mesh upgrade (ci-clustermesh)
Cilium E2E Upgrade (ci-e2e-upgrade)
Cilium IPsec upgrade (ci-ipsec-upgrade)
CiliumEndpointSlice migration (ci-ces-migrate)
Cluster Mesh Scale Test (scale-clustermesh)
Conformance Cluster Mesh (ci-clustermesh)
Conformance Delegated IPAM (ci-delegated-ipam)
Conformance External Workloads (ci-external-workloads)
Conformance Gateway API (ci-gateway-api)
Conformance IPsec E2E (ci-ipsec-e2e)
Conformance Ingress (ci-ingress)
Conformance K8s Kind
Conformance K8s Upstream Network
Conformance Kind Envoy Embedded
Conformance Multi Pool IPAM (ci-multi-pool)
Conformance Runtime (ci-runtime)
Cyclonus Network Policy Test
Hubble CLI integration tests
Smoke Test
Smoke Test with IPv6
Metric:
cilium_feature_datapath_internet_protocol
address_family=ipv4-ipv6-dual-stack
Click here to see
Cilium Cluster Mesh upgrade (ci-clustermesh)
Cilium E2E Upgrade (ci-e2e-upgrade)
Cilium IPsec upgrade (ci-ipsec-upgrade)
Conformance Cluster Mesh (ci-clustermesh)
Conformance Delegated IPAM (ci-delegated-ipam)
Conformance IPsec E2E (ci-ipsec-e2e)
Conformance Kind Envoy Embedded
Conformance Runtime (ci-runtime)
Metric:
cilium_feature_datapath_internet_protocol
address_family=ipv4-only
Click here to see
CiliumEndpointSlice migration (ci-ces-migrate)
Cluster Mesh Scale Test (scale-clustermesh)
Conformance Cluster Mesh (ci-clustermesh)
Conformance Delegated IPAM (ci-delegated-ipam)
Conformance External Workloads (ci-external-workloads)
Conformance Gateway API (ci-gateway-api)
Conformance Ingress (ci-ingress)
Conformance K8s Kind
Conformance K8s Upstream Network
Conformance Multi Pool IPAM (ci-multi-pool)
Cyclonus Network Policy Test
Hubble CLI integration tests
Smoke Test
Metric:
cilium_feature_datapath_internet_protocol
address_family=ipv6-only
Click here to see
Conformance Cluster Mesh (ci-clustermesh)
Smoke Test with IPv6
Metric:
cilium_feature_datapath_network
mode=direct-routing
Click here to see
Cilium E2E Upgrade (ci-e2e-upgrade)
Cilium IPsec upgrade (ci-ipsec-upgrade)
CiliumEndpointSlice migration (ci-ces-migrate)
Conformance Cluster Mesh (ci-clustermesh)
Conformance Delegated IPAM (ci-delegated-ipam)
Conformance IPsec E2E (ci-ipsec-e2e)
Conformance Multi Pool IPAM (ci-multi-pool)
Smoke Test with IPv6
Metric:
cilium_feature_datapath_network
mode=overlay-geneve
Click here to see
Cilium E2E Upgrade (ci-e2e-upgrade)
Cilium IPsec upgrade (ci-ipsec-upgrade)
Conformance Cluster Mesh (ci-clustermesh)
Conformance IPsec E2E (ci-ipsec-e2e)
Metric:
cilium_feature_datapath_network
mode=overlay-vxlan
Click here to see
Cilium Cluster Mesh upgrade (ci-clustermesh)
Cilium E2E Upgrade (ci-e2e-upgrade)
Cilium IPsec upgrade (ci-ipsec-upgrade)
Cluster Mesh Scale Test (scale-clustermesh)
Conformance Cluster Mesh (ci-clustermesh)
Conformance External Workloads (ci-external-workloads)
Conformance Gateway API (ci-gateway-api)
Conformance IPsec E2E (ci-ipsec-e2e)
Conformance Ingress (ci-ingress)
Conformance K8s Kind
Conformance K8s Upstream Network
Conformance Kind Envoy Embedded
Conformance Runtime (ci-runtime)
Cyclonus Network Policy Test
Hubble CLI integration tests
Smoke Test
Metric:
cilium_feature_network_policies_cidr_policies
mode=nodes
Click here to see
Cilium E2E Upgrade (ci-e2e-upgrade)
Metric:
cilium_feature_network_policies_cilium_clusterwide_network_policies_total
action=add
Click here to see
Cilium E2E Upgrade (ci-e2e-upgrade)
Metric:
cilium_feature_network_policies_cilium_clusterwide_network_policies_total
action=delete
Click here to see
Cilium E2E Upgrade (ci-e2e-upgrade)
Metric:
cilium_feature_network_policies_cilium_envoy_config_total
action=add
Click here to see
Cilium E2E Upgrade (ci-e2e-upgrade)
Cilium IPsec upgrade (ci-ipsec-upgrade)
Conformance Cluster Mesh (ci-clustermesh)
Conformance Gateway API (ci-gateway-api)
Conformance IPsec E2E (ci-ipsec-e2e)
Conformance Ingress (ci-ingress)