❌ Gradle check result for b5fddb1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

❌ Gradle check result for 24a40ad: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

❌ Gradle check result for 3bcd5bb: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

❌ Gradle check result for b2a1078: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

❌ Gradle check result for ee85066: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

❌ Gradle check result for eda1c04: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

❌ Gradle check result for 5038cf3: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

❌ Gradle check result for 1e5d354: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

❌ Gradle check result for 00d4eb1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

❌ Gradle check result for 6151622: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

❌ Gradle check result for b8d88d2: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

❌ Gradle check result for b404cee: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

❌ Gradle check result for 41e70eb: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

❌ Gradle check result for e985043: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

❌ Gradle check result for bbabecd: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

❌ Gradle check result for 853b206: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

❌ Gradle check result for da16c66: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

❌ Gradle check result for ecd998e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

✅ Gradle check result for 6fc1453: SUCCESS

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-16534-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 7a0e8fbed7116173e11f52236c347affd4e22a16
# Push it to GitHub
git push --set-upstream origin backport/backport-16534-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-16534-to-2.x.

@finnegancarroll Can you create a manual backport for this?

@finnegancarroll how aux transport publish address is known to other nodes of the cluster to create a client?

@finnegancarroll how aux transport publish address is known to other nodes of the cluster to create a client?

Currently there is no endpoint for discovering published ports of an aux transport as we expect aux transports to implement this endpoint themselves if applicable to their specific client/server transport. The gRPC plugin will have an equivalent to _cat/nodes when service definitions are added to the plugin.

@finnegancarroll @reta I was wondering if we should maintain the mapping of aux transport type and port used by it in DiscoverNode?

@finnegancarroll @reta I was wondering if we should maintain the mapping of aux transport type and port used by it in DiscoverNode?

@rishabhmaurya are we replacing the transport, like default TCP/Netty4 -> TCP/Apache Arrow Flight? If yes, than the transport port would be part of the DiscoverNode. Otherwise we probably should not maintain any mappings in DiscoverNode I think (this is not essential to the cluster management).

@reta I get your point to not make DiscoveryNode and eventually cluster state bulky and it is also not relevant to cluster formation. But the downside being, the plugin dependent on aux transport, will need an additional call to get published port info when nodes are added/removed. This is also an overhead on the cluster as cluster manager node needs to get this information from each of the node so internally num of nodes times API calls to fetch this info from each node.

But the downside being, the plugin dependent on aux transport, will need an additional call to get published port info when nodes are added/removed.

@rishabhmaurya correct, this is how our sniffer for HTTP works:

• each node publishes HTTP transport info
• the clients (other plugins) may periodically pull the nodes

What I am trying to understand is:

• do we have this aux transport to be installed on every node in the cluster?
• is it supposed to be used for node-to-node as transport? or both?
• if yes, is it a replacement for transport?
• if no, is it an additional transport, not something auxiliary but necessary for cluster to function properly?

Could you please help me here by clarifying these questions? That would help to find the right place for it, thank you.

thanks @reta

do we have this aux transport to be installed on every node in the cluster?

Yes, it will be installed on all non-cluster manager nodes.

is it supposed to be used for node-to-node as transport? or both?

Initially only for node-to-node transport and eventually for both.

if yes, is it a replacement for transport?

No. It probably will always be auxiliary in nature.

if no, is it an additional transport, not something auxiliary but necessary for cluster to function properly?

The cluster formation and functionality will not be affected with this transport. It will be a replacement for some of the search APIs to begin with.

Thank you @rishabhmaurya , I honestly believe that we could start with sniffer like approach (from your explanation, it does not seem to be critical to immediately reflect topology changes with nodes joining / leaving the cluster and should not happen often). We could reconsider this later on or combine some operations with cluster discovery (when nodes leaves the cluster, just remove the aux transport entry for this node as well).

sounds reasonable thanks @reta