/help

@Jeffwan:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

/assign

As we discussed today on the AutoML and Training meeting, we want to use Python scripts instead of deprecated ksonnet to build Argo Workflows for the AWS CI infra.
This is the current tf-operator Argo workflow: https://github.com/kubeflow/tf-operator/blob/master/test/workflows/components/workflows.libsonnet#L98-L327. For all operators they will be similar.

To transfer this workflow to Python we should do the similar steps as for the Kubeflow Notebook Controller:

• Add python_paths to prow config. I think for us it will be kubeflow/tf-operator/test/e2e or kubeflow/tf-operator/test/workflows. We can think about the folder name to store python scripts.

• Define py_func which creates Argo Workflow.

• Create function which returns Argo Workflow with all required steps.

I think we can simplify workflow utils for the Training Operator to leave only necessary functions.

Does it sound good @Jeffwan @thunderboltsid @johnugeorge ?

cc @kimwnasptd for reference.

I created PR in Katib: kubeflow/katib#1674 to switch from ksonnet to python.
@thunderboltsid @Jeffwan I think we can keep the similar process for Training Operator.

we want to use Python scripts instead of deprecated ksonnet to build Argo Workflows for the AWS CI infra.

Sounds really good

Using python, we can still leverage kubeflow/testing common code base, right?
I am thinking if we should write test codes in golang later. Bring up this topic since we will anyway rewrite the tests.

Using python, we can still leverage kubeflow/testing common code base, right?
I am thinking if we should write test codes in golang later. Bring up this topic since we will anyway rewrite the tests.

@Jeffwan Yes, we can.
Check here: https://github.com/kubeflow/katib/blob/f2af5d5ab7385eedbb903060dcf64f6b42fcdcf6/test/e2e/v1beta1/argo_workflow.py#L312.

I am using shared infra utils.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.