Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(v2): iterator and sub-DAG basic support #6985

Merged
merged 9 commits into from
Dec 9, 2021

Conversation

Bobgy
Copy link
Contributor

@Bobgy Bobgy commented Nov 30, 2021

Description of your changes:
Part of #6160

This adds basic sub-DAG and iterator support.

The following advanced features will be supported separately:

  • DAG publisher (set DAG final status as succeed or failure)
  • artifact iterator
  • parameter/artifact aggregation

Checklist:

@google-cla google-cla bot added the cla: yes label Nov 30, 2021
@Bobgy Bobgy changed the title feat(v2): iterator basic support feat(v2): iterator and sub-DAG basic support Nov 30, 2021
@Bobgy
Copy link
Contributor Author

Bobgy commented Nov 30, 2021

/hold
wait for #6983 to be merged first

@Bobgy Bobgy changed the title feat(v2): iterator and sub-DAG basic support WIP - feat(v2): iterator and sub-DAG basic support Nov 30, 2021
@Bobgy
Copy link
Contributor Author

Bobgy commented Nov 30, 2021

I need to first upgrade test cluster mlmd version.

@Bobgy Bobgy force-pushed the v2-iterator branch 7 times, most recently from c0c23be to 5661677 Compare December 7, 2021 01:13
@Bobgy Bobgy changed the title WIP - feat(v2): iterator and sub-DAG basic support feat(v2): iterator and sub-DAG basic support Dec 7, 2021
Comment on lines -43 to -46
'inputs': {
'artifacts': [],
'parameters': {}
},
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed all the empty structs for more concise comparison

v2/Makefile Outdated
Comment on lines 40 to 43
.PHONY: test-update
test-update: mlmd
# Updating compiled argo YAML golden files...
go test ./... -- --update
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

set up a new make target to easily update v2 compiler golden compiled results

@Bobgy
Copy link
Contributor Author

Bobgy commented Dec 9, 2021

/unhold
dependent PRs have been merged

@Bobgy
Copy link
Contributor Author

Bobgy commented Dec 9, 2021

/approve
I'm merging to unblock upcoming PRs. Looking forward to follow up reviews as well.

@google-oss-prow
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Bobgy

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@Bobgy
Copy link
Contributor Author

Bobgy commented Dec 9, 2021

/retest

@Bobgy
Copy link
Contributor Author

Bobgy commented Dec 9, 2021

/test all

@google-oss-prow google-oss-prow bot merged commit 713558c into kubeflow:master Dec 9, 2021
@Bobgy Bobgy deleted the v2-iterator branch December 9, 2021 23:14
@Bobgy Bobgy mentioned this pull request Dec 10, 2021
1 task
abaland pushed a commit to abaland/pipelines that referenced this pull request May 29, 2022
* chore(samples): add loop_output_v2 sample

* feat(v2): iterator support

* refactor v2 backend compiler tests to golden files

* fix unit tests

* test(v2): verify loop_output_test MLMD data

* clean up

* fix

* fix2

* fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant