Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build component in Pipelines SDK V2 to run in vertex ai pipelines #3244

Closed
rafaelgildin opened this issue May 5, 2022 · 3 comments
Closed

Comments

@rafaelgildin
Copy link

Folowwing this doc I'm trying to create a json to run my kedro pipeline in vertex ai, but when I generate the json end_date variable isn't recognize as an input like node_name and env. So, how can I pass end_date inside create_step_get_lines ?

My variable create_step_get_lines inside build_kubeflow_pipeline.py file:
create_step_get_lines = components.load_component_from_text("""
name: """ + name + """
inputs:
- {name: run_id, type: String}
- {name: end_date, type: String}
- {name: node_name, type: String}
- {name: env, type: String}
implementation:
container:
image: MY_IMAGE
command: [
kedro, run, --node, {inputValue: node_name},
--env, {inputValue: env},
--params, "run_id: """ + run_id + """,end_date: {inputValue: end_date}"]
""")

The generated json from running python build_kubeflow_pipeline.py MY_IMAGE:
"command": [
"kedro",
"run",
"--node",
"{{$.inputs.parameters['node_name']}}",
"--env",
"{{$.inputs.parameters['env']}}",
"--params",
"run_id: 2022-05-03-17-14-56,end_date: {inputValue: end_date}"
],
"image": MY_IMAGE

@varodrig
Copy link
Contributor

@rafaelgildin closing this issue since the webpage has moved and updated to https://www.kubeflow.org/docs/components/pipelines/user-guides/components/compose-components-into-pipelines/
please let us know if you need anything or still facing any issues.

@varodrig
Copy link
Contributor

/close

Copy link

@varodrig: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants