Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add output schemas for runners that just have a generic response #6102

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

rebrowning
Copy link
Contributor

@rebrowning rebrowning commented Dec 22, 2023

add a generic output schema to the local, remote, and winrm runners. St2 already appears to return the data in this format, this is simply formalizing the response so that validations can be added for commands leveraging these runners to better allow users of st2 to verify the integrity of the data their pipelines are processing

I'm curious about any issues people are aware of this would cause. Initially the thought was this would cause regressions in existing jobs, but the results are identical between the new and old changes, so those existing jobs would be unaffected. As I'm new to the st2 code base and the community in general I'm curious if there are any other potential issues that would prevent this change from moving forwards.

Also if anyone knows where the appropriate location to add tests for these cases is, please lmk. I did an initial dig through the code but didn't find where the serialization was happening... I'm beginning to think it's happening in the DB itself, but can't confirm that's the case.

I've provided 4 examples below with the core.local, which uses the local-shell-cmd runner. The first two examples are what currently happens, and the second two examples are with the change. You'll notice they're equivalent, but with the change it allows for validations to be executed.

without changes

core.local -> echo "hello world"

result:

{
  "failed": false,
  "succeeded": true,
  "return_code": 0,
  "stdout": "hello world",
  "stderr": ""
}

core.local -> curl <http endpoint that returns json>

the simple http endpoint in this example returns the following object:

{
  "name": "billy",
  "age": 15,
  "addr": {
      "street": "abc",
      "number": 123,
      "state": "zz"
  }
}

result in st2:

{
  "failed": false,
  "succeeded": true,
  "return_code": 0,
  "stdout": {
    "name": "billy",
    "age": 15,
    "addr": {
      "street": "abc",
      "number": 123,
      "state": "zz"
    }
  },
  "stderr": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r100    85  100    85    0     0    123      0 --:--:-- --:--:-- --:--:--   123"
}

with the output_schema changes

core.local -> echo "hello world"

{
  "failed": false,
  "succeeded": true,
  "return_code": 0,
  "stdout": "hello world",
  "stderr": ""
}

core.local -> curl <same http endpoint as before>

result:

{
  "failed": false,
  "succeeded": true,
  "return_code": 0,
  "stdout": {
    "name": "billy",
    "age": 15,
    "addr": {
      "street": "abc",
      "number": 123,
      "state": "zz"
    }
  },
  "stderr": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r100    85  100    85    0     0    223      0 --:--:-- --:--:-- --:--:--   223"
}

Json object inside of full result

core.local -> echo "hello world" && curl <same http endpoint as before>
This will not parse to json, so the resulting stdout will just be a string.

{
  "failed": false,
  "succeeded": true,
  "return_code": 0,
  "stdout": "hello world\n{\"name\": \"billy\", \"age\": 15, \"addr\": {\"street\": \"abc\", \"number\": 123, \"state\": \"zz\"}}",
  "stderr": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r100    85  100    85    0     0    223      0 --:--:-- --:--:-- --:--:--   223"
}

@pull-request-size pull-request-size bot added the size/L PR that changes 100-499 lines. Requires some effort to review. label Dec 22, 2023
…St2 already appears to return the data in this format, this is simply formalizing the response so that validations can be added for commands leveraging these runners to better allow users of st2 to verify the integrity of the data their pipelines are processing
…hat st2 is currently outputting, so they were failing with the new schema being applied
…t's easier to find the issues... That flag can be removed if necessary
@rebrowning rebrowning force-pushed the add_output_schemas_for_shell_runners branch from 036f1cd to 1bcd9e4 Compare January 12, 2024 18:39
Comment on lines +44 to +49
stdout:
anyOf:
- type: "object"
- type: "string"
- type: "array"
- type: "number"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be any of the simple json types? Why these 4?
https://github.com/StackStorm/st2/blob/master/st2common/st2common/util/schema/action_params.json#L19

I guess we need to look at how stdout is converted from json to see if there are any restrictions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size/L PR that changes 100-499 lines. Requires some effort to review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants