Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add new test suite for context merging #293

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
+85 −0
Open

feat: Add new test suite for context merging #293

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a13fa4c
feat: Add new test suite for context merging #13
chrfwow Feb 6, 2025
bfcc985
add tags
chrfwow Feb 25, 2025
b78543b
add even more tags
chrfwow Feb 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions specification/assets/gherkin/contextMerging.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
Feature: Context merging precedence

Background:
Given a stable provider with retrievable context is registered

Scenario Outline: A context entry is added to a single level
Given A context entry with key "key" and value "value" is added to the "<level>" level
When Some flag was evaluated
Then The merged context contains an entry with key "key" and value "value"

Examples:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we might want to start adding tags here, not all sdks support eg. hooks or transaction - hence i think maybe adding a tag for context merges and the level would be good something

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, but here we also have the problem that tags can only be used on a whole table, not on individual lines inside a table

| level |
| API |
| Transaction |
| Client |
| Invocation |
| Before Hooks |

Scenario: A context entry is added to each level with different keys
Given A context entry with key "API" and value "API value" is added to the "API" level
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you prefer the same line with different values each time, or a data table for the Given and Then parts of this scenario?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A datatable could here be beneficial, can we tag on datatables, because we do have the same problem as above, what if not all levels are supported by the sdk, how can we limit this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apparently tags can only enable/disable a whole table, not individual lines in a table. So we would need a table for every permutation of possible implementations if I understood that correctly.
Alternatively I could add a tag for each of the levels, or at least for transaction and hook

Copy link
Member

@aepfli aepfli Feb 26, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i am not sure tags work on steps, normally they only work on scenarios or examples. from the docs at https://cucumber.io/docs/cucumber/api#tags:

Tags can be placed above the following Gherkin elements:

    Feature
    Rule
    Scenario
    Scenario Outline
    Examples

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, that's a problem. So how can we easily enable/disable these steps? Should we really duplicate all tests?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long as we only need to duplicate gherkin tests, and not have additional effort in code, i think that is a feasible solution. Gherkin scenarios are cheap, not a lot of complexity and easy to grasp, there is no code to understand nor bigger complexity to grasp.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried something new by dublicating most of the tests. I don't really like this solution that much, but we should now at least be able to support implementations with and without hooks and/or transactions. Refactoring this into a Datatable would make the code implementation a bit more complicated I think, so I left it for now as individual steps

And A context entry with key "Transaction" and value "Transaction value" is added to the "Transaction" level
And A context entry with key "Client" and value "Client value" is added to the "Client" level
And A context entry with key "Invocation" and value "Invocation value" is added to the "Invocation" level
And A context entry with key "Before Hooks" and value "Before Hooks value" is added to the "Before Hooks" level
When Some flag was evaluated
Then The merged context contains an entry with key "API" and value "API value"
And The merged context contains an entry with key "Transaction" and value "Transaction value"
And The merged context contains an entry with key "Client" and value "Client value"
And The merged context contains an entry with key "Invocation" and value "Invocation value"
And The merged context contains an entry with key "Before Hooks" and value "Before Hooks value"

Scenario Outline: A context entry in one level overwrites values with the same key from preceding levels
Given A table with levels of increasing precedence
| API |
| Transaction |
| Client |
| Invocation |
| Before Hooks |
And Context entries for each level from API level down to the "<level>" level with key "key" and value "<level>"
When Some flag was evaluated
Then The merged context contains an entry with key "key" and value "<level>"

Examples:
| level |
| API |
| Transaction |
| Client |
| Invocation |
| Before Hooks |
Loading