TestOps: How to link automated Cucumber test to Jira ticket

[mariaklimenko]

Hello! Our company is currently exploring TestOps Cloud trial and I successfully integrated it with our TeamCity and Jira Cloud. However, I'm having trouble importing our tests stored in Bitbucket (git) into Allure TestOps. Our tests are in Cucumber format and mostly automated, with plans to fully automate them. I'm unable to find a way to upload Cucumber feature files into Allure TestOps (it accepts CSV only). Our goal is to maintain the main source of tests in Bitbucket (because we develop tests in IntelliJ Idea and don't plan to use TestOps for test design) and then we need our tests to be automatically synced with Allure TestOps (in order to use test/requirements traceability to Jira and also to keep test run history). Is this possible?

[mariaklimenko]

Update: I have found how to upload Cucumber tests from test runs - I had to press the "close launch" button to import all my tests.
However, I have another issue which seems to be a show stopper if not resolved. We need to be able to upload tests results and tests with links to originating jira Stories and Bugs. I am using Cucumber tag @issue=JSMO-3520 to attach Jira to my tests, e.g.
@automated @issue=JSMO-3520
Scenario Outline: Card is set with the ValidityExpiry
Given <...>

The issue is that the Jira link directs to the "Links" section instead of "Issue Links" (refer to attached screenshot). This results in the absence of tests and launches in the Jira ticket. Manually creating the issue link resolves the integration with Jira, however, after the next test execution, the link will be deleted due to the replacement of the test with a new version. Please, help! We want to onboard TestOps mainly for the test/requirements traceability feature and reporting, so it is quite important for us

[mariaklimenko]

We are using allure-cucumber6-jvm 2.20.1

Example of allure test report in json:

[cheshi-mantu]

Hi
@issue=JSMO-3520 will work for Allure Report only.
for Allure TestOps you need to use a custom field to pass this information. See the explanation and examples here: https://docs.qameta.io/allure-testops/faq/labels/

[ameduza]

@mariaklimenko , I have a question to you, if you don't mind.
From what I see I have same need as your project based on the description. Did you manage import of not yet automated gherkin scenarios?

My initial plan was also to import them somehow, but I was told by TestOps folks that it is not smth right and I had to still populate test execution for manual tests in order to include them into the suite.

[mariaklimenko]

Hi Aleksandr. Unfortunately, I don't think Allure TestOps provide the option to upload Cucumber feature files using the portal (though their competitor Cucumber Studio does that!). I keep all my tests in Bitbucket, most of them are automated and run in TeamCity every night, and test results flow to TestOps automatically. There is one workaround that you might want to try - if you run your manual cucumber tests using a test framework (they will fail because there are some steps that are not defined or due to other reasons), and then you can upload test results into Allure TestOps. After you close the corresponding test launch, it should create manual tests in the Test Cases section

[ameduza]

Hi Maria, thanks a lot for your input.
I just tried to see how this "failed manual tests" execution looks like after importing back to the TestOps. There are two issues from what I see:

1. they are marked as automated. Yes, they all have correctly assigned @manual tag, but I can't find any way on how to convert manual TC in TestOps into manual one.
2. Imported TCs doesn't have complete scenario content. It is basically b/c test execution stopped on first failed step in a scenario and that's it. That means, that for every step in @manual scenarios seems I have to implement stub steps implementation with only repeating their definition from the initial gherkin outline.
   Here is how this failed\broken manual test looks like in TestOps:
   

@cheshi-mantu , @baev , do you have any thoughts on my issues above?

[mariaklimenko]

You can do "Detach automation" for all those failed manual tests and they will move into the "Manual" category as a result. I have no idea how to overcome your issue 2, maybe try to run tests in "dry run" mode (it is just the idea, not sure if it is going to work). Ideally, Allure guys need to implement the import from cucumber files feature... that would be awesome! And also it would be great to be able to add attachments to tests in automatic launches that failed and have been re-run manually (i.e. attach some test evidence to manually re-run tests)