RRE rre-report:report produces no results

[risdenk]

RRE - No evaluation data has been generated - no reports will be produced.

katas/000_setting_up_chorus.md and katas/001_optimize_a_query.md both have a section on using RRE to evaluate search results, but it looks to fail silently. Issue #92 I think shows a symptom of this failure as well

docker-compose run rre mvn rre-report:report
[+] Running 5/0
 ⠿ Container keycloak  Running                                                          0.0s
 ⠿ Container zoo3      Running                                                          0.0s
 ⠿ Container zoo2      Running                                                          0.0s
 ⠿ Container zoo1      Running                                                          0.0s
 ⠿ Container solr1     Running                                                          0.0s
[INFO] Scanning for projects...
[INFO]
[INFO] --------------< org.querqy.chorus:chorus-rre-regression >---------------
[INFO] Building chorus-rre-regression 1.0
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- rre-maven-report-plugin:1.1:report (default-cli) @ chorus-rre-regression ---
[INFO] No evaluation data has been generated - no reports will be produced.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.665 s
[INFO] Finished at: 2022-01-24T18:59:21Z
[INFO] ------------------------------------------------------------------------

this doesn't seem to change even if the evaluation is run in the same docker-compose run:

docker-compose exec rre mvn rre:evaluate rre-report:report

The same error occurs even though RRE looks like it ran correctly in the evaluate stage.

Potentially relates to SeaseLtd/rated-ranking-evaluator#140

[epugh]

Theoretically the /select end point doesn't have any authentication required...? Looking at https://github.com/querqy/chorus/blob/master/solr/security.json I think that RRE should be able to connect? I can look a bit more.

[epugh]

@risdenk I've banged my head on security.json and jsut tested it again, and the anonymous isn't working... Want to take a crack at it? Sigh.

[david-fisher]

curl 'http://localhost:8983/solr/ecommerce/select?q=laptop' works anonymously just fine from the terminal and as a URL in Safari, so I don't think the RRE problem is actually due to authentication alone.

On further inspection, RRE supported SOLR version is 8.3.0 (ref rre/pom.xml, https://github.com/SeaseLtd/rated-ranking-evaluator/wiki/Supported%20Versions), while the SOLR version in the container is 9.0.0 (ref solr/Dockerfile). While none of the major changes (https://github.com/apache/solr/blob/main/solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-9.adoc) to the security section look like they broke RRE, it is a possibility. None of the changes between 8.3.0 and 8.11.0 look to be the culprit.

Backing out of the SOLR 9.0 changes might verify that the problem is due to SOLR version number.

[epugh]

So, for fun I ran:

docker-compose run rre mvn rre:evaluate -X

The -X enables debugging, and it appears looking at the HTTP traffic (yeah, looking at traffic now) that we do get results back!

[DEBUG] http-outgoing-1 << "- Notebook/Laptop Computer Carrying Cases & Bags[0xf6][0x8]@8[0x8]f[0xb][0xa3][0xf4]%32477[0xf5]? Targus SPRINT BACKPAC notebook case 38.1 cm (15") Backpack case[0xf6][0x8]@6^[0xb4][0xb][0xa3][0xf4]%53558[0xf5]?+Targus Sprint Backpac nylon blue notebook case 38.1 cm (15") Backpack case[0xf6][0x8]@6Q[0xe][0xb][0xa3][0xf4]&458486[0xf5]?[0x3]Newstar Notebook Riser Transparent[

So the WARN I see at the end:

[DEBUG] Connection released: [id: 1][route: {}->http://solr1:8983][state: class org.apache.solr.client.solrj.impl.HttpSolrClient][total kept alive: 1; route allocated: 2 of 10000; total allocated: 2 of 10000]
[DEBUG] Connection released: [id: 0][route: {}->http://solr1:8983][state: class org.apache.solr.client.solrj.impl.HttpSolrClient][total kept alive: 2; route allocated: 2 of 10000; total allocated: 2 of 10000]
[INFO ]   ... completed all 2 evaluations.
[INFO ] RRE: External Apache Solr Evaluation complete - preparing for shutdown
[WARN ] No queries recorded - returning empty evaluation

makes me wonder if we don't have permissions for RRE, in Docker, to write out the data?

[epugh]

Closing as we removed RRE.