[98] add query for curator comments

[stanislaw-zakrzewski]

Solves: #98

Changes:

• Free-text search now is capable of searching through the curator comments
• Highlighter now works with combinations of terms (quoted)
• DataGuideDialog component content was updated

Screenshots:

Searching for a case with free-text (content in curators comment):

Updated DataGuideDialog content:

[shaileshmahajanBCH]

Hello @stanislaw-zakrzewski ,

• The data guide and single word search looks good.
  

• The indexes look good
  

• I do see following issue with search for terms like hello $, hello /, hello, this. Can you please take a look if there are some restrictions on search terms

[stanislaw-zakrzewski]

Hey @shaileshmahajanBCH
There were some issues regarding the filers and text search combinations. Hopefully I managed to resolve them as well as searching for special characterd.

[shaileshmahajanBCH]

Thanks @stanislaw-zakrzewski for all the work on this PR.

• Did notice that the search is not returning case records some times. I have 2 cases with the word "this" in it, But search result does not return that case

[stanislaw-zakrzewski]

Thanks @stanislaw-zakrzewski for all the work on this PR.

• Did notice that the search is not returning case records some times. I have 2 cases with the word "this" in it, But search result does not return that case

@shaileshmahajanBCH Initially I thought that it is an issue of parsing a query, but it is much more interesting 😮

The problem with searching for this term is actually part of the MongoDB feature. When we create indexes for the text search we specify the language:

    {
        "name": "commentIdx",
        "key": {
            "comment": -1
        },
        "collation": {
            "locale": "en_US",
            "strength": 2
        }
    },

According to documentation one of the features of text search with specified language (locale) is dropping "stop words" from search query: https://www.mongodb.com/docs/v5.3/core/index-text/#supported-languages-and-stop-words (full list of stop words for English language: https://github.com/igorbrigadir/stopwords/blob/master/en/mongodb.txt). And one of the stop words is this, which causes MongoDB to ignore it in a search query.

We can resolve this problem by disabling all language specific features (stemming and stop words) by adding "default_language": "none" to textIdx definition:

    {
        "name": "textIdx",
        "key": {
            "demographics.occupation": "text",
            "location.country": "text",
            "location.admin1": "text",
            "location.admin2": "text",
            "location.admin3": "text",
            "caseReference.sourceUrl": "text",
            "caseStatus": "text",
            "comment": "text"
        },
        "default_language": "none"
    },

And now we can search for this:

But I'm not sure we should do that. Maybe we can just add a disclaimer that some of the "meaningless" words are not going to be taken care of by the search?

[shaileshmahajanBCH]

@stanislaw-zakrzewski You are correct. Stop words are not indexed.

• This link says the same - https://www.mongodb.com/docs/manual/core/indexes/index-types/index-text/specify-text-index-language/

• May be quickly run it by Aimee or Product person to get their blessings, As they may need to know/document the stop words.