Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
DelaramRajaei authored Jul 18, 2024
1 parent 66c85b9 commit b1a3cf0
Showing 1 changed file with 15 additions and 1 deletion.
16 changes: 15 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,14 +141,28 @@ These samples are taken from an ANTIQUE dataset that has been refined using a ba
| 1509982 | How is the Chemistry is a basic of Science? | How is chemistry a principle of science | 0.75 | 0.285714 | 0.75 | 0.75 | 0 | [0.5714285714285714, 0.16666666666666666, 0.0, 0.0] | 0.651439058 | 0.7 | 7 | 10 | 0.7796929 |

### [`['rag']`](./src/param.py#L12)
Retrieval-augmented models aim to enhance response accuracy and reliability through two phases: retrieval and generation. When activated, these models use an external source like Wikipedia to search the original query and find relevant documents and content. If no relevant information is found, 'None' is returned. The original query, along with any retrieved documents, is then passed to a llm such as 't5', which generates a response.

### [`['search']`](./src/param.py#L17)
We search the relevant documents for both the original query and each of the `potential` refined queries. We need to set an information retrieval method, called ranker, that retrieves relevant documents and ranks them based on relevance scores. We integrate [`pyserini`](https://github.com/castorini/pyserini), which provides efficient implementations of sparse and dense rankers, including `bm25` and `qld` (query likelihood with Dirichlet smoothing).

### [`['rag_fusion']`](./src/refinement/refiner_param.py#L9)
If this command is activated, it will fuse the results based on the selected ['categories'](./src/param.py#L15) and ['fusion method'](./src/param.py#L16) specified in the parameters.
Categories are as follows:
- all: Considers all documents retrieved for all query variations.
- global: Considers only the documents retrieved for queries generated by global refiners.
- local: Considers only the documents retrieved for queries generated by local refiners.
- bt: Considers only the documents retrieved for queries generated by translators models.
- bt_nllb: Considers only the documents retrieved for queries generated by the nllb translator model.

Fusion methods are:
- rrf: Calculate reciprocal rank fusion for a k.
- rrf_multi_k: Calculate reciprocal rank fusion for a list of ks.
- condorcet: Calculate Condorcet Fuse.
- random: Randomly assign scores to the documents and then fuse them.

### [`['eval']`](./src/param.py#L20)
The search results of each potential refined queries are evaluated based on how they improve the performance with respect to an evaluation metric like `map` or `mrr`.
The search results of each potential refined queries are evaluated based on how they improve the performance with respect to an evaluation metric like `map`, `mrr`, or `ndcg`.


### [`['agg']`](./src/param.py#L12)
Expand Down

0 comments on commit b1a3cf0

Please sign in to comment.