Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: don't print relevance score by default #11

Merged
merged 2 commits into from
Feb 5, 2025

Conversation

Miyou
Copy link
Contributor

@Miyou Miyou commented Feb 4, 2025

For post-processing RAG output with an LLM we don't want to influence the LLM by including the relevance score.

See this PR for the post-processing: ErikBjare/gptme#435


Important

Adds --print-relevance flag to search command in gptme_rag/cli.py to control printing of relevance scores, defaulting to not printing them.

  • Behavior:
    • Adds --print-relevance flag to search command in gptme_rag/cli.py to control printing of relevance scores.
    • By default, relevance scores are not printed unless --print-relevance is specified.
  • Functions:
    • Updates search() function to include print_relevance parameter and conditionally print relevance scores in get_expanded_content() and summary view.

This description was created by Ellipsis for 1aa5797. It will automatically update as commits are pushed.

Copy link

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Reviewed everything up to 1aa5797 in 54 seconds

More details
  • Looked at 43 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 5 drafted comments based on config settings.
1. gptme_rag/cli.py:477
  • Draft comment:
    Include print_relevance condition here so that relevance is printed only when requested. Consider adding a brief comment explaining that relevance is intentionally hidden by default for LLM processing.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
2. gptme_rag/cli.py:538
  • Draft comment:
    Guard the relevance printing in summary view with the print_relevance flag. This ensures consistency with the full view update.
  • Reason this comment was not posted:
    Comment looked like it was already resolved.
3. gptme_rag/cli.py:358
  • Draft comment:
    Good addition of the '--print-relevance' flag. Consider updating the search command's docstring to mention that relevance scores are hidden by default unless this flag is set.
  • Reason this comment was not posted:
    Confidence changes required: 0%
    None
4. gptme_rag/cli.py:483
  • Draft comment:
    The condition now checks both 'distances' and 'print_relevance' before printing the relevance score in full format. This meets the intended behavior.
  • Reason this comment was not posted:
    Confidence changes required: 0%
    None
5. gptme_rag/cli.py:535
  • Draft comment:
    Similarly, in the summary view the check is updated to ensure relevance is printed only when explicitly requested. This change is consistent with the new flag.
  • Reason this comment was not posted:
    Confidence changes required: 0%
    None

Workflow ID: wflow_DNii9swN1saNOhBc


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@Miyou Miyou force-pushed the miyou/dont-print-relevance branch from 1aa5797 to eba8be5 Compare February 4, 2025 14:57
Copy link
Owner

@ErikBjare ErikBjare left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a nit about parameter naming

@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 0% with 4 lines in your changes missing coverage. Please review.

Project coverage is 48.16%. Comparing base (6c1900f) to head (eba8be5).

Files with missing lines Patch % Lines
gptme_rag/cli.py 0.00% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master      #11      +/-   ##
==========================================
- Coverage   48.23%   48.16%   -0.08%     
==========================================
  Files          10       10              
  Lines        1362     1364       +2     
==========================================
  Hits          657      657              
- Misses        705      707       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ErikBjare ErikBjare merged commit b21a189 into ErikBjare:master Feb 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants