Skip to content
This repository has been archived by the owner on Feb 15, 2025. It is now read-only.

docs: add ADR for RAG Evaluations Framework #842

Merged
merged 25 commits into from
Oct 4, 2024
Merged

docs: add ADR for RAG Evaluations Framework #842

merged 25 commits into from
Oct 4, 2024

Conversation

jalling97
Copy link
Contributor

@jalling97 jalling97 commented Jul 26, 2024 •
edited by justinthelaw
Loading

  • Adds the ADR for the overall RAG Evaluations Framework
  • This will be added to for each component of the RAG Evals MVP Epic and closed when all pieces are done and all decisions have been documented in the ADR.

@jalling97 jalling97 linked an issue Jul 26, 2024 that may be closed by this pull request
ADR: RAG Evaluations Framework #823
Closed
@netlify Netlify
Copy link

netlify bot commented Jul 26, 2024 •
edited
Loading

✅ Deploy Preview for leapfrogai-docs canceled.

Name Link
🔨 Latest commit 04a482f
🔍 Latest deploy log https://app.netlify.com/sites/leapfrogai-docs/deploys/66ff126d5a9ad6000840e81f

@jalling97 jalling97 self-assigned this Jul 26, 2024
@jalling97 jalling97 added the ADR 🧐 Architecture Decision Record label Jul 26, 2024
@jalling97 jalling97 changed the title (ADR) RAG Evaluations Framework adr: RAG Evaluations Framework Aug 27, 2024
@jalling97 jalling97 changed the title adr: RAG Evaluations Framework docs: RAG Evaluations Framework Aug 27, 2024
@jalling97 jalling97 added the documentation Improvements or additions to documentation label Aug 27, 2024
@jalling97 jalling97 changed the title docs: RAG Evaluations Framework docs: add RAG Evaluations Framework Aug 27, 2024
@jalling97 jalling97 changed the title docs: add RAG Evaluations Framework docs: add ADR for RAG Evaluations Framework Aug 27, 2024
barronstone
barronstone reviewed Sep 10, 2024
View reviewed changes
@jalling97 jalling97 marked this pull request as ready for review September 30, 2024 21:36
@jalling97 jalling97 requested a review from a team as a code owner September 30, 2024 21:36
justinthelaw
justinthelaw reviewed Oct 1, 2024
View reviewed changes
Copy link
Contributor

@justinthelaw justinthelaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks excellent so far! Some minor comments and nits.

jalling97 and others added 3 commits October 1, 2024 11:24
@jalling97 @justinthelaw
Update adr/0007-rag-eval-framework.md
d208e66 
Co-authored-by: Justin Law <81255462+justinthelaw@users.noreply.github.com>
@jalling97 @justinthelaw
Update adr/0007-rag-eval-framework.md
4965111 
Co-authored-by: Justin Law <81255462+justinthelaw@users.noreply.github.com>
@jalling97
Update 0007-rag-eval-framework.md
13fce77
justinthelaw
justinthelaw previously approved these changes Oct 3, 2024
View reviewed changes
Copy link
Contributor

@justinthelaw justinthelaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love it!

gphorvath
gphorvath previously approved these changes Oct 3, 2024
View reviewed changes
CollectiveUnicorn
CollectiveUnicorn suggested changes Oct 3, 2024
View reviewed changes
CollectiveUnicorn
CollectiveUnicorn reviewed Oct 3, 2024
View reviewed changes
@jalling97 jalling97 dismissed stale reviews from gphorvath and justinthelaw via c654c25 October 3, 2024 20:43
gregclark-defenseunicorns
gregclark-defenseunicorns reviewed Oct 3, 2024
View reviewed changes
Copy link
Contributor

@gregclark-defenseunicorns gregclark-defenseunicorns left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! The only thing I want to point out is the about the model cards living in UDS runtime. That is definitely an assumption and needs validation from the UDS teams in general so maybe it is worth specifying that so it is not misunderstood.

@jalling97 jalling97 merged commit e3baa4d into main Oct 4, 2024
18 checks passed
@jalling97 jalling97 deleted the 823-adr-rag-evaluations-framework branch October 4, 2024 15:28
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@barronstone barronstone barronstone left review comments

@gregclark-defenseunicorns gregclark-defenseunicorns gregclark-defenseunicorns left review comments

@justinthelaw justinthelaw justinthelaw approved these changes

@CollectiveUnicorn CollectiveUnicorn CollectiveUnicorn approved these changes

@gphorvath gphorvath gphorvath left review comments

Assignees

@jalling97 jalling97

Labels
ADR 🧐 Architecture Decision Record documentation Improvements or additions to documentation
Projects
None yet
Milestone
Current - RAG UX Enhancements | Model...
Development

Successfully merging this pull request may close these issues.

ADR: RAG Evaluations Framework
6 participants
@jalling97 @barronstone @gphorvath @justinthelaw @CollectiveUnicorn @gregclark-defenseunicorns