Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate DeepEval and ConfidentAI #9

Open
iramykytyn opened this issue Sep 11, 2024 · 2 comments
Open

Investigate DeepEval and ConfidentAI #9

iramykytyn opened this issue Sep 11, 2024 · 2 comments
Assignees

Comments

@iramykytyn
Copy link
Contributor

https://docs.confident-ai.com/
https://docs.confident-ai.com/docs/confident-ai-introduction

Try to use this tool instead of our self-written solution.
Create separate implementation of validation and prompt testing with deepeval.
Describe any difficulties or bugs discovered on the way.
Create an article or blog post describing proc and cons of using this tool for prompt testing, validation, and results visualisation comparing to current solution.
Which other problems this tool could solve for us in the future?

@eLQeR
Copy link
Collaborator

eLQeR commented Sep 25, 2024

DeepEval Documentation Review and Metric Selection

I am currently conducting an in-depth review of the DeepEval documentation, focusing on the detailed specifications of various evaluation metrics. My goal is to determine which metric best suits our specific test case. I have also implemented the initial version of testing through DeepEval, as shown in the attached screenshot, and successfully executed the first tests on the dataset. Moving forward, I plan to continue deepening my expertise in this area and refining the testing process.
image
image
image
image

@eLQeR
Copy link
Collaborator

eLQeR commented Sep 29, 2024

I have successfully updated the CSV file testing functionality, expanded the application's overall functionality, and implemented real-time manual testing of LLM responses using DeepEval metrics.
image
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

When branches are created from issues, their pull requests are automatically linked.

2 participants