You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In #1665 we also discussed evaluation of retrieval and potentially using the CISI dataset for Information Retrieval for benchmarking but the Issue was scoped down to focus on the read and write performance to highlight the tradeoffs to users.
This ticket is more comprehensive and should have subtickets to coordinate the complexity involved.
💡 Why is this needed? What if we don't build it?
This is needed to give more information to users when using VectorDB providers, as it'll influence default choices for inline and what providers we recommend to users for remote.
Other thoughts
N/A
The text was updated successfully, but these errors were encountered:
🚀 Describe the new functionality needed
As mentioned in #1165 (comment), there's interest in creating a comprehensive benchmark of VectorDBs for Llama Stack.
The folks at @zilliztech have created https://github.com/zilliztech/VectorDBBench which is an extensive comparison.
In #1665 we also discussed evaluation of retrieval and potentially using the CISI dataset for Information Retrieval for benchmarking but the Issue was scoped down to focus on the read and write performance to highlight the tradeoffs to users.
This ticket is more comprehensive and should have subtickets to coordinate the complexity involved.
💡 Why is this needed? What if we don't build it?
This is needed to give more information to users when using VectorDB providers, as it'll influence default choices for inline and what providers we recommend to users for remote.
Other thoughts
N/A
The text was updated successfully, but these errors were encountered: