Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add lexical/lambda default configuration for docs search #192

Open
eskibars opened this issue Dec 6, 2023 · 2 comments
Open

Add lexical/lambda default configuration for docs search #192

eskibars opened this issue Dec 6, 2023 · 2 comments
Assignees
Labels
enhancement New feature or request

Comments

@eskibars
Copy link
Contributor

eskibars commented Dec 6, 2023

Idea

Technical documentation in particular tends to have a lot of terms that may have never shown up in the neural net training data. This means that not only having a non-zero value for lambda is important, but that actually it needs to be higher than average typically for the best effect. I recommend we start with a value of 0.1 for now

cc @pwoznic -- we also should actually improve the documentation around lambda. Right now it's bundled into hybrid and it's not clear how to really use it from the docs without tab-switching to the playground

@cjcenizal
Copy link
Collaborator

When we experiment with various lambda values, we can try searching for “textless”, “custom dimensions”, and “query” to see how the quality of the results changes.

@eskibars
Copy link
Contributor Author

eskibars commented Dec 6, 2023

@cjcenizal yes, we can test with these, but there are even more extreme examples of things that are even rarer. e.g. searching for lxml or epub should find https://docs.vectara.com/docs/api-reference/indexing-apis/file-upload/file-upload-filetypes, 272725718 should find the MMR reranker. AdminService should find https://docs.vectara.com/docs/api-reference/admin-apis/admin

@pwoznic pwoznic added the enhancement New feature or request label Aug 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants