Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve search performance #227

Merged
merged 5 commits into from
Jan 11, 2025
Merged

Improve search performance #227

merged 5 commits into from
Jan 11, 2025

Conversation

spq
Copy link
Owner

@spq spq commented Jan 8, 2025

This patchset improves the performance of compatible searches heavily (from a couple of seconds to ~300ms in my current testset)
Searches using or'ed conditions are currently not compatible but it would not be too difficult to support that.

spq added 3 commits January 7, 2025 20:22
…ming search if those can not reach the result list:

if the sorting criteria has a lookup and we are able to iterate the possible streams in the order described by that lookup, we can abort the search as soon as the result list is full
@peace-maker
Copy link
Collaborator

What kind of queries are faster with this? Tracking execution time of some queries in ci would be nice to see regressions over time

@spq
Copy link
Owner Author

spq commented Jan 8, 2025

The empty query was my primary test case

spq added 2 commits January 9, 2025 23:06
- adds support for or-connected queries
- disables using the sort lookup if not useful (no limit)
- perform some searches in file order to make loading more efficient if no early exit is possible
@spq spq merged commit 259a39e into main Jan 11, 2025
4 checks passed
@spq spq deleted the performance_improvements branch January 11, 2025 12:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants