Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

analyzer: configurable MaxBackoffTimeout #837

Merged
merged 1 commit into from
Dec 19, 2024

Conversation

ptrus
Copy link
Member

@ptrus ptrus commented Dec 18, 2024

This adds a configurable MaxBackoffTime to analyzers. This is useful for analyzers such as validatorstakinghistory, where once caught up, we only expect new work on epoch transitions and don't need to query every 6 seconds.

Additional note: the queries for fetching validatorstakinghistory work are not the most efficient (take ~2 seconds). I noticed this because these queries are currently using a lot of the database's CPUs—even if there's no work to be done. The idea is that with the MaxBackoffTime change, we don't need to bloat the DB with additional indexes and just be smarter about how frequently we do the queries.

Alternatives considered:

  • We could manually update the deployment configuration for validatorstakinghistory to use a "fixed-interval" once caught up. But if the analyzer falls behind (for whatever reason. e.g. node downtime etc) we would need manual intervention again to update the config. With implemented solution it should recover automatically and use a shorter timeout if there's more work to do.

  • "Epoch"-aware Item analyzer. If the ItemAnalyzer would be aware of epochs, it could then know when to call the analyzer if it is caught up. However this solution would require more (non-trivial) work to implement, which doesn't seem worth for this specific case.

@ptrus ptrus force-pushed the ptrus/feature/max-backoff-timeout branch 3 times, most recently from fce2dfa to 27b7f4e Compare December 18, 2024 17:27
@ptrus ptrus self-assigned this Dec 18, 2024
@ptrus ptrus force-pushed the ptrus/feature/max-backoff-timeout branch from 27b7f4e to 571cf83 Compare December 19, 2024 07:21
@ptrus
Copy link
Member Author

ptrus commented Dec 19, 2024

I'm merging since we want to do a new release today. I can apply the potential changes requested in a future PR.

@ptrus ptrus merged commit 0b91113 into main Dec 19, 2024
16 checks passed
@ptrus ptrus deleted the ptrus/feature/max-backoff-timeout branch December 19, 2024 07:27
@ptrus ptrus mentioned this pull request Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant