Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: improve chunkMessages function in terms of cpu/mem #153

Merged
merged 2 commits into from
Dec 21, 2024

Conversation

Abdulsametileri
Copy link
Member

When I run batch consumer, kafka-konsumer runs the chunkMessages function for every incoming batch of messages (consume function in batch_consumer.go).

I wrote more effective chunkMessages.

Here the benchmark results (I used benchstat, random test data etc)

Execution Time (sec/op)

Original Function: 96.90µs per operation
Optimized Function: 48.35µs per operation
Improvement: %50 faster

Memory Usage (B/op)

Original Function: 299.1KiB per operation
Optimized Function: 170.1KiB per operation
Improvement: Approximately 43% reduction in memory usage

Number of Allocations (allocs/op)

Original Function: 809 allocations per operation
Optimized Function: 102 allocations per operation
Improvement: Approximately 87% reduction in allocations

You can also find benchmark tests, and run (make run-benchmarks)

@Abdulsametileri Abdulsametileri changed the base branch from v2 to batch-performance December 21, 2024 23:51
@Abdulsametileri Abdulsametileri merged commit 4e447d6 into batch-performance Dec 21, 2024
5 checks passed
Abdulsametileri added a commit that referenced this pull request Dec 29, 2024
* feat: add rolling timer implementation for batch consumer (#155)

* feat: improve chunkMessages function in terms of cpu/mem (#153)

* feat: implement rolling timer in single consumer mode
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant