feat: improve chunkMessages function in terms of cpu/mem #153
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When I run batch consumer, kafka-konsumer runs the chunkMessages function for every incoming batch of messages (consume function in batch_consumer.go).
I wrote more effective chunkMessages.
Here the benchmark results (I used benchstat, random test data etc)
Execution Time (sec/op)
Original Function: 96.90µs per operation
Optimized Function: 48.35µs per operation
Improvement: %50 faster
Memory Usage (B/op)
Original Function: 299.1KiB per operation
Optimized Function: 170.1KiB per operation
Improvement: Approximately 43% reduction in memory usage
Number of Allocations (allocs/op)
Original Function: 809 allocations per operation
Optimized Function: 102 allocations per operation
Improvement: Approximately 87% reduction in allocations
You can also find benchmark tests, and run (make run-benchmarks)