Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(tm2/pkg/cmap): add benchmarks to show true impact of contention #3540

Merged

Conversation

odeke-em
Copy link
Contributor

The Go standard library's sync.Map is touted as great for cases with high load and is commonly known knowledge but the benchmark that I am committing shows otherwise that for this library's usage, it is so much more expensive hence this benchmark will avoid someone committing sync.Map without seeing the true implications.

$ benchstat map_w_mutex.txt stdlib_sync_map.txt
name                               old time/op    new time/op    delta
CMapConcurrentInsertsDeletesHas-8     1.72s ±11%     1.92s ± 3%   +11.66%  (p=0.000 n=10+9)
CMapHas-8                             109ns ± 9%     118ns ± 3%    +8.26%  (p=0.002 n=10+8)

name                               old alloc/op   new alloc/op   delta
CMapConcurrentInsertsDeletesHas-8    1.18GB ± 2%    3.21GB ± 3%  +172.09%  (p=0.000 n=10+10)
CMapHas-8                             16.0B ± 0%     16.0B ± 0%      ~     (all equal)

name                               old allocs/op  new allocs/op  delta
CMapConcurrentInsertsDeletesHas-8      824k ± 0%     4433k ± 0%  +437.89%  (p=0.000 n=10+10)
CMapHas-8                              2.00 ± 0%      1.60 ±38%      ~     (p=0.065 n=9+10)

Updates #3505

@Gno2D2
Copy link
Collaborator

Gno2D2 commented Jan 17, 2025

🛠 PR Checks Summary

All Automated Checks passed. ✅

Manual Checks (for Reviewers):
  • IGNORE the bot requirements for this PR (force green CI check)
  • The pull request description provides enough details (checked by @n2p5)
Read More

🤖 This bot helps streamline PR reviews by verifying automated checks and providing guidance for contributors and reviewers.

✅ Automated Checks (for Contributors):

🟢 Maintainers must be able to edit this pull request (more info)
🟢 Pending initial approval by a review team member, or review from tech-staff

☑️ Contributor Actions:
  1. Fix any issues flagged by automated checks.
  2. Follow the Contributor Checklist to ensure your PR is ready for review.
    • Add new tests, or document why they are unnecessary.
    • Provide clear examples/screenshots, if necessary.
    • Update documentation, if required.
    • Ensure no breaking changes, or include BREAKING CHANGE notes.
    • Link related issues/PRs, where applicable.
☑️ Reviewer Actions:
  1. Complete manual checks for the PR, including the guidelines and additional checks if applicable.
📚 Resources:
Debug
Automated Checks
Maintainers must be able to edit this pull request (more info)

If

🟢 Condition met
└── 🟢 The pull request was created from a fork (head branch repo: odeke-em/gno)

Then

🟢 Requirement satisfied
└── 🟢 Maintainer can modify this pull request

Pending initial approval by a review team member, or review from tech-staff

If

🟢 Condition met
└── 🟢 Not (🔴 Pull request author is a member of the team: tech-staff)

Then

🟢 Requirement satisfied
└── 🟢 If
    ├── 🟢 Condition
    │   └── 🟢 Or
    │       ├── 🟢 At least 1 user(s) of the organization reviewed the pull request (with state "APPROVED")
    │       ├── 🟢 At least 1 user(s) of the team tech-staff reviewed pull request
    │       └── 🔴 This pull request is a draft
    └── 🟢 Then
        └── 🟢 Not (🔴 This label is applied to pull request: review/triage-pending)

Manual Checks
**IGNORE** the bot requirements for this PR (force green CI check)

If

🟢 Condition met
└── 🟢 On every pull request

Can be checked by

  • Any user with comment edit permission
The pull request description provides enough details

If

🟢 Condition met
└── 🟢 And
    ├── 🟢 Not (🔴 Pull request author is a member of the team: core-contributors)
    └── 🟢 Not (🔴 Pull request author is user: dependabot[bot])

Can be checked by

  • team core-contributors

@odeke-em odeke-em changed the title test(tm2/pkg/cmap): adds benchmarks to show true impact of contention test(tm2/pkg/cmap): add benchmarks to show true impact of contention Jan 17, 2025
Copy link

codecov bot commented Jan 17, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

📢 Thoughts on this report? Let us know!

@notJoon notJoon added the review/triage-pending PRs opened by external contributors that are waiting for the 1st review label Jan 17, 2025
@jefft0
Copy link
Contributor

jefft0 commented Jan 22, 2025

@odeke-em , can you resolve the CI check error for "TM2 / Run TM2 suite / Go Lint" ?

Copy link
Member

@thehowl thehowl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good overall

tm2/pkg/cmap/cmap_test.go Outdated Show resolved Hide resolved
@Gno2D2 Gno2D2 requested a review from a team January 23, 2025 17:04
@Gno2D2 Gno2D2 removed the review/triage-pending PRs opened by external contributors that are waiting for the 1st review label Jan 23, 2025
@odeke-em odeke-em force-pushed the cmap-add-benchmarks-to-test-contention branch 2 times, most recently from dba0590 to 4820bc8 Compare January 24, 2025 07:26
The Go standard library's sync.Map is touted as great for cases with
high load and is commonly known knowledge but the benchmark that I am
committing shows otherwise that for this library's usage, it is so much
more expensive hence this benchmark will avoid someone committing
sync.Map without seeing the true implications.

```shell
$ benchstat map_w_mutex.txt stdlib_sync_map.txt
name                               old time/op    new time/op    delta
CMapConcurrentInsertsDeletesHas-8     1.72s ±11%     1.92s ± 3%   +11.66%  (p=0.000 n=10+9)
CMapHas-8                             109ns ± 9%     118ns ± 3%    +8.26%  (p=0.002 n=10+8)

name                               old alloc/op   new alloc/op   delta
CMapConcurrentInsertsDeletesHas-8    1.18GB ± 2%    3.21GB ± 3%  +172.09%  (p=0.000 n=10+10)
CMapHas-8                             16.0B ± 0%     16.0B ± 0%      ~     (all equal)

name                               old allocs/op  new allocs/op  delta
CMapConcurrentInsertsDeletesHas-8      824k ± 0%     4433k ± 0%  +437.89%  (p=0.000 n=10+10)
CMapHas-8                              2.00 ± 0%      1.60 ±38%      ~     (p=0.065 n=9+10)
```

Updates gnolang#3505
@odeke-em odeke-em force-pushed the cmap-add-benchmarks-to-test-contention branch from 4820bc8 to e37fe61 Compare January 31, 2025 20:27
@n2p5 n2p5 merged commit c24f69f into gnolang:master Jan 31, 2025
60 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
📦 🌐 tendermint v2 Issues or PRs tm2 related
Projects
Development

Successfully merging this pull request may close these issues.

6 participants