Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Community radar/feed API performance optimisation #750

Merged
merged 5 commits into from
Jan 10, 2025

Conversation

shadrach-tayo
Copy link
Contributor

Closes #722

Description of the Problem / Feature

- Call communityService.getCommunityRadar:
    - Largeish 4 table join to get node attestations et al
    - Takes between 1 and >2.5s depending on db cache
- async map attestationService.getNodeEngagementSignals over all nodes:
    - aggregate engagement signals for each node with a largeish join query
    - attach the aggregate signals to the node objects
- map resolveLatestNode over all nodes:
    - node discovery query
    - node versions query
    - fetch manifest over IPFS
    - fall back to repo manifest fetch
- sort resulting (node+engagement+manifest) combo by engagements and claim timestamps

Explanation of the solution

Community radar API performance improvements
Create a new table called community-radar-entry(nodes + community joins)

When a node joins a community radar we add an entry to this table
when a node leaves we remove the entry
When the node radar engagement signal is updated we update radar-entry
the radar entry has columns for lastClaimedAt(last time an entry attestation was claimed), verifications, comments, reactions for this node
When we query we sort by verification count + lastClaimedAt column
Implement cursor pagination on table queries
Prerequisites:
Create database schema + migration
Implement script to generate entries for the community-radar-entry table from Node attestations table

Instructions on making this work

After merging make sure to run this script
yarn yarn script:backfill-radar

Note: PR contains database migration

UI changes for review

When major UI changes will happen with this PR, please include links to URLS to compare or screenshots demonstrating the difference and notify design

@shadrach-tayo shadrach-tayo merged commit 96ef618 into develop Jan 10, 2025
1 check passed
@shadrach-tayo shadrach-tayo deleted the tay/community-radar branch January 10, 2025 11:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Community routes performance issues
1 participant