-
Notifications
You must be signed in to change notification settings - Fork 391
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: load packages lazily #3669
base: master
Are you sure you want to change the base?
Conversation
🛠 PR Checks SummaryAll Automated Checks passed. ✅ Manual Checks (for Reviewers):
Read More🤖 This bot helps streamline PR reviews by verifying automated checks and providing guidance for contributors and reviewers. ✅ Automated Checks (for Contributors):No automated checks match this pull request. ☑️ Contributor Actions:
☑️ Reviewer Actions:
📚 Resources:Debug
|
Codecov ReportAttention: Patch coverage is
📢 Thoughts on this report? Let us know! |
2dd7d52
to
8fef745
Compare
// Set up a PackageGetter that will do the actual preprocessing when needed | ||
pkgGetter := func(pkgPath string, store Store) (*PackageNode, *PackageValue) { | ||
// Get the original MemPackage | ||
memPkg := store.GetMemPackage(pkgPath) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a quick note: the problem with this is that nodes which are restarted have a different set of packages in the cache compared to ones which have been running with a cache; consequently, loading a DB from the store needs to happen with a clean gas meter.
I would also suggest that the best solution is one that keeps packages in the store using a cache, so that the amount of packages kept in the store is actually bounded to a specific amount
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am thinking about alternative implementations because the one in the PR seems to have issues with persistence. I'll keep this in mind whether i continue with this or go in a different direction.
Closes 3564