Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add custom pickling to ItemList for compactness #460

Merged
merged 2 commits into from
Jul 31, 2024

Conversation

mdekstrand
Copy link
Member

This adds custom pickle logic to ItemList, so that it doesn't try to pickle the entire vocabulary when we send an item list across the wire.

@mdekstrand mdekstrand added this to the 2024.1 milestone Jul 31, 2024
@mdekstrand mdekstrand added the data Data management support. label Jul 31, 2024
Copy link

The GitHub 🤖 has run the tests on your PR.

Covered 100.00% of diff (coverage changed -0.01% from 92.12% to 92.11%).

origin/main...HEAD, staged and unstaged changes
  • lenskit/lenskit/data/items.py (89.5%): Missing lines 346-347

Summary

  • Total: 19 lines
  • Missing: 2 lines
  • Coverage: 89%
Source Coverage Report
Name Stmts Miss Cover
lenskit-funksvd/lenskit/funksvd.py 187 8 96%
lenskit-hpf/lenskit/hpf.py 24 0 100%
lenskit-implicit/lenskit/implicit.py 94 9 90%
lenskit/lenskit/algorithms/__init__.py 67 8 88%
lenskit/lenskit/algorithms/als/__init__.py 3 0 100%
lenskit/lenskit/algorithms/als/common.py 129 2 98%
lenskit/lenskit/algorithms/als/explicit.py 121 3 98%
lenskit/lenskit/algorithms/als/implicit.py 112 1 99%
lenskit/lenskit/algorithms/basic.py 161 4 98%
lenskit/lenskit/algorithms/bias.py 150 3 98%
lenskit/lenskit/algorithms/knn/__init__.py 3 0 100%
lenskit/lenskit/algorithms/knn/item.py 303 17 94%
lenskit/lenskit/algorithms/knn/user.py 178 11 94%
lenskit/lenskit/algorithms/mf_common.py 61 0 100%
lenskit/lenskit/algorithms/ranking.py 75 11 85%
lenskit/lenskit/algorithms/svd.py 75 4 95%
lenskit/lenskit/batch/__init__.py 2 0 100%
lenskit/lenskit/batch/_predict.py 30 2 93%
lenskit/lenskit/batch/_recommend.py 46 4 91%
lenskit/lenskit/crossfold.py 136 2 99%
lenskit/lenskit/data/__init__.py 8 0 100%
lenskit/lenskit/data/checks.py 37 0 100%
lenskit/lenskit/data/dataset.py 364 19 95%
lenskit/lenskit/data/fetch.py 38 28 26%
lenskit/lenskit/data/items.py 147 3 98%
lenskit/lenskit/data/matrix.py 115 5 96%
lenskit/lenskit/data/movielens.py 96 18 81%
lenskit/lenskit/data/mtarray.py 57 3 95%
lenskit/lenskit/data/tables.py 25 0 100%
lenskit/lenskit/data/vocab.py 88 7 92%
lenskit/lenskit/diagnostics.py 4 0 100%
lenskit/lenskit/math/__init__.py 0 0 100%
lenskit/lenskit/math/solve.py 6 0 100%
lenskit/lenskit/metrics/__init__.py 0 0 100%
lenskit/lenskit/metrics/predict.py 32 0 100%
lenskit/lenskit/metrics/topn.py 212 1 99%
lenskit/lenskit/parallel/__init__.py 4 0 100%
lenskit/lenskit/parallel/chunking.py 20 1 95%
lenskit/lenskit/parallel/config.py 65 8 88%
lenskit/lenskit/parallel/invoker.py 31 2 94%
lenskit/lenskit/parallel/pool.py 54 9 83%
lenskit/lenskit/parallel/sequential.py 22 0 100%
lenskit/lenskit/parallel/serialize.py 51 1 98%
lenskit/lenskit/parallel/worker.py 43 3 93%
lenskit/lenskit/topn.py 109 25 77%
lenskit/lenskit/util/__init__.py 72 19 74%
lenskit/lenskit/util/envcheck.py 57 44 23%
lenskit/lenskit/util/logging.py 19 0 100%
lenskit/lenskit/util/random.py 26 3 88%
lenskit/lenskit/util/test.py 103 19 82%
lenskit/lenskit/util/timing.py 28 0 100%
TOTAL 3890 307 92%

@mdekstrand mdekstrand merged commit 4bd8df1 into lenskit:main Jul 31, 2024
38 checks passed
@mdekstrand mdekstrand deleted the feature/item-list-pickle branch July 31, 2024 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data Data management support.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

1 participant