Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enforce shard index codec chain invariant #30

Merged
merged 4 commits into from
Jul 11, 2024
Merged

Conversation

zoj613
Copy link
Owner

@zoj613 zoj613 commented Jul 11, 2024

closes #15

The V3 spec insists that the codec chain for a shard index array
must not include compression codecs like Gzip since it would make
it impractical to calculate index bytes size. This commit
ensures this invariant is not violated by making a shard's index
codec chain correct by construction. That is, specifying a non-fixed
size bytes-to-bytes codec in the index_codec field is a compile time
error.

@zoj613 zoj613 added the codecs relating to codecs label Jul 11, 2024
Copy link

codecov bot commented Jul 11, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.42%. Comparing base (e2bf47c) to head (0724de8).

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #30      +/-   ##
==========================================
+ Coverage   95.01%   95.42%   +0.40%     
==========================================
  Files          13       13              
  Lines        1084     1092       +8     
==========================================
+ Hits         1030     1042      +12     
+ Misses         54       50       -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

The V3 spec insists that the codec chain for a shard index array
must not include compression codecs like Gzip since it would make
it impractical to calculate index bytes size. This commit
ensures this invariant is not violated by making a shard's index
codec chain correct by construction. That is, specifying a non-fixed
size bytes-to-bytes codec in the `index_codec` field is a compile time
error.
@zoj613 zoj613 force-pushed the index_codec_invariant branch from d47551d to 8518c7b Compare July 11, 2024 13:33
@zoj613 zoj613 marked this pull request as ready for review July 11, 2024 13:33
@zoj613 zoj613 changed the title enforce index codec invariant Enforce shard index codec chain invariant Jul 11, 2024
@zoj613 zoj613 added the enhancement New feature or request label Jul 11, 2024
@zoj613 zoj613 merged commit ef185d5 into main Jul 11, 2024
4 checks passed
@zoj613 zoj613 deleted the index_codec_invariant branch July 11, 2024 21:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
codecs relating to codecs enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Consider using GADTs to encode invariant of not using compression codecs for index codec chain.
1 participant