Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract common object store key builder helpers #2438

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

pcholakov
Copy link
Contributor

@pcholakov pcholakov commented Dec 17, 2024

This PR factors out the object store key construction logic into a few methods that are easier to follow and reuse.

Closes: #2389

Copy link

github-actions bot commented Dec 17, 2024

Test Results

  7 files  ±0    7 suites  ±0   4m 30s ⏱️ -9s
 47 tests ±0   46 ✅ ±0  1 💤 ±0  0 ❌ ±0 
182 runs  ±0  179 ✅ ±0  3 💤 ±0  0 ❌ ±0 

Results for commit 9d41368. ± Comparison against base commit 1ac1f70.

♻️ This comment has been updated with latest results.

@pcholakov pcholakov force-pushed the snapshots/object-key-construction-refactoring branch from 269f4f1 to 45583ac Compare December 23, 2024 15:45
@pcholakov pcholakov force-pushed the snapshots/object-key-construction-refactoring branch from 45583ac to aea27d7 Compare December 23, 2024 16:37
@pcholakov pcholakov marked this pull request as ready for review December 23, 2024 16:38
Copy link
Contributor

@muhamadazmy muhamadazmy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much @pcholakov for this PR. The changes looks good to me.
I think also it can be handy to use Path in place of String for paths. It can be very handy specially when using it with .child() method where u can chain paths naturally without the need to use format! string.

let prefix = Path::from("prefix");
let child1 = prefix.child("child1");
let deep = prefix.child("child2").child("deep");

pub fn from_snapshot(snapshot: &PartitionSnapshotMetadata, path: String) -> Self {
pub fn from_snapshot(
snapshot: &PartitionSnapshotMetadata,
snapshot_unique_key: String,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really recommend to create a wrapper type on top of String. For example struct PaddedSnapshotKey(String) or even better struct PaddedSnapshotKey{min_applied_lsn, snapshot_id} and then build string when it's needed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this a lot! 👍

@pcholakov
Copy link
Contributor Author

The reason I've been avoiding Path/PathBuf even though they're a very logical fit, is to avoid dealing with OsStr conversions which add noise. The object_store::path::Path object takes a String but it doesn't have an equivalent to the join method of std::path::Path, hence all the format! string munging. Let me play around with thin a wrapper over object_store::path::Path that adds a simple join operation, and see how that turns out.

@muhamadazmy
Copy link
Contributor

The join method for the object_store::path::Path is child. It acts in similar way to the std::Path::PathBuf::join

@pcholakov
Copy link
Contributor Author

🤦‍♂️

@pcholakov
Copy link
Contributor Author

Thank you for the suggestions @muhamadazmy, this is much better!

@pcholakov pcholakov requested a review from muhamadazmy January 3, 2025 16:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Refactor object store snapshot key creation logic to reduce duplication
2 participants