-
Notifications
You must be signed in to change notification settings - Fork 36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Append the hash by default. #47
Comments
Cool, that'd be nice to have! |
B2 doesn't support chunked transfers, so this just got a lot less exciting. |
Why isn't it recommended?
What advantages would chunked transfers provide? |
I mistyped, I meant it's not recommended by Backblaze. I don't know why, and I still haven't thought of anything. It's been a while, but I think the reason I wanted chunked transfers was so that I wouldn't have to set the content-length header on upload. For byte streams (which is my use case) I'd still have to buffer the whole file somewhere, so I didn't develop this. Do you have a use-case for it? |
btw this is no longer true, it only makes one pass in this instance. |
Oh, here's a good reason: if the file changes during upload, appending the hash will happily submit the corrupted file, but a two-pass solution would have the upload be rejected as the hashes wouldn't match. |
By default, blazer will do a pass over the data to compute the hash so that it can be sent at the start of the upload request, as described in https://www.backblaze.com/b2/docs/uploading.html
Blazer can either do this in memory, which is the default, or can copy the input into a temp file, or it can just make two passes over the input if the input supports seeking. Each of these has side effects that might not be obvious to a user (e.g., if you're using blazer to copy files out of NFS, blazer will happily read every file twice).
Blazer should append the hash at the end by default. This is not recommended by blazer, but if there's a functional difference between the two options it's not obvious to me what it is.
The text was updated successfully, but these errors were encountered: