Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose .abort() method in S3OutputStream for cancelling partially completed uploads #1199

Closed
zhemaituk opened this issue Aug 27, 2024 · 3 comments
Labels
component: s3 S3 integration related issue type: enhancement Smaller enhancement in existing integration
Milestone

Comments

@zhemaituk
Copy link
Contributor

Type: Feature

Is your feature request related to a problem? Please describe.
If writing a file through S3OutputStream has started - there is no possibility to abort the upload.
My scenario:

  1. User uploads large file via UI
  2. Backend accepts the file, starts validating file and proxy writes it to S3 for storage
  3. Something goes wrong (user abandons the upload, file turns out invalid, etc.)

Sample code:

try (
    S3OutputStream s3Stream = inMemoryBufferingS3StreamProvider.create(uploadLocation,
            ObjectMetadata.builder()
                    .contentType("text/csv")
                    .build());
    CSVWriter writer = new CSVWriter(new OutputStreamWriter(s3Stream))
) {
    try {
        String[] headers = reader.readNext();
        validateHeaders(headers);
        writer.writeNext(headers);

        String[] row;
        while ((row = reader.readNext()) != null) {
            // Further data validation is done at this stage.
            writer.writeNext(row);
        }
    } catch (CsvValidationException|IOException e) {
        // TODO: Abort the upload?!
        // TODO: s3Stream.abort();
    }
}

Describe the solution you'd like
New method S3OutputStream#abort which can be called if the stream is not closed yet.

Describe alternatives you've considered

  1. Use reflection to call InMemoryBufferingS3OutputStream#abortMultiPartUpload
  2. Delete the file if upload (partially) fails. Does not work that well on versioned buckets/objects.
  3. Copy-paste InMemoryBufferingS3OutputStream and expose abort method
    None of the alternatives are appealing.

Additional context

@maciejwalkowiak
Copy link
Contributor

As I understand exposing this method would make sense for InMemoryBufferingS3OutputStream but not for all the implementations of S3OutputStream. Would you like to submit a PR?

zhemaituk pushed a commit to zhemaituk/spring-cloud-aws that referenced this issue Sep 18, 2024
zhemaituk pushed a commit to zhemaituk/spring-cloud-aws that referenced this issue Sep 18, 2024
…ng external implementations of the class
@zhemaituk
Copy link
Contributor Author

Created #1221 PR.
It seems applicable for the other implementations of S3OutputStream , e.g. if a user needs to cancel the upload mid-way using the stream.
E.g. currently DiskBufferingS3OutputStream#close would always result in a file on S3, even if the user's code fails to write everything what was intended. #abort gives the user ability to cancel if required.

@maciejwalkowiak maciejwalkowiak added this to the 3.3.0 milestone Sep 19, 2024
@maciejwalkowiak maciejwalkowiak added component: s3 S3 integration related issue type: enhancement Smaller enhancement in existing integration labels Sep 19, 2024
@maciejwalkowiak
Copy link
Contributor

Fixed in #1221

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: s3 S3 integration related issue type: enhancement Smaller enhancement in existing integration
Projects
None yet
Development

No branches or pull requests

2 participants