Feature Request: Add S3Template upload ability of InputStream and content length

[apetrelli]

Type: Feature

I notice that, for a simple upload of an InputStream, an entire S3Resource has to be created along with the S3OutputStream. This results in putting all the data or in memory or to disk

It would be nice to have, in S3Operations and S3Template, an upload method that has the following signature:

S3Resource upload(String bucketName, String key, InputStream inputStream, long contentLength)

This should call S3Client.putObject with RequestBody.fromInputStream, that uses the input stream and the content length so not to read the file twice, or dump in memory or on file.

[maciejwalkowiak]

The S3OutputStream implementations we provide don't store complete file in memory or on disk, see InMemoryBufferingS3OutputStream.

I understand the reasoning but we would have to make it very clear that this particular method does not use the S3OutputStream but goes directly to SDK.

[apetrelli]

@maciejwalkowiak another name is perfect for me as long as it works :-)

[apetrelli]

Thinking about it, in fact what I wanted is something that uploads the InputStream without hitting the filesystem or putting everything in memory when the file is huge, so I guess that InMemoryBufferingS3OutputStream is perfect.
I leave this issue open though, because it might be interesting for someone else.

[zhemaituk]

A decent implementation available in this lib: InMemoryBufferingS3OutputStream.
In a Spring app usage is quite clean, just inject S3OutputStreamProvider:

try (S3OutputStream s3Stream = inMemoryBufferingS3StreamProvider.create(bucket, key
                            ObjectMetadata.builder()
                                    .contentType("text/csv")
                                    .build())) {
   ...
}