S3 MRAP Endpoint authentication issues

[pebarna]

Hi there,

When pointing to a S3 MRAP endpoint using the SDK v2 we're getting:
Cannot find the software.amazon.awssdk.authcrt.signer.AwsCrtS3V4aSigner class. To invoke a request that requires a SigV4a signer, such as region independent signing, the 'auth-crt' core module must be on the class path.

The MRAP endpoint works fine connecting through the CLI and the SDK client can connect to both buckets behind the MRAP endpoint.

Is there any client config example available?

[debora-ito]

@pebarna can you share a repro code?
And which version of the SDK are you using?

[pebarna]

Hi @debora-ito, thanks for your reply.
We're using v2.19.18.
We're using the SDK in the same way we've been doing for a single region bucket, but without the region.

S3Client client = S3Client.builder() .credentialsProvider(WebIdentityTokenFileCredentialsProvider.create()) .build();

and uploading with:

final PutObjectRequest request = PutObjectRequest.builder().bucket(bucketName).key(folderPath + file.getName()).build()

s3Client.putObject(request, RequestBody.fromString(file.getContents()))

where bucketName is the MRAP arn of the form: arn:aws:s3::xxxxxxxxxxxx:accesspoint/xxxxxxxxxxxxx.mrap

[pebarna]

Any ideas on what we might be doing wrong here @debora-ito ?

[debora-ito]

Based on the error message, you need to add the auth-crt module as a dependency.

What I'm trying to find is if we have any documentation mentioning that this is expected when you use MRAP endpoints, or if this was an oversight.

Do you see the error when auth-crt is added as a dependency?

[debora-ito]

@pebarna confirmed with the team, this is the expected behavior. Adding the auth-crt module should fix the issue.

I'll see to add this info to a documentation page somewhere in the Developer Guide.

Do you have a suggestion of where you'd look for this info in the Dev Guide? Maybe not in the Java SDK Dev Guide but in the S3 MRAP docs? I'm trying to define where this info is most likely to be found.

[pebarna]

Thank you for your reply @debora-ito. Got it worming with the auth-crt library. I feel it could go as a reference under Making requests through a Multi-Region Access Point

[pebarna]

The solution above works but we're hitting another issue. The CRT libraries are not supporting Alpine Linux - see #2868

Error: java.lang.UnsatisfiedLinkError: /tmp/AWSCRT_1614799983975474755libaws-crt-jni.so: Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /tmp/AWSCRT_1614799983975474755libaws-crt-jni.so)

[maheshgohil1989]

@debora-ito Do we have any workaround for this ? Can we use any other client instead of AWS-CRT to support S3 MRAP ?
Please let me know , I am also block due to the same issue. Thanks @debora-ito

Thanks @pebarna for starting this discussions. It really helps.

[debora-ito]

@maheshgohil1989 to be clear, AWS-CRT does support S3 MRAP, you just need to add auth-crt as a dependency. What AWS-CRT doesn't support is Alpine Linux.

To answer your question, MRAP should work fine with the other HTTP Clients (Netty, Apache, UrlConnection)

[maheshgohil1989]

Actually, we are using alpine Linux only. So my question is whether we can use other clients to support S3 MRAP in alpine linux ? We are using roles.

[pebarna]

@maheshgohil1989 Since there's no ETA on #2868, a possible approach is to use AWS Corretto or something else on top of AWS Linux

[maheshgohil1989]

Yes, we migrated from alpine to other linux docker image.
Thanks.

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.

[karrisri]

Hi Team,

When pointing to a S3 MRAP endpoint using the java SDK v2(2.20.153) we're getting:
Exception in thread "main" java.lang.IllegalStateException: Failed to create software.amazon.awssdk.authcrt.signer.AwsCrtS3V4aSigner
at software.amazon.awssdk.auth.signer.SignerLoader.initializeV4aSigner(SignerLoader.java:60)
at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1708)
at software.amazon.awssdk.auth.signer.SignerLoader.get(SignerLoader.java:46)
at software.amazon.awssdk.auth.signer.SignerLoader.getS3SigV4aSigner(SignerLoader.java:42)

The MRAP endpoint works fine connecting through the CLI and the lamada function.
I kept the auth-crt jar of version 2.20.153. but still same issue.

Is there any Config I am missing here?

sample java code I am using as below.
public static void main(String[] args) {

	String bucketName = "arn:aws:s3::XXXXXXXXX:accesspoint/XXXXXXX.mrap";
	String keyName = "test/test_sub1/example.txt";
	S3Client s3 = S3Client.builder() .credentialsProvider(WebIdentityTokenFileCredentialsProvider.create()) .build();
	putS3Object(s3, bucketName, keyName);
}

public static void putS3Object(S3Client s3, String bucketName, String objectKey) {
	PutObjectRequest putObjectRequest = PutObjectRequest.builder().bucket(bucketName).key(objectKey).build();
	s3.putObject(putObjectRequest, RequestBody.fromString("Hello S3!"));
	System.out.println("Uploaded: " + objectKey);
}

[debora-ito]

@karrisri can you share the full stacktrace with the error?

[karrisri]

Hi Debora, Thanks for the response. Sorry for the delay from my side.
Sharing the code that we used and full stack trace error below:
we are using the role arn as --> arn:aws:iam::329602474061:role/br_gtodev_dcrpoctste1_app_enginecontrol_ec2_role, which is an ec2 instance role.

If I use the role WRT account role i.e: arn:aws:iam::329602474061:role/ADFS-BR_DCRPOC_DEV_DeveloperPower_User_Role its working fine.

Please let me know if I am missing out any of the permissions or anything else.

Code:

public static void main(String[] args) {

	String bucketName = "arn:aws:s3::******:accesspoint/******.mrap";
	String objectKey = "test/test_sub1/Test.txt";
	    	
    String roleArn = "arn:aws:iam::329602474061:role/br_gtodev_dcrpoctste1_app_enginecontrol_ec2_role";
    String sessionName = "vastava";

    // Set up STS client
    StsClient stsClient = StsClient.builder().region(Region.EU_WEST_2).build();

    // Assume Role to get temporary credentials
    AssumeRoleRequest assumeRoleRequest = AssumeRoleRequest.builder()
            .roleArn(roleArn)
            .roleSessionName(sessionName)
            .build();

    AssumeRoleResponse assumeRoleResponse = stsClient.assumeRole(assumeRoleRequest);
    Credentials temporaryCredentials = assumeRoleResponse.credentials();

    // Set up S3 client with temporary credentials
    AwsSessionCredentials awsSessionCredentials = AwsSessionCredentials.create(
            temporaryCredentials.accessKeyId(),
            temporaryCredentials.secretAccessKey(),
            temporaryCredentials.sessionToken()
    );
    
    S3Client s3Client = S3Client.builder()
            .region(Region.EU_WEST_2) // Replace with your desired region
            .credentialsProvider(StaticCredentialsProvider.create(awsSessionCredentials))
            .build();
    String path = "C:/Users/karrisr/demo/home/dev-nbdcr/inbound/";
    // Upload object to S3 bucket
    PutObjectRequest putObjectRequest = PutObjectRequest.builder().bucket(bucketName).key(objectKey).build();
	s3Client.putObject(putObjectRequest, RequestBody.fromString("Hello S3!"));

    System.out.println("Object uploaded successfully!");
}

Error:

Exception in thread "main" java.lang.IllegalStateException: Failed to create software.amazon.awssdk.authcrt.signer.AwsCrtS3V4aSigner
at software.amazon.awssdk.auth.signer.SignerLoader.initializeV4aSigner(SignerLoader.java:60)
at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1708)
at software.amazon.awssdk.auth.signer.SignerLoader.get(SignerLoader.java:46)
at software.amazon.awssdk.auth.signer.SignerLoader.getS3SigV4aSigner(SignerLoader.java:42)
at software.amazon.awssdk.awscore.util.SignerOverrideUtils.overrideSignerIfNotOverridden(SignerOverrideUtils.java:51)
at software.amazon.awssdk.services.s3.endpoints.internal.S3EndpointAuthSchemeInterceptor.modifyRequest(S3EndpointAuthSchemeInterceptor.java:52)
at software.amazon.awssdk.core.interceptor.ExecutionInterceptorChain.modifyRequest(ExecutionInterceptorChain.java:68)
at software.amazon.awssdk.awscore.internal.AwsExecutionContextBuilder.runInitialInterceptors(AwsExecutionContextBuilder.java:181)
at software.amazon.awssdk.awscore.internal.AwsExecutionContextBuilder.invokeInterceptorsAndCreateExecutionContext(AwsExecutionContextBuilder.java:117)
at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.invokeInterceptorsAndCreateExecutionContext(AwsSyncClientHandler.java:69)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:78)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:179)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:76)
at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:56)
at software.amazon.awssdk.services.s3.DefaultS3Client.putObject(DefaultS3Client.java:8766)
at citynet.scheduler.TestHarnessForMRAP2.main(TestHarnessForMRAP2.java:71)
Caused by: java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at software.amazon.awssdk.auth.signer.SignerLoader.initializeV4aSigner(SignerLoader.java:53)
... 16 more
Caused by: java.lang.NoClassDefFoundError: software/amazon/awssdk/crt/http/HttpRequestBodyStream
at software.amazon.awssdk.authcrt.signer.internal.AwsCrt4aSigningAdapter.(AwsCrt4aSigningAdapter.java:41)
at software.amazon.awssdk.authcrt.signer.internal.DefaultAwsCrtS3V4aSigner.(DefaultAwsCrtS3V4aSigner.java:60)
at software.amazon.awssdk.authcrt.signer.internal.DefaultAwsCrtS3V4aSigner.(DefaultAwsCrtS3V4aSigner.java:42)
at software.amazon.awssdk.authcrt.signer.internal.DefaultAwsCrtS3V4aSigner$BuilderImpl.build(DefaultAwsCrtS3V4aSigner.java:213)
at software.amazon.awssdk.authcrt.signer.internal.DefaultAwsCrtS3V4aSigner.create(DefaultAwsCrtS3V4aSigner.java:64)
at software.amazon.awssdk.authcrt.signer.AwsCrtS3V4aSigner.create(AwsCrtS3V4aSigner.java:51)
... 21 more
Caused by: java.lang.ClassNotFoundException: software.amazon.awssdk.crt.http.HttpRequestBodyStream
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
... 27 more

Please look into the error above and let me know anything to be changed from my side.

Thanks and Regards,
sri vatsava.

[debora-ito]

@karrisri

Caused by: java.lang.ClassNotFoundException: software.amazon.awssdk.crt.http.HttpRequestBodyStream

"ClassNotFoundException" indicates you are using mismatched versions of SDK modules. You need to make sure all SDK modules are in the exact same version. You can check your dependency tree for the modules with the mismatched version - you can run mvn dependency:tree if using maven, or gradle dependencies if using gradle. Look for the software.amazon.awssdk:aws-crt-client module.

This is not an issue with iam roles, it's a Java project building issue.