Implementation of "Hello S3" application tutorial was successful using SDK built with WinHTTP but not cURL?

[KreedM]

After having built the SDK using MSVC on my windows machine (both with and without cURL support) in accordance with the instructions, I tried implementing the "Hello S3" application tutorial.

Running the application using the default SDK built with WinHTTP, I was able to successfully obtain a list of S3 buckets I owned. However, I received the following error when I attempted to run the same application that used the SDK built with cURL:

Failed authentication
Failed with error: HTTP response code: -1
Resolved remote host IP address:
Request ID:
Exception name: Xml Parse Error
Error message: XML_ERROR_PARSING_ATTRIBUTE
0 response headers:

When I examined the log file, it seemed like the program encountered a lot of SSL connection timeouts and failed connections. Though it was eventually able to complete a series of TLS handshakes and verify my certificate file, the final http response was unable to be parsed.

I have attached the source code and a snippet of my log file (not really sure how much I should include). Why did authentication seemingly fail and why is there a parsing error? I suspect that for the former, the code confirmed authentication using cURL without knowing where the certificate file was and I'm not sure how to set that. As for the latter, I have no clue.

#include <aws/core/Aws.h>
#include <aws/s3/S3Client.h>
#include <iostream>
#include <aws/core/auth/AWSCredentialsProviderChain.h>
using namespace Aws;
using namespace Aws::Auth;

int main(int argc, char **argv) {
    Aws::SDKOptions options;
    options.loggingOptions.logLevel = Utils::Logging::LogLevel::Trace;

    Aws::InitAPI(options);

    int result = 0;

    {
        auto provider = Aws::MakeShared<DefaultAWSCredentialsProviderChain>("alloc-tag");
        auto creds = provider->GetAWSCredentials();
        if (creds.IsEmpty()) {
            std::cerr << "Failed authentication" << std::endl;
        }

        Aws::Client::ClientConfiguration clientConfig;
        clientConfig.requestTimeoutMs = 10000;
        clientConfig.enableHttpClientTrace = true;
        clientConfig.httpRequestTimeoutMs = 10000;
        clientConfig.connectTimeoutMs = 10000;
        clientConfig.caFile = "C:\\curl\\bin\\curl-ca-bundle.crt";

        Aws::S3::S3Client s3Client(clientConfig);
        auto outcome = s3Client.ListBuckets();
        
        if (!outcome.IsSuccess()) {
            std::cerr << "Failed with error: " << outcome.GetError() << std::endl;
            result = 1;
        } else {
            std::cout << "Found " << outcome.GetResult().GetBuckets().size()
                      << " buckets\n";
            for (auto &bucket: outcome.GetResult().GetBuckets()) {
                std::cout << bucket.GetName() << std::endl;
            }
        }
    }

    Aws::ShutdownAPI(options);
    return result;
}

aws_sdk_2024-08-27-19.log

[bhavya2109sharma]

Hello @KreedM

Thanks for reaching out!
After reviewing the logs, I observed that the request to retrieve the credentials is failing. To resolve this issue, please ensure that your credentials are correctly added to the .aws/credentials file. This can be configured using the AWS CLI.

For detailed instructions, you may refer to the AWS DOCUMENTATION.

Do let me know if anything else is needed.

Thanks
~Bhavya

[KreedM]

Hi,

I understand that I probably need a credentials file, but when I followed the instructions to authenticate with IAM identity center using aws configure sso and aws sso login, no credentials file was created? Am I supposed to independently generate that?

Plus, I'm unsure how not having a credentials file would be an issue for running the code on the cURL compiled sdk but works just fine on the WinHTTP sdk...

Thanks!

[KreedM]

Update: I managed to get the cURL version of the executable to work after creating a credentials file like you suggested. However, I'm confused why that wasn't automatically handled by aws configure sso and how the WinHTTP version ran without the credentials file...

[bhavya2109sharma]

Hello @KreedM

When authenticating using aws configure sso and aws sso login commands, the credentials are not stored in a local credentials file. Instead, AWS SDK retrieve temporary credentials from AWS SSO and store them in memory.

To use the AWS SDK for C++ with AWS SSO, you need to configure the SDK to use the AWS SSO credentials provider, such as Aws::Auth::STSAssumeRoleWebIdentityCredentialsProvider, instead of the DefaultAWSCredentialsProviderChain. The DefaultAWSCredentialsProviderChain checks for credentials in environment variables, the AWS credentials file, ECS container credentials,EC2 Instance Profile credentials,etc. but it does not include AWS SSO as a credential source.

When using aws configure without AWS SSO, it generates the ~/.aws/credentials file, and the DefaultAWSCredentialsProviderChain can find and use the credentials from this file.

The key difference is that when using AWS SSO, the temporary credentials are not stored in a file but are dynamically retrieved and refreshed by the SDK, requiring the use of the appropriate AWS SSO credentials provider, like Aws::Auth::STSAssumeRoleWebIdentityCredentialsProvider.

For more detailed information you can go through this
Let me know if you need anything else.

Thanks
~Bhavya

[KreedM]

Actually, my log file for the application built with WinHTTP suggests the default credentials provider chain is actually able to get credentials using SSO! So does this github pull request. The log file for the same app built with cURL also attempts to get credentials from SSO, but seems to fail because it is unable to provide a certificate. From my limited knowledge of the SDK, I know that the path to the local certificate file must be specified to a client, but it doesn't seem like there's a way to do that for the default credentials provider...

cURL Log.txt

[KreedM]

I have confirmed this is indeed the case! I set the client config's caFile to the location of my certificate in the SDK source code file
aws-sdk-cpp/src/aws-cpp-sdk-core/source/auth/SSOCredentialsProvider.cpp. Then, I used the modified file to rebuild the sdk with cURL support. My application was then finally able to retrieve credentials and display my buckets with the new libraries.

Should an issue be made to fix this?

Modified file (lines 85-88):
SSOCredentialsProvider.cpp.txt

[bhavya2109sharma]

Hello @KreedM,

Thank you for bringing this issue to our attention.
The solution you are providing is indeed not the right one.

After investigating the issue, I can confirm that the root cause lies in the SSOCredentialsProvider.cpp file within the AWS SDK for C++. Specifically, the new configuration is being applied independently without considering any previously set high-level configurations, which is causing the authentication failure.

We acknowledge this issue and are looking into it. However, in the meantime, I would like to suggest a workaround that you can implement.

The workaround involves subclassing the SSOCredentialsProvider and customizing it according to your specific requirements. This will allow you to create your client with the modified credentials provider.

Here's a high-level overview of the steps you can follow:

-Subclass the SSOCredentialsProvider and provide your custom implementation.
-Add a constructor that accepts client configuration as a parameter.
-In your application code, create an instance of your custom SSOCredentialsProvider subclass, passing the required client configuration.
-Use this instance when creating your credentials provider chain.
By following this approach, you can effectively address the authentication issue and continue working with the AWS SDK for C++ until we release an official fix.

Do let me know if anything else is needed.

Thanks,
~Bhavya

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.