support AWS creds as env vars (for deployment outside EKS) #235
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Context
I'm deploying Gen3 using
helm
but outside of EKS. As noted in themanifestservice
documentation, 5daffd6 removed support for AWS credentials read from the/var/gen3/config/config.json
file. As further suggested in that doc, a workaround for non-EKS users is to export the credentials to environment variables in the pod container. This PR is implementing the change for helm deployments foreshadowed in the doc hereMotivation
I want to deploy Gen3 outside of EKS context (e.g. for dev work on a local k8s cluster) and therefore can't use IRSA for auth. I want to use the environment variable workaround suggested in the documentation, with a simple helm deployment.
Overview
This PR uses the existing
awsaccesskey
andawssecretkey
configuration points undermanifestservice.manifestserviceG3auto
to set optional environment variablesAWS_ACCESS_KEY_ID
andAWS_SECRET_ACCESS_KEY
which will be used for credentials by calls without auth arguments toboto3.Session()
, as for example inmanifestservice.manifests.__init__.py
.If the key id and secret are not both present, the environment vars are not set.
Discussion
In the interests of a minimal change this PR does not remove AWS credentials written into
config.json
atMANIFEST_SERVICE_CONFIG_PATH
, even though they do not appear to be used any more bymanifestservice
since 5daffd6 removed them frommanifestservice.api.app.config
. But perhaps they should be removed e.g. from here to reduce cruft? Is anyone relying on them being there?Testing
I have tested this change in both the positive and noop cases (set the env vars, or don't, if the
values.yaml
values are not present), and it works for me.New Features
manifestservice
env vars to support non-EKS deploymentBreaking Changes
Bug Fixes
Improvements
Dependency updates
Deployment changes