forked from awspring/spring-cloud-aws
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Automatically populate PropertySource with instance metadata when run…
…ning within an EC2-based environment (awspring#962) --------- Co-authored-by: Maciej Walkowiak <walkowiak.maciej@yahoo.com>
- Loading branch information
1 parent
cf4349a
commit b5df124
Showing
14 changed files
with
788 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
[#spring-cloud-aws-imds] | ||
== Instance Metadata Service Integration | ||
|
||
Spring Cloud AWS applications can use the Instance MetaData Service (IMDS) to acquire EC2 instance metadata when running within an EC2-based compute environment. This metadata can be used for a wide variety of reasons, including detecting the availability zone, public IP address, MAC address, and so on. When available, properties can be referenced using the @Value annotation: | ||
|
||
[source,java] | ||
---- | ||
@Value("placement/availability-zone") String availabilityZone; | ||
@Value("public-ipv4") String publicIPAddress; | ||
@Value("mac") String macAddress; | ||
---- | ||
|
||
A full list of instance metadata tags is available in the AWS reference documentation at link:https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-categories.html[AWS EC2 User Guide - Instance Metadata Categories]. Spring Cloud AWS always retrieves the "latest" categories of metadata and removes the prefix so that "/latest/meta-data/instance-id" is available as "instance-id". The "spring.cloud.aws" prefix is also omitted. | ||
|
||
=== Enabling | ||
|
||
To enable instance metadata, add the spring-cloud-aws-starter-imds starter. | ||
|
||
[source,xml] | ||
---- | ||
<dependency> | ||
<groupId>io.awspring.cloud</groupId> | ||
<artifactId>spring-cloud-aws-starter-imds</artifactId> | ||
</dependency> | ||
---- | ||
|
||
This adds the software.amazon.awssdk/imds dependency to the classpath which is used to query the IMDS. Depending on resources, metadata loading can add a half-second delay to application start time. Loading can be explicitly disabled by setting spring.cloud.aws.imds.enabled propery: | ||
|
||
[source,properties] | ||
---- | ||
spring.cloud.aws.imds.enabled=false | ||
---- | ||
|
||
Instance metadata is generally available on any EC2-based compute environment, which includes EC2, Elastic Beanstalk, Elastic Container Service (ECS), Elastic Kubernetes Service (EKS), etc. It is not available in non-EC2 environments such as Lambda or Fargate. Even within EC2-based compute environments instance metadata may be disabled or may be subject to an internal firewall which prohibits it. Whenever instance metadata is unavailable, including when running on a local environment, the autoconfiguration process silently ignores its absence. | ||
|
||
=== Considerations | ||
|
||
Instance metadata is retrieved on a best effort basis and not all keys are always available. For example, the "ipv6" key would only be present if IPv6 addresses were being used, "public-hostname" would only be available for instances running in public subnets with DNS hostnames enabled. | ||
|
||
Instance metadata is retrieved at application start time and is not updated as the application runs. Both IDMS v1 and v2 are supported. Certain keys / ranges are not retrieved, including "block-device-mapping/\*", "events/\*", "iam/security-credentials/\*", "network/interfaces/\*", "public-keys/\*", "spot/\*" for various reasons including security. For example, Some keys such as "spot/termination-time" are only reliable if polled on an interval; presenting their static values obtained at startup time would be deceptive. If you have such a requirement, consider polling the key yourself using the Ec2MetadataClient from the SDK: | ||
|
||
[source,java] | ||
---- | ||
import software.amazon.awssdk.core.exception.SdkClientException; | ||
import software.amazon.awssdk.imds.Ec2MetadataClient; | ||
import software.amazon.awssdk.imds.Ec2MetadataResponse; | ||
... | ||
@Autowired Ec2MetadataClient client; | ||
client.get("/latest/meta-data/spot/termination-time"); | ||
---- | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
59 changes: 59 additions & 0 deletions
59
...toconfigure/src/main/java/io/awspring/cloud/autoconfigure/imds/ImdsAutoConfiguration.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,59 @@ | ||
/* | ||
* Copyright 2013-2022 the original author or authors. | ||
* | ||
* Licensed under the Apache License, Version 2.0 (the "License"); | ||
* you may not use this file except in compliance with the License. | ||
* You may obtain a copy of the License at | ||
* | ||
* https://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
package io.awspring.cloud.autoconfigure.imds; | ||
|
||
import org.springframework.boot.autoconfigure.AutoConfiguration; | ||
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass; | ||
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; | ||
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; | ||
import org.springframework.context.annotation.Bean; | ||
import org.springframework.core.env.ConfigurableEnvironment; | ||
import software.amazon.awssdk.imds.Ec2MetadataClient; | ||
|
||
/** | ||
* Configuration for managing Instance Meta Data Service metadata. | ||
* | ||
* @author Ken Krueger | ||
* @since 3.1.0 | ||
*/ | ||
|
||
@AutoConfiguration | ||
@ConditionalOnClass(Ec2MetadataClient.class) | ||
@ConditionalOnProperty(name = "spring.cloud.aws.imds.enabled", havingValue = "true", matchIfMissing = true) | ||
public class ImdsAutoConfiguration { | ||
|
||
@Bean | ||
@ConditionalOnMissingBean | ||
public ImdsPropertySource imdsPropertySource(ConfigurableEnvironment env, ImdsUtils imdsUtils) { | ||
ImdsPropertySource propertySource = new ImdsPropertySource("Ec2InstanceMetadata", imdsUtils); | ||
propertySource.init(); | ||
env.getPropertySources().addFirst(propertySource); | ||
return propertySource; | ||
} | ||
|
||
@Bean | ||
@ConditionalOnMissingBean | ||
public ImdsUtils imdsUtils(Ec2MetadataClient ec2MetadataClient) { | ||
return new ImdsUtils(ec2MetadataClient); | ||
} | ||
|
||
@Bean | ||
@ConditionalOnMissingBean | ||
public Ec2MetadataClient ec2MetadataClient() { | ||
return Ec2MetadataClient.create(); | ||
} | ||
|
||
} |
66 changes: 66 additions & 0 deletions
66
...-autoconfigure/src/main/java/io/awspring/cloud/autoconfigure/imds/ImdsPropertySource.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
/* | ||
* Copyright 2013-2022 the original author or authors. | ||
* | ||
* Licensed under the Apache License, Version 2.0 (the "License"); | ||
* you may not use this file except in compliance with the License. | ||
* You may obtain a copy of the License at | ||
* | ||
* https://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
package io.awspring.cloud.autoconfigure.imds; | ||
|
||
import io.awspring.cloud.core.config.AwsPropertySource; | ||
import java.util.LinkedHashMap; | ||
import java.util.Map; | ||
import org.springframework.lang.Nullable; | ||
|
||
/** | ||
* Adds properties from the EC2 Instance MetaData Service (IDMS) when it is available. | ||
* | ||
* @author Ken Krueger | ||
* @since 3.1.0 | ||
*/ | ||
public class ImdsPropertySource extends AwsPropertySource<ImdsPropertySource, ImdsUtils> { | ||
|
||
private final String context; | ||
|
||
private final ImdsUtils imdsUtils; | ||
|
||
private final Map<String, String> properties = new LinkedHashMap<>(); | ||
|
||
public ImdsPropertySource(String context, ImdsUtils imdsUtils) { | ||
super(context, imdsUtils); | ||
this.context = context; | ||
this.imdsUtils = imdsUtils; | ||
} | ||
|
||
@Override | ||
public ImdsPropertySource copy() { | ||
return new ImdsPropertySource(context, source); | ||
} | ||
|
||
@Override | ||
public void init() { | ||
if (!imdsUtils.isRunningOnCloudEnvironment()) | ||
return; | ||
properties.putAll(imdsUtils.getEc2InstanceMetadata()); | ||
} | ||
|
||
@Override | ||
public String[] getPropertyNames() { | ||
return properties.keySet().stream().toArray(String[]::new); | ||
} | ||
|
||
@Override | ||
@Nullable | ||
public Object getProperty(String name) { | ||
return properties.get(name); | ||
} | ||
|
||
} |
134 changes: 134 additions & 0 deletions
134
...cloud-aws-autoconfigure/src/main/java/io/awspring/cloud/autoconfigure/imds/ImdsUtils.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,134 @@ | ||
/* | ||
* Copyright 2013-2022 the original author or authors. | ||
* | ||
* Licensed under the Apache License, Version 2.0 (the "License"); | ||
* you may not use this file except in compliance with the License. | ||
* You may obtain a copy of the License at | ||
* | ||
* https://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
package io.awspring.cloud.autoconfigure.imds; | ||
|
||
import java.util.Arrays; | ||
import java.util.LinkedHashMap; | ||
import java.util.Map; | ||
import org.slf4j.Logger; | ||
import org.slf4j.LoggerFactory; | ||
import software.amazon.awssdk.core.exception.SdkClientException; | ||
import software.amazon.awssdk.imds.Ec2MetadataClient; | ||
import software.amazon.awssdk.imds.Ec2MetadataResponse; | ||
|
||
/** | ||
* Utility object for working with EC2 instance metadata service (IDMS). Determines if the service and its data are | ||
* available and provides utility methods for loading a PropertySource. Instance metadata is generally available - with | ||
* exceptions - when an application is running within EC2, Elastic Beanstalk, ECS, EKS, etc. The presence of instance | ||
* metadata can be used as a general indicator of whether an application is running within the AWS cloud or on a local | ||
* environment, however there are exceptions to the general principal, such as when EC2 instance deliberately disables | ||
* IMDS. Non-EC2 compute environments such as Lambda or Fargate do not provide the IMDS service. | ||
* | ||
* Works with either IMDS v1 or v2. | ||
* | ||
* @author Ken Krueger | ||
* @since 3.1.0 | ||
*/ | ||
public class ImdsUtils { | ||
|
||
private static Logger logger = LoggerFactory.getLogger(ImdsUtils.class); | ||
|
||
private final Ec2MetadataClient client; | ||
|
||
private Boolean isCloudEnvironment; | ||
|
||
private final String prefix = "/latest/meta-data/"; | ||
|
||
private final String[] keys = { "ami-id", "ami-launch-index", "ami-manifest-path", "hostname", "instance-action", | ||
"instance-id", "instance-life-cycle", "instance-type", "local-hostname", "local-ipv4", "mac", "profile", | ||
"public-hostname", "public-ipv4", "reservation-id", "security-groups", "ipv6", "kernel-id", "iam/info", | ||
"product-codes", "ramdisk-id", "reservation-id", "services/domain", "services/partition", "tags/instance", | ||
"autoscaling/target-lifecycle-state", "placement/availability-zone", "placement/availability-zone-id", | ||
"placement/group-name", "placement/host-id", "placement/partition-number", "placement/region" | ||
|
||
}; | ||
|
||
public ImdsUtils(Ec2MetadataClient client) { | ||
this.client = client; | ||
} | ||
|
||
public boolean isRunningOnCloudEnvironment() { | ||
if (isCloudEnvironment == null) { | ||
isCloudEnvironment = false; | ||
try { | ||
Ec2MetadataResponse response = client.get("/latest/meta-data/ami-id"); | ||
isCloudEnvironment = response.asString() != null && response.asString().length() > 0; | ||
} | ||
catch (SdkClientException e) { | ||
if (e.getMessage().contains("retries")) { | ||
// Ignore any exceptions about exceeding retries. | ||
// This is expected when instance metadata is not available. | ||
} | ||
else { | ||
logger.debug("Error occurred when accessing instance metadata.", e); | ||
} | ||
} | ||
catch (Exception e) { | ||
logger.error("Error occurred when accessing instance metadata.", e); | ||
} | ||
finally { | ||
if (isCloudEnvironment) { | ||
logger.info("EC2 Instance MetaData detected, application is running within an EC2 instance."); | ||
} | ||
else { | ||
logger.info( | ||
"EC2 Instance MetaData not detected, application is NOT running within an EC2 instance."); | ||
} | ||
} | ||
} | ||
return isCloudEnvironment; | ||
} | ||
|
||
/** | ||
* Load EC2 Instance Metadata into a simple Map structure, suitable for use populating a PropertySource. | ||
* @return Map of metadata properties. Empty Map if instance metadata is not available. | ||
* | ||
* @see ImdsPropertySource | ||
* @see ImdsAutoConfiguration | ||
*/ | ||
public Map<String, String> getEc2InstanceMetadata() { | ||
Map<String, String> properties = new LinkedHashMap<>(); | ||
if (!isRunningOnCloudEnvironment()) | ||
return properties; | ||
|
||
Arrays.stream(keys).forEach(t -> mapPut(properties, t)); | ||
|
||
return properties; | ||
} | ||
|
||
/** | ||
* Internal utility method for safely loading a candidate key into the given Map. Silently ignores various expected | ||
* cases where keys are not present. | ||
* @param map | ||
* @param key | ||
*/ | ||
private void mapPut(Map<String, String> map, String key) { | ||
try { | ||
Ec2MetadataResponse response = client.get(prefix + key); | ||
if (response != null) { | ||
map.put(key, response.asString()); | ||
} | ||
} | ||
catch (SdkClientException e) { | ||
logger.debug("Unable to read property " + prefix + key + ", exception message: " + e.getMessage()); | ||
} | ||
catch (RuntimeException e) { | ||
logger.debug( | ||
"Exception occurred reading property " + prefix + key + ", exception message: " + e.getMessage()); | ||
} | ||
} | ||
|
||
} |
22 changes: 22 additions & 0 deletions
22
...ud-aws-autoconfigure/src/main/java/io/awspring/cloud/autoconfigure/imds/package-info.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
/* | ||
* Copyright 2013-2022 the original author or authors. | ||
* | ||
* Licensed under the Apache License, Version 2.0 (the "License"); | ||
* you may not use this file except in compliance with the License. | ||
* You may obtain a copy of the License at | ||
* | ||
* https://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
|
||
/** | ||
* Auto-configuration for IMDS (EC2 Instance MetaData Service) integration. | ||
*/ | ||
@org.springframework.lang.NonNullApi | ||
@org.springframework.lang.NonNullFields | ||
package io.awspring.cloud.autoconfigure.imds; |
1 change: 1 addition & 0 deletions
1
...esources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.