EC2 instances can’t just talk to the metadata service without proving they are who they say they are, and the old way of doing that was like leaving your front door wide open.
Imagine you’ve got a web server running on EC2, and it needs to know its own public IP address or its instance ID to do its job. It asks the metadata service, which lives at 169.254.169.254. Historically, any process on the instance could just send a request to that IP and get back sensitive information about the instance, including credentials if it had an IAM role attached. This was a huge vulnerability. If an attacker could run code on your instance, they could easily steal those credentials and impersonate your EC2 instance, gaining access to your AWS resources.
To fix this, AWS introduced IMDSv2 (Instance Metadata Service Version 2). It’s a hop-by-hop authentication mechanism that requires a session-oriented, token-based approach. Instead of a one-off request, you first obtain a temporary token, and then use that token to access the metadata. This means any process trying to access metadata has to actively obtain a token for a specific session, making it much harder for an attacker to passively sniff or replay requests.
Here’s how you enforce it. You need to configure your EC2 instances to require IMDSv2. This is done at the instance or AMI level.
Enforcing IMDSv2 on Existing Instances:
-
Check Current Setting: You can check the IMDSv2 setting for an existing instance using the AWS CLI:
aws ec2 describe-instance-attribute --instance-id i-0123456789abcdef0 --attribute metadata-optionsLook for
HttpTokensandHttpPutResponseHopLimit. IfHttpTokensisoptionalordisabled, IMDSv2 is not enforced. -
Modify Instance Attribute: To enforce IMDSv2, you’ll modify the instance’s metadata options. This requires a
PUTrequest to the instance metadata endpoint itself, but it’s a special kind of request that modifies the service’s behavior for that instance.aws ec2 modify-instance-attribute --instance-id i-0123456789abcdef0 --no-disable-api-termination --metadata-options HttpTokens=required,HttpPutResponseHopLimit=1HttpTokens=required: This is the key. It forces all requests to the metadata service to include a valid IMDSv2 token.HttpPutResponseHopLimit=1: This limits the number of network hops the metadata response can take. Setting it to1means the response can only go directly back to the originating instance, preventing it from being forwarded to other systems. This is a crucial security measure.
Enforcing IMDSv2 During Instance Launch (or via AMI):
When launching a new EC2 instance or creating an AMI, you can set the IMDSv2 enforcement.
-
Via AWS CLI Launch:
aws ec2 run-instances \ --image-id ami-0abcdef1234567890 \ --instance-type t3.micro \ --subnet-id subnet-0123456789abcdef0 \ --metadata-options HttpTokens=required,HttpPutResponseHopLimit=1The
metadata-optionsparameter works the same way as inmodify-instance-attribute. -
Via AMI: You can also set this as a default for an AMI. When you create an AMI from an instance, the metadata options are often copied. If you’re building AMIs from scratch, you’d typically bake this into your build process using tools like Packer. The underlying mechanism is the same as modifying an instance.
How to Use IMDSv2 from an Instance:
Once IMDSv2 is required, your applications or scripts running on the instance need to adapt. They can no longer make direct GET requests to 169.254.169.254/latest/meta-data/.
-
Get a Session Token: First, obtain a session-specific token. The TTL (Time To Live) for the token can be set, but a common and secure practice is to use a short TTL, like 6 hours.
TOKEN=$(curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")This command sends a
PUTrequest to the metadata endpoint specifically asking for a token, and the response will be the token itself.21600seconds is 6 hours. -
Access Metadata with the Token: Now, use this token in the
X-aws-ec2-metadata-tokenheader for subsequent requests to retrieve metadata.INSTANCE_ID=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id) echo "My instance ID is: $INSTANCE_ID"This demonstrates fetching the instance ID using the obtained token.
Common Pitfalls and Why They Happen:
- Application Incompatibility: Many older applications or scripts are hardcoded to use the IMDSv1 direct GET method. When you enforce IMDSv2, these applications will fail to retrieve metadata, leading to errors like "401 Unauthorized" or "Failed to retrieve instance metadata."
- Diagnosis: Check application logs for metadata retrieval failures. If you see 401s or timeouts when trying to fetch
http://169.254.169.254/..., it’s likely an IMDSv1-only app. - Fix: Update the application or script to use the IMDSv2 token-based approach described above. This often involves modifying the network client or SDK calls.
- Diagnosis: Check application logs for metadata retrieval failures. If you see 401s or timeouts when trying to fetch
- Orchestration Tools: Tools like
cloud-initor custom user data scripts might be written for IMDSv1.- Diagnosis: Review your
cloud-initlogs (/var/log/cloud-init.log) or user data script output for similar metadata access errors. - Fix: Ensure your
cloud-initconfiguration or user data scripts are updated to handle IMDSv2 token acquisition. Many moderncloud-initversions have built-in support.
- Diagnosis: Review your
- Incorrect
HttpPutResponseHopLimit: Setting the hop limit too low can sometimes interfere with legitimate network configurations, though this is rare. A common mistake is setting it to0or a very high number.- Diagnosis: If you can get a token but still can’t retrieve metadata, and your application is correctly using the token, check the hop limit.
- Fix: Ensure
HttpPutResponseHopLimitis set to1for maximum security and standard operation.
- AWS CLI/SDK Version: Older versions of AWS CLI or SDKs might not have native support for IMDSv2.
- Diagnosis: If you’re running scripts with older AWS CLI/SDKs and they fail to get tokens or metadata, check the version.
- Fix: Update your AWS CLI to the latest version (
pip install --upgrade awscli) and your SDKs to their latest releases.
- Network ACLs/Security Groups: While IMDSv2 is an authentication mechanism, overly restrictive Network ACLs or Security Groups could, in theory, block access to
169.254.169.254. However, this IP is a link-local address and is generally accessible from within the instance.- Diagnosis: This is a less common cause for IMDSv2 enforcement issues, but if you’re seeing complete connection failures to the metadata IP.
- Fix: Ensure your Security Group allows outbound traffic to
169.254.169.254on all ports (or at least UDP/TCP 80). However, this is usually not the culprit for IMDSv2 specific failures.
- Instance Connect/Session Manager: If you use these services to connect to your instance, they might also rely on metadata. Ensure they are compatible with IMDSv2.
- Diagnosis: If you can’t SSH or use Session Manager after enforcing IMDSv2.
- Fix: Ensure your SSM Agent or EC2 Instance Connect agent is up-to-date. Newer versions are IMDSv2 aware.
The next challenge you’ll likely face is managing the lifecycle of these IMDSv2 tokens within your applications, ensuring they are refreshed before they expire.