Send Custom Metrics from EC2 to CloudWatch with the Agent (2026)

The EC2 agent can ingest custom metrics from your applications and send them to CloudWatch, but it doesn’t collect them itself; it’s a passive transporter.

Let’s see it in action. Imagine you have a Python script that tracks how many times a specific function is called.

import time
import json
import os
from datetime import datetime

# Simulate some work and metric generation
def process_request(request_id):
    print(f"Processing request: {request_id}")
    # Simulate some processing time
    time.sleep(0.1)
    print(f"Finished processing request: {request_id}")

    # Generate a custom metric: number of successful requests
    metric_data = {
        "MetricName": "SuccessfulRequests",
        "Namespace": "MyApplication",
        "Dimensions": [
            {"Name": "Environment", "Value": "Production"}
        ],
        "Value": 1,
        "Unit": "Count",
        "Timestamp": datetime.utcnow().isoformat() + "Z"
    }

    # Write to a file that the CloudWatch agent monitors
    metric_file_path = "/opt/my_app_metrics/metrics.json"
    os.makedirs(os.path.dirname(metric_file_path), exist_ok=True)
    with open(metric_file_path, "a") as f:
        f.write(json.dumps(metric_data) + "\n")

# Simulate incoming requests
for i in range(5):
    process_request(f"req-{i+1}")
    time.sleep(0.5)

This script writes a JSON object for each successful request to /opt/my_app_metrics/metrics.json. The CloudWatch agent, configured to watch this file, will pick up these lines and send them as custom metrics to CloudWatch.

The problem this solves is getting application-level insights into CloudWatch without baking CloudWatch API calls directly into every application. The agent acts as a unified collector and forwarder, simplifying management and reducing boilerplate code in your applications.

Here’s how it works internally:

1. Application Generates Metric: Your application, or a script it runs, writes metric data in a specific JSON format to a designated file. The key fields are MetricName, Namespace, Value, and Unit. Dimensions are optional but highly recommended for filtering and grouping.
2. CloudWatch Agent Configuration: You configure the agent (typically via /opt/aws/amazon-cloudwatch-agent/bin/config.json or using the amazon-cloudwatch-agent-ctl command) to monitor this file. You’ll specify the log_group_name (which becomes your metric namespace in this context) and the file_path to your metric file. The agent’s metrics_collected section will have a logs subsection with metrics_collected enabled.
3. Agent Reads and Parses: The agent periodically reads the specified log file. It parses each line as a JSON object, extracting the metric details.
4. API Call to CloudWatch: For each valid metric found, the agent makes a PutMetricData API call to CloudWatch, sending the collected data.

The primary lever you control is the format of the JSON output from your application and the configuration of the agent’s log collection.

This mechanism is also how you can send metrics from log files that aren’t generated by your application, but rather are standard application logs. If you can parse a line from a standard log file (like an Apache access log) and extract a numerical value and associated metadata, you can configure the agent to treat that value as a custom metric. For example, you could parse a log line for a specific error code and send a "LogErrorCount" metric.

The next concept you’ll likely explore is how to aggregate metrics before sending them to CloudWatch, either within your application or by leveraging the agent’s aggregation capabilities if you’re sending system-level metrics.