EventBridge Lambda Destinations are a better way to handle asynchronous failures than a Dead Letter Queue (DLQ) for Lambda functions.
Let’s see EventBridge Lambda Destinations in action.
Imagine a scenario where your Lambda function processes incoming customer orders. It needs to update a database, send an email notification, and then trigger another downstream service. If any of these steps fail asynchronously (meaning the Lambda function itself completed successfully, but a subsequent call it made failed), you need a robust way to handle these failures.
Here’s a simplified Lambda function that uses a Destination for an asynchronous failure:
import json
import boto3
def lambda_handler(event, context):
print(f"Received event: {json.dumps(event)}")
try:
# Simulate a successful database update
print("Updating database...")
# ... database update logic ...
# Simulate a call to a downstream service that might fail
print("Calling downstream service...")
downstream_response = boto3.client('lambda').invoke(
FunctionName='arn:aws:lambda:us-east-1:123456789012:function:my-downstream-function',
InvocationType='Event', # Asynchronous invocation
Payload=json.dumps({"order_id": event['order_id'], "details": "process_payment"})
)
print("Downstream service invoked.")
# Simulate sending an email
print("Sending email notification...")
# ... email sending logic ...
return {
'statusCode': 200,
'body': json.dumps('Order processed successfully!')
}
except Exception as e:
print(f"An error occurred: {e}")
# This exception will be caught by the Lambda runtime and, if configured,
# sent to the onFailure destination.
raise e
Now, let’s configure an EventBridge Lambda Destination for the onFailure event.
First, you’d create a new EventBridge target. This could be an SQS queue, another Lambda function, an SNS topic, or even an EventBridge event bus. For this example, let’s assume we’re sending failed invocations to a dedicated SQS queue named order-processing-failure-queue.
In your Lambda function’s configuration (either via the AWS Console, CLI, or IaC like CloudFormation/Terraform), you would set the onFailure destination.
AWS CLI Example:
aws lambda update-function-event-invoke-config \
--function-name my-order-processing-function \
--destination-on-failure '{"Destination": "arn:aws:sqs:us-east-1:123456789012:order-processing-failure-queue"}' \
--destination-on-success '{"Destination": "arn:aws:sqs:us-east-1:123456789012:order-processing-success-queue"}' # Optional, but good practice
When the my-order-processing-function is invoked asynchronously and it throws an unhandled exception, or if the invocation itself fails (e.g., the downstream Lambda invocation times out or returns an error code that Lambda interprets as a failure), the Lambda service will automatically send a detailed event payload to the configured onFailure destination.
This payload is much richer than what a traditional DLQ might provide. It includes:
requestContext: Information about the invocation, including therequestId,functionArn, andinvokedFunctionArn.responseContext: Details about the function’s execution, such asstatusCode,functionError, andexecutionError.invokedFunctionArn: The ARN of the function that was invoked.payload: The original event payload that was sent to the function.
This detailed context allows you to build a much more intelligent failure handling system. Instead of just having a queue of failed messages, you have structured data that tells you why it failed, what was being processed, and how it was invoked.
The core problem this solves is distinguishing between a Lambda function that completed with an error (which your code handles and perhaps logs) and an asynchronous invocation failure where the Lambda runtime itself couldn’t complete the task as requested. Traditional DLQs often just capture the raw event that was sent, making it harder to correlate with the specific failure. Destinations provide structured metadata about the invocation outcome.
The key levers you control are the Destination ARN for both onFailure and onSuccess, and the MaximumRetryAttempts and MaximumEventAgeInSeconds for asynchronous invocations. These latter two settings determine how many times Lambda will retry an asynchronous invocation before considering it a failure and sending it to the onFailure destination.
What most people don’t realize is that EventBridge Lambda Destinations are a runtime feature of asynchronous invocations. When you invoke a Lambda function with InvocationType='Event', the Lambda service manages the retry logic and the delivery to your chosen destination automatically. You don’t need to implement polling or manual retry mechanisms within your function for these specific asynchronous failures.
The next concept you’ll likely encounter is how to process and de-duplicate messages from your failure destination, especially if you have multiple Lambda functions sending to the same SQS queue.