Argo CD can send notifications about sync status to Slack and email, but it’s not just a simple webhook; it’s a full-fledged event-driven system that can be surprisingly complex to configure correctly.
Let’s see it in action. Imagine a Git repository for your application’s deployment.
# Example Argo CD Application manifest
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: my-app
namespace: argocd
spec:
project: default
source:
repoURL: https://github.com/my-org/my-app-config.git
targetRevision: HEAD
path: overlays/production
destination:
server: https://kubernetes.default.svc
namespace: my-app
syncPolicy:
automated:
prune: true
selfHeal: true
When Argo CD detects a change in this repository and successfully synchronizes the cluster with the desired state, it can trigger a notification.
The core of Argo CD’s notification system is the argocd-notifications controller. This controller watches for events emitted by Argo CD (like ApplicationSucceeded, ApplicationFailed, SyncStatusChanged) and, based on its configuration, sends out messages to various destinations.
Here’s how it works internally:
- Event Generation: Argo CD components (like the application controller) publish events to a shared message bus (typically NATS, but can be configured differently).
- Notification Controller Subscription: The
argocd-notificationscontroller subscribes to these events. - Notification Configuration: You define
Notificationresources that specify which events trigger which actions, and what those actions should be. This includes definingServiceresources for destinations like Slack or email. - Action Execution: When a matching event occurs, the controller looks up the associated
Notificationresource, resolves any templates, and then calls the configuredServiceto send the message.
The main components you’ll be configuring are:
Notificationresources: These define the trigger conditions and the actions to take.Serviceresources: These define how to connect to external notification services (Slack, Email, etc.) including credentials and endpoints.
Here’s a Service definition for Slack:
apiVersion: argoproj.io/v1alpha1
kind: ConfigMap
metadata:
name: argocd-notifications-cm
namespace: argocd # Make sure this is your Argo CD namespace
data:
# This section configures the notification services
# The keys here (slack, email) are arbitrary names you can use in Notification resources
slack: |
api.url: https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX
email: |
# SMTP server details
host: smtp.example.com
port: 587
username: notifications@example.com
password: YOUR_APP_PASSWORD
# TLS configuration
tls: |
insecure: false
serverName: smtp.example.com
And here’s a Notification resource that sends a Slack message when an application sync succeeds:
apiVersion: argoproj.io/v1alpha1
kind: ApplicationNotification
metadata:
name: app-sync-success
namespace: argocd # Match the namespace of your Argo CD installation
spec:
# This is the event that triggers the notification
trigger:
# Event type: application is the source, succeeded is the specific event
type: application
event:
# You can filter by application name, namespace, etc.
object: my-app
# Filter by namespace of the application
namespace: default
# This defines the actions to take when the trigger matches
actions:
- send:
# This refers to the 'slack' service defined in the ConfigMap above
service: slack
# The 'template' defines the message content.
# We're using a predefined template here.
template: app-sync-success-template
The actual message content is defined in ConfigMap resources, often named argocd-notifications-templates-cm. Argo CD uses Go templating for dynamic content.
apiVersion: argoproj.io/v1alpha1
kind: ConfigMap
metadata:
name: argocd-notifications-templates-cm
namespace: argocd
data:
# Define templates here. The keys are the template names used in Notification resources.
templates:
app-sync-success-template: |
{
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "*Application Sync Succeeded*\n*App*: {{.object.metadata.name}}\n*Namespace*: {{.object.metadata.namespace}}\n*Revision*: {{.object.status.sync.revision}}\n*Status*: {{.object.status.sync.status}}"
}
}
]
}
The {{.object.metadata.name}} syntax accesses fields from the Argo CD application object that triggered the event. This allows for highly customized messages.
A less obvious but crucial aspect of the notification system is the argocd-notifications controller itself. If this controller isn’t running or isn’t properly configured, no notifications will be sent, regardless of how well your Service and Notification resources are defined. You can check its status with kubectl get pods -n argocd.
The argocd-notifications-cm ConfigMap is where all your service credentials and endpoints live. If your Slack or email notifications are failing, this is the first place to look for typos in URLs, incorrect API tokens, or wrong SMTP server details. The api.url for Slack is particularly sensitive; a single character wrong will break it.
When defining your Notification resources, pay close attention to the trigger.event.object and trigger.event.namespace fields. These act as filters. If your application is named my-app-prod but your trigger is set to my-app, it won’t fire. The namespace here refers to the Kubernetes namespace where the Argo CD Application resource is defined, not the destination.namespace in the application spec.
The argocd-notifications controller relies on a message queue to receive events. While NATS is the default, if you’ve customized your Argo CD installation to use a different message bus, ensure the argocd-notifications controller is configured to connect to it. Mismatched queue configurations will lead to a silent failure where events are generated but never consumed by the notification controller.
One of the most common pitfalls is misinterpreting the template field in the Notification resource. This field doesn’t directly contain the message text for Slack or email. Instead, it’s a reference to a key within the argocd-notifications-templates-cm ConfigMap. If the template name in your Notification resource doesn’t exactly match a key in the templates section of the ConfigMap, the notification will be sent, but it will likely be empty or contain an error message about the missing template.
The final piece of the puzzle is ensuring your argocd-notifications controller has the necessary permissions. It needs to be able to read Application resources to extract details for templating. If it can’t, your notifications might be sent but lack crucial context like the application name or sync revision.
Once you have notifications working for Slack and email, the next logical step is to explore integrating with other platforms like Microsoft Teams or PagerDuty, each requiring its own Service configuration.