Argo Workflows doesn’t just run tasks; it lets you define complex dependencies between them, creating sophisticated pipelines.
Let’s say you have a workflow that first builds a Docker image, then pushes it to a registry, and finally deploys it to Kubernetes. You can’t push the image before it’s built, and you can’t deploy it before it’s pushed. Argo Workflows manages these sequential steps naturally.
Here’s a simplified Argo Workflow YAML that demonstrates this:
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: sequential-pipeline-
spec:
entrypoint: build-and-push
templates:
- name: build-and-push
dag:
tasks:
- name: build-docker
template: build-template
- name: push-docker
template: push-template
dependencies: [build-docker]
- name: deploy-app
template: deploy-template
dependencies: [push-docker]
- name: build-template
container:
image: docker:latest
command: ["sh", "-c"]
args: ["echo 'Building Docker image...' && sleep 5 && echo 'Image built.'"]
- name: push-template
container:
image: docker:latest
command: ["sh", "-c"]
args: ["echo 'Pushing Docker image...' && sleep 5 && echo 'Image pushed.'"]
- name: deploy-template
container:
image: alpine:latest
command: ["sh", "-c"]
args: ["echo 'Deploying application...' && sleep 5 && echo 'Application deployed.'"]
In this example:
entrypoint: build-and-pushspecifies the starting point.- The
build-and-pushtemplate uses adag(Directed Acyclic Graph) to define task relationships. build-dockeris the first task.push-dockerhas adependencies: [build-docker]clause, meaning it will only start afterbuild-dockercompletes successfully.- Similarly,
deploy-appdepends onpush-docker.
When you submit this workflow, Argo Workflows will execute build-docker. Once it finishes, push-docker will start. Only after push-docker is done will deploy-app begin.
The dag structure is key here. It allows you to define not just linear sequences but also parallel execution. For instance, if you had two independent build steps that could run concurrently before a final deployment, you’d simply list them without dependencies on each other, and both would start after the preceding stage (if any) completes.
The power comes from nesting these DAGs or combining them with other workflow constructs like steps for simpler linear sequences or parallel for true concurrency. For instance, you could have a steps template that calls a dag template.
Consider a scenario where you need to run integration tests after deployment. You can add another task to the deploy-app’s dependencies:
# ... inside the build-and-push template's dag:
- name: deploy-app
template: deploy-template
dependencies: [push-docker]
- name: run-tests
template: test-template
dependencies: [deploy-app]
Now, the run-tests task will only start after deploy-app has successfully completed.
The most surprising thing about Argo Workflows’ sequential orchestration is how granularly you can control this flow. You’re not limited to just "task A then task B." You can define complex fan-in and fan-out patterns, where multiple tasks must complete before a single subsequent task can start (fan-in), or where one task can trigger multiple downstream tasks (fan-out). This is all managed by the dependencies field within the dag structure, allowing for incredibly flexible pipeline definitions.
The next concept to explore is how to pass artifacts (like build outputs or configuration files) between these sequentially dependent tasks.