Argo Workflows isn’t just for running jobs; it’s a full-blown distributed task execution engine that can model complex, multi-stage processes as DAGs or steps.
Let’s see it in action. Imagine a simple CI pipeline that checks out code, builds a Docker image, and then pushes it to a registry.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: ci-pipeline-
spec:
entrypoint: build-and-push
templates:
- name: checkout
container:
image: alpine:latest
command: ["sh", "-c"]
args: ["echo 'Checking out code...' && sleep 2"]
- name: build-image
container:
image: docker:latest
command: ["sh", "-c"]
args: ["echo 'Building Docker image...' && sleep 3"]
- name: push-image
container:
image: docker:latest
command: ["sh", "-c"]
args: ["echo 'Pushing image...' && sleep 2"]
- name: build-and-push
dag:
tasks:
- name: checkout-code
template: checkout
- name: build-docker
template: build-image
dependencies: [checkout-code]
- name: push-docker
template: push-image
dependencies: [build-docker]
When this workflow runs, Argo orchestrates these steps. The checkout-code task runs first. Once it completes successfully, build-docker starts, and after that, push-docker executes. This is a Directed Acyclic Graph (DAG) where dependencies dictate execution order.
The problem Argo Workflows solves is the brittle, imperative nature of traditional CI scripts. Instead of a monolithic Jenkinsfile or a shell script that’s hard to debug and scale, Argo Workflows treat each step as an independent, containerized task. This makes your CI logic:
- Observable: Each step is a pod, so you can
kubectl logsinto it. - Reproducible: The exact container image and command are defined in the YAML.
- Parallelizable: If steps don’t depend on each other, they can run concurrently.
- Resilient: If a step fails, you can re-run just that step or the entire workflow.
The core components you’ll interact with are Workflow and WorkflowTemplate. A Workflow is an instance of a workflow run, while a WorkflowTemplate is a reusable definition. You can also use Step templates for linear sequences or DAG templates for more complex dependencies.
The dag structure is where the magic happens for complex pipelines. You define tasks within the dag, and each task can have dependencies on other tasks. This allows you to model branching, parallel execution, and conditional logic. For instance, you could have a task that runs unit tests, and only if it succeeds, proceed to integration tests.
Consider how you’d manage secrets like Docker registry credentials. You wouldn’t hardcode them. Instead, you’d use Kubernetes Secrets and mount them into your workflow pods as environment variables or volumes.
# Inside a task's container spec
env:
- name: DOCKER_USERNAME
valueFrom:
secretKeyRef:
name: my-docker-creds
key: username
- name: DOCKER_PASSWORD
valueFrom:
secretKeyRef:
name: my-docker-creds
key: password
This keeps your sensitive information out of your workflow definitions.
When you define a DAG, each task within it is essentially an independent workflow run, but orchestrated by the parent Workflow. The dependencies field is how you tell Argo the order. If task B depends on task A, B will not start until A has completed successfully. If A fails, B and any subsequent tasks that depend on A will be marked as skipped or failed.
The entrypoint field in the Workflow spec specifies which template to start with. This can be a single template, a steps template (for linear execution), or a dag template. You can also pass parameters to your workflows, making them dynamic.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: parameterized-build-
spec:
entrypoint: build-and-push
arguments:
parameters:
- name: image-tag
value: latest
templates:
- name: build-and-push
dag:
tasks:
- name: build
template: build-image
arguments:
parameters:
- name: tag
value: "{{workflow.parameters.image-tag}}" # Referencing workflow parameters
# ... rest of the DAG
- name: build-image
container:
image: docker:latest
command: ["sh", "-c"]
args: ["echo 'Building Docker image with tag: {{=<% ctx.build_tag %>}}' && sleep 3"] # Referencing task parameters
# This is a bit advanced, but you can pass parameters *into* a template from a task
# by defining them in the task's arguments section. Here, build_tag is passed to the container.
# In a real scenario, you'd likely pass this to the docker build command.
The {{workflow.parameters.image-tag}} syntax is how you access parameters defined at the workflow level. {{=<% ctx.build_tag %>}} is a more advanced context variable, often used when passing parameters into a template from a task.
Argo Workflows offers a powerful way to define and manage complex computational workflows, moving beyond simple script execution to a more robust, observable, and scalable system.
The next logical step is to explore how Argo Workflows can integrate with other Kubernetes-native tools, like Argo CD for GitOps deployments, to create end-to-end CI/CD pipelines.