Argo Workflows don’t just run tasks; they allow you to orchestrate complex pipelines where the output of one step becomes the input for another.
Imagine you have a workflow that first fetches some data, then processes it, and finally stores the result. You need a way to pass that fetched data from the first step to the second.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: pass-parameters-
spec:
entrypoint: main
templates:
- name: main
dag:
tasks:
- name: fetch-data
template: fetch-data-template
- name: process-data
template: process-data-template
dependencies: [fetch-data]
arguments:
parameters:
- name: raw-data
value: "{{tasks.fetch-data.outputs.parameters.data}}"
- name: store-result
template: store-result-template
dependencies: [process-data]
arguments:
parameters:
- name: processed-data
value: "{{tasks.process-data.outputs.parameters.result}}"
- name: fetch-data-template
container:
image: alpine:latest
command: ["sh", "-c"]
args:
- |
echo "Fetching raw data..."
echo "{\"key\": \"value\", \"count\": 123}" > /tmp/data.json
echo "Data fetched."
outputs:
parameters:
- name: data
valueFrom:
path: /tmp/data.json
- name: process-data-template
inputs:
parameters:
- name: raw-data
container:
image: alpine:latest
command: ["sh", "-c"]
args:
- |
echo "Processing data: {{inputs.parameters.raw-data}}"
# In a real scenario, you'd parse and transform this JSON
PROCESSED_DATA="Processed: {{inputs.parameters.raw-data}}"
echo "$PROCESSED_DATA" > /tmp/processed.txt
echo "Data processed."
outputs:
parameters:
- name: result
valueFrom:
path: /tmp/processed.txt
- name: store-result-template
inputs:
parameters:
- name: processed-data
container:
image: alpine:latest
command: ["sh", "-c"]
args:
- |
echo "Storing result: {{inputs.parameters.processed-data}}"
echo "Result stored."
Here’s how it works:
-
Define Outputs: In the
fetch-data-template, we useoutputs.parametersto declare that this step will produce a parameter nameddata. ThevalueFrom.pathtells Argo to capture the content of/tmp/data.jsonas the value for this parameter. -
Pass as Arguments: In the
maintemplate’s DAG, theprocess-datatask depends onfetch-data. To pass the output, we definearguments.parametersforprocess-data. Thevaluefield uses Argo’s{{tasks.<task-name>.outputs.parameters.<parameter-name>}}templating syntax to reference the output from thefetch-datatask. -
Receive as Inputs: The
process-data-templatedeclares aninputs.parametersnamedraw-data. This name must match thenamein thearguments.parametersof the calling task. Inside the container, you can reference this input using{{inputs.parameters.raw-data}}. -
Chain Further: This pattern repeats for passing the
processed-datafromprocess-datatostore-result.
The core mechanism is Argo’s templating language, which allows you to reference outputs from previous steps using {{tasks.<task-name>.outputs.parameters.<parameter-name>}}. This output can then be assigned to an argument of a downstream task, which the downstream template receives as an input.
What most people miss is that you can pass not just simple strings but also entire files or JSON blobs as parameters, which are then accessible via paths within the downstream container. For example, if /tmp/data.json in fetch-data-template was a large JSON object, {{tasks.fetch-data.outputs.parameters.data}} would contain the entire JSON string, which you could then parse in process-data-template using tools like jq within your container.
The next step is understanding how to manage secrets and configuration values as parameters.