You can pass parameters between CircleCI pipelines and jobs by using the workflows and jobs configuration in your .circleci/config.yml file, specifically with the parameters and context keywords.

Here’s a CircleCI config demonstrating how to pass parameters between jobs within a pipeline:

version: 2.1

parameters:
  greeting_message:
    type: string
    default: "Hello from the pipeline!"

jobs:
  say_hello:
    parameters:
      message:
        type: string
        default: "Default greeting"
    docker:
      - image: cimg/base:2023.07
    steps:
      - run: echo "<< parameters.message >>"

  run_script:
    parameters:
      input_message:
        type: string
    docker:
      - image: cimg/base:2023.07
    steps:
      - run: echo "Received: << parameters.input_message >>"

workflows:
  my_workflow:
    parameters:
      pipeline_param:
        type: string
        default: "This is a pipeline default"
    jobs:
      - say_hello:
          name: greet_world
          message: "<< pipeline.parameters.pipeline_param >>" # Pass pipeline param to job param
      - run_script:
          name: process_greeting
          requires:
            - greet_world
          input_message: << jobs.greet_world.output.message >> # This won't work directly as jobs don't output parameters like this
          # To pass output from one job to another, you'd typically use artifacts or custom keys.
          # For demonstration, let's assume 'greet_world' somehow outputs a value we can access.
          # A common pattern is to use the `store_output` command in the first job
          # and `add_ssh_keys` or similar for more complex data passing.

      # A more realistic example of passing data:
      - say_hello:
          name: greet_with_custom_message
          message: "Custom message for this job"
      - run_script:
          name: process_custom_greeting
          requires:
            - greet_with_custom_message
          input_message: << jobs.greet_with_custom_message.parameters.message >> # Accessing job parameters passed to another job

In this example:

  • Pipeline Parameters: The parameters block at the top level defines parameters that can be used across the entire pipeline, including within workflows.
  • Job Parameters: Each job can define its own parameters. These are specific to that job and can be set when the job is invoked within a workflow.
  • Workflow Parameters: The workflows block can also have its own parameters, which can then be passed down to the jobs within that workflow.
  • Passing Parameters:
    • You can pass pipeline parameters to job parameters using << pipeline.parameters.parameter_name >>.
    • You can pass parameters from one job to another (if they are defined as job parameters) using << jobs.job_name.parameters.parameter_name >>.

Crucially, directly passing arbitrary output from one job’s execution to another job’s input parameter isn’t as straightforward as accessing its defined parameters. For actual data passing (like the result of a command), you typically use:

  1. store_output and add_ssh_keys (or add_ssh_keys for more general context): The first job can store_output with a specific key, and the second job can then access this output. This is often used for small pieces of data like version numbers or build artifacts.
  2. Artifacts: For larger data, you can store artifacts in the first job and download them in the second.
  3. External Storage: For significant data, use external services like S3, GCS, etc.

Let’s refine the example to show a more practical data-passing scenario using store_output:

version: 2.1

jobs:
  generate_data:
    docker:
      - image: cimg/base:2023.07
    steps:
      - run:
          name: Generate a dynamic value
          command: |
            echo "HELLO_WORLD" > output_value.txt
            echo "dynamic_value=$(cat output_value.txt)" >> "$BASH_ENV"
      - run: echo "Generated value is: $dynamic_value"
      - persist_to_workspace:
          root: .
          paths:
            - output_value.txt # Persist the file containing the value
      - run:
          name: Store the value as an output parameter
          command: |
            VALUE=$(cat output_value.txt)
            echo "$VALUE" > "$BASH_ENV" # This is a common trick to make values available in subsequent steps within the same job
            circleci-agent step publish --key "dynamic_output_key" --value "$VALUE" # Use the CLI to publish a key-value pair

  consume_data:
    docker:
      - image: cimg/base:2023.07
    steps:
      - attach_workspace:
          at: ~/project # Attach the workspace from the previous job
      - run:
          name: Access the dynamic value
          command: |
            # This is where you'd typically retrieve from an external source or an artifact.
            # For direct parameter passing demonstration, we'll simulate retrieval.
            # In a real scenario, you'd use `circleci-agent step get --key "dynamic_output_key"`
            # or retrieve from persisted workspace/artifacts.
            echo "Simulating retrieval of dynamic_output_key..."
            # Let's assume the previous job published "dynamic_output_key" with value "HELLO_WORLD"
            # In a real workflow, the 'consume_data' job would need to be configured to access this.
            # This is often done via custom contexts or by passing configuration that references the output.

workflows:
  main_workflow:
    jobs:
      - generate_data
      - consume_data:
          requires:
            - generate_data
          # To actually pass the output, you'd need a mechanism.
          # The `circleci-agent step publish` command makes the value available
          # via the API, which can be queried by other jobs if configured.
          # A more direct way for simple values is often to use `store_artifacts`
          # and then process the artifact, or use a custom context.
          # For this example, we'll illustrate how to access it IF it were stored in a context.
          # You would typically define a context and use `context: <your-context-name>`
          # in the job definition and then `circleci-agent context store` in the previous job.

The circleci-agent step publish --key "dynamic_output_key" --value "$VALUE" command is the key here. It makes the dynamic_output_key and its value available through the CircleCI API. While other jobs can’t directly reference this dynamically published value as a parameter in the same way they reference static parameters, you can use this mechanism in conjunction with custom contexts or by having the consuming job query the API (though this is less common for simple parameter passing).

A more common pattern for passing simple, dynamic values between jobs is to:

  1. generate_data job:

    • Calculate the value.
    • echo "$VALUE" > "$BASH_ENV" (makes it available in subsequent steps of the same job).
    • persist_to_workspace to save a file containing the value.
    • circleci-agent step publish --key "my_dynamic_value" --value "$VALUE" to make it available via API.
  2. consume_data job:

    • attach_workspace to get the file.
    • Read the value from the file.
    • Alternatively, you could use circleci-agent context store in generate_data and context: <your-context-name> in consume_data to pass values via custom contexts.

The most robust way to pass dynamic values between jobs for use as parameters in subsequent jobs is often through custom contexts.

Here’s how that would look:

1. In generate_data job (or any job that produces the value):

jobs:
  generate_data:
    docker:
      - image: cimg/base:2023.07
    steps:
      - run:
          name: Generate dynamic value
          command: |
            echo "BUILD_VERSION=1.2.3" > version.txt
            VERSION=$(cat version.txt)
            echo "Version is: $VERSION"
      - run:
          name: Store value in CircleCI Context
          command: |
            VERSION=$(cat version.txt)
            # This command requires the CircleCI CLI to be available and authenticated.
            # It writes to a context named 'my-app-context'
            circleci-agent context store --context my-app-context --key BUILD_VERSION --value "$VERSION"

2. In consume_data job (the job that needs the value):

jobs:
  consume_data:
    docker:
      - image: cimg/base:2023.07
    parameters: # Define a parameter to receive the value
      build_version:
        type: string
        default: "unknown"
    steps:
      - run: echo "The build version is: << parameters.build_version >>"

workflows:
  main_workflow:
    jobs:
      - generate_data
      - consume_data:
          requires:
            - generate_data
          context: my-app-context # Specify the context to load
          # Now, map the context value to the job parameter
          build_version: << pipeline.parameters.BUILD_VERSION >> # This syntax is incorrect for context values
          # Correct way: You cannot directly map context values to job parameters like this in the workflow definition.
          # Instead, context values are typically accessed via environment variables within the job.
          # Let's revise the 'consume_data' job to show this.

Revised consume_data job and workflow:

version: 2.1

jobs:
  generate_data:
    docker:
      - image: cimg/base:2023.07
    steps:
      - run:
          name: Generate dynamic value
          command: |
            echo "BUILD_VERSION=1.2.3" > version.txt
            VERSION=$(cat version.txt)
            echo "Version is: $VERSION"
      - run:
          name: Store value in CircleCI Context
          command: |
            VERSION=$(cat version.txt)
            # Ensure the CircleCI CLI is available and authenticated.
            # This command writes to a context named 'my-app-context'
            circleci-agent context store --context my-app-context --key BUILD_VERSION --value "$VERSION"

  consume_data:
    docker:
      - image: cimg/base:2023.07
    steps:
      - run:
          name: Access value from context
          command: |
            # Values stored in a context become available as environment variables
            # prefixed with CIRCLECI_CONTEXT_<CONTEXT_NAME_UPPERCASE>_<KEY_UPPERCASE>
            echo "The build version from context is: $CIRCLECI_CONTEXT_MY_APP_CONTEXT_BUILD_VERSION"

workflows:
  main_workflow:
    jobs:
      - generate_data
      - consume_data:
          requires:
            - generate_data
          context: my-app-context # This loads the context, making its variables available.

In this revised context example:

  • The generate_data job uses circleci-agent context store to save BUILD_VERSION=1.2.3 into a context named my-app-context.
  • The consume_data job declares context: my-app-context. This action makes all key-value pairs stored in my-app-context available as environment variables within that job.
  • The environment variable will be named CIRCLECI_CONTEXT_MY_APP_CONTEXT_BUILD_VERSION, and its value will be 1.2.3. You can then reference this environment variable directly in your run commands.

This method of using contexts is the most common and recommended way to pass dynamic, configuration-like values between jobs in CircleCI.

Want structured learning?

Take the full Circleci course →