You can define reusable configuration snippets in CircleCI to avoid repetition and keep your CI/CD pipelines DRY.
Let’s see how this works with a practical example. Imagine you have several jobs that all need to build a Docker image, run some tests within that image, and then push the image to a registry. Instead of writing the same steps in each job, you can define a reusable command.
Here’s a simplified config.yml demonstrating this:
version: 2.1
# Define reusable commands
commands:
build_and_push_docker_image:
parameters:
image_name:
type: string
default: "my-app"
tag:
type: string
default: "latest"
steps:
- checkout
- setup_remote_docker:
docker_layer_caching: true
- run:
name: "Build Docker Image"
command: docker build -t ${DOCKERHUB_USERNAME}/${image_name}:${tag} .
- run:
name: "Push Docker Image"
command: docker push ${DOCKERHUB_USERNAME}/${image_name}:${tag}
# Define reusable executors
executors:
docker-image-executor:
docker:
- image: cimg/node:18.17.1
jobs:
build-frontend:
executor:
name: docker-image-executor
steps:
- build_and_push_docker_image:
image_name: "frontend"
tag: "<< pipeline.git.tag >>" # Use a pipeline tag for the image
- run:
name: "Install Frontend Dependencies"
command: npm install
- run:
name: "Run Frontend Tests"
command: npm test
build-backend:
executor:
name: docker-image-executor
steps:
- build_and_push_docker_image:
image_name: "backend"
tag: "<< pipeline.git.tag >>"
- run:
name: "Install Backend Dependencies"
command: pip install -r requirements.txt
- run:
name: "Run Backend Tests"
command: pytest
workflows:
version: 2.1
build_and_deploy:
jobs:
- build-frontend
- build-backend
In this example, build_and_push_docker_image is a command that takes two parameters: image_name and tag. These parameters allow us to customize how the command is used in different jobs. The steps within the command are executed whenever the command is invoked.
Notice how checkout and setup_remote_docker are part of the command. This means any job using this command automatically gets these setup steps.
The docker-image-executor is a reusable executor. It defines the Docker image that the job will run in. This is useful if multiple jobs need to run in the same environment.
Now, let’s break down the mental model.
The core problem CircleCI configuration aims to solve is managing complex build and deployment processes across multiple projects or different parts of a single project. Without reusability, you end up with identical or near-identical blocks of configuration scattered everywhere, making updates a nightmare and increasing the chance of errors.
CircleCI’s configuration language (YAML) allows you to define three main types of reusable elements: executors, commands, and jobs.
-
Executors are the environments where your jobs run. You can define an executor once, specifying the Docker image, resource class, or machine type, and then reuse it across multiple jobs. This ensures consistency in your execution environments. In our example,
docker-image-executorspecifiescimg/node:18.17.1as the base image. -
Commands are sequences of steps that can be reused. They are like functions in programming. You define a command with a name and a list of steps. You can also define parameters for commands, allowing them to be customized when called. Our
build_and_push_docker_imagecommand encapsulates the common logic for Docker image building and pushing. It takesimage_nameandtagas parameters, making it flexible. -
Jobs are the fundamental units of work in CircleCI. A job has an executor and a list of steps. You can also use commands within a job’s steps.
The workflows section orchestrates these jobs. It defines which jobs run, in what order, and under what conditions.
When a job uses a command, CircleCI essentially "inlines" the steps defined in that command into the job’s execution flow. If the command has parameters, those parameters are substituted. For example, when build-frontend calls build_and_push_docker_image, CircleCI executes the checkout, setup_remote_docker, docker build, and docker push steps, substituting frontend for image_name and << pipeline.git.tag >> for tag.
The << pipeline.git.tag >> syntax is a CircleCI dynamic configuration value. It allows you to inject information about the current pipeline run, such as Git tags, branch names, or commit SHAs, directly into your configuration. This is incredibly powerful for dynamic versioning and deployment strategies.
The most surprising thing about CircleCI’s reusable configuration is how deeply you can nest and parameterize these elements. You can have commands that call other commands, and you can pass parameters down through these layers. This allows for the creation of highly abstract and composable configuration patterns, enabling you to build sophisticated CI/CD systems from small, independent, and testable building blocks. It’s not just about avoiding copy-pasting; it’s about creating a DSL (Domain Specific Language) for your build and deployment processes.
The next logical step after mastering reusable commands and executors is to explore conditional execution within workflows and jobs.