CircleCI workspaces are the mechanism that lets you pass files between jobs in your pipeline, but they’re not just for passing files; they’re fundamentally about controlling the blast radius of your build artifacts.
Let’s see this in action. Imagine a pipeline where you first build a Docker image, and then in a subsequent job, you want to deploy that image.
version: 2.1
jobs:
build_image:
docker:
- image: cimg/base:2023.09
steps:
- checkout
- run:
name: Create dummy artifact
command: echo "This is my image tag" > image_tag.txt
- persist_to_workspace:
root: .
paths:
- image_tag.txt
deploy_image:
docker:
- image: cimg/base:2023.09
steps:
- attach_workspace:
at: ./workspace # Files from previous job will be in this directory
- run:
name: Use artifact
command: cat ./workspace/image_tag.txt
When build_image runs, it creates a image_tag.txt file. The persist_to_workspace step then takes this file (and any other files specified in paths) and uploads it to CircleCI’s artifact storage associated with this particular build. The root parameter specifies the directory within the workspace where the paths are relative to. In this case, root: . means image_tag.txt is at the top level of the workspace.
The deploy_image job, when it starts, doesn’t automatically have access to the files from build_image. The attach_workspace step is crucial here. It downloads the artifacts from the previous job’s workspace and unpacks them into the directory specified by at: ./workspace. Now, the run command in deploy_image can cat ./workspace/image_tag.txt and successfully display its contents.
This system solves the problem of needing intermediate build outputs without having to push them to external storage like S3 or Docker Hub between every single step. It’s a lightweight, in-pipeline artifact passing system. The checkout step, by default, brings your entire repository into the job’s working directory. persist_to_workspace then selects specific files or directories from that working directory to be saved as artifacts. attach_workspace on a subsequent job retrieves those saved artifacts.
You control what gets passed by being explicit in the paths array of persist_to_workspace. If you don’t persist a file, it’s gone. If you persist a directory, everything inside that directory is uploaded. The root directive is key for organizing your workspace artifacts. If build_image created a dist/ directory with compiled assets, you might use root: dist and paths: [.] to upload only the contents of dist/. When attach_workspace downloads this, the unpacked files would appear directly in the job’s working directory (or wherever at: points), not nested under a dist/ folder.
The most surprising detail is that attach_workspace by default attaches all workspaces from all previous jobs in the same pipeline. If you have multiple jobs that produce workspaces and you only want to attach specific ones, you’ll need to use the workspace filter within attach_workspace to specify which job’s workspace to retrieve. Without this, you might end up with unexpected files from unrelated jobs if you’re not careful about your root and paths configurations.
The next logical step is to understand how to filter which workspaces are attached when multiple jobs might be contributing artifacts.