Split Tests Across Parallel CircleCI Executors to Cut CI Time (2026)

You can run the same set of tests in parallel on different CircleCI executors, and then use the results of those tests to decide which executor is faster for your specific workload, ultimately reducing your overall CI time.

Let’s see this in action. Imagine you have a large test suite that takes 30 minutes to run on your standard docker executor. You suspect that a machine executor, with its dedicated hardware, might be significantly faster. You can set up a split test to verify this.

Here’s a simplified config.yml demonstrating the concept:

version: 2.1

jobs:
  run_tests:
    parameters:
      executor_type:
        type: string
        default: "docker"
    executor: <<parameters.executor_type>>
    steps:
      - checkout
      - run:
          name: "Run Tests on <<parameters.executor_type>>"
          command: |
            echo "Running tests on executor: <<parameters.executor_type>>"
            # Replace with your actual test command
            ./run_my_tests.sh --executor <<parameters.executor_type>>
            echo "Tests finished on executor: <<parameters.executor_type>>"

  # Define your executors
  docker-executor:
    docker:
      - image: cimg/ruby:3.1
    # ... other docker specific config

  machine-executor:
    machine:
      image: ubuntu-20.04:current
    # ... other machine specific config

workflows:
  version: 2
  split_test_workflow:
    jobs:
      - run_tests:
          name: "Tests on Docker"
          executor_type: "docker-executor"
      - run_tests:
          name: "Tests on Machine"
          executor_type: "machine-executor"

In this setup, the split_test_workflow explicitly triggers the run_tests job twice, each time with a different executor_type parameter. The run_tests job itself is designed to be generic, accepting the executor type and then executing your test suite. Your actual test script (./run_my_tests.sh in this example) would need to be smart enough to record or report the execution time for each run. You might log the start and end times of the test execution to a file, or use a CI-aware tool that can report metrics.

The core problem this solves is that choosing the right CI infrastructure is often a guess. You might assume a more powerful executor is always faster, but that’s not always true. Network overhead, I/O performance, and even the specific dependencies of your tests can interact with different executor types in unexpected ways. By running the exact same test suite on different environments side-by-side, you gather empirical data. This data allows you to make an informed decision about which executor provides the best performance-to-cost ratio for your specific project. You’re not just looking at raw speed; you’re looking at speed for your tests.

Internally, CircleCI orchestrates these jobs. When you trigger the workflow, it schedules two independent run_tests jobs. Each job spins up its designated executor (either a Docker container or a dedicated machine) and executes the steps defined. The key is that these jobs run concurrently, and their outputs are captured independently. You then manually (or with a subsequent automation step) compare the execution times reported by each job.

The exact levers you control are primarily within your config.yml:

• Executor Definitions: docker vs. machine are the main choices, each with different performance characteristics and cost implications. You can also specify different Docker images or machine OS versions.
• Job Parameters: Using parameters like executor_type makes your job definitions reusable and easier to manage for split testing.
• Workflow Orchestration: You decide which jobs run concurrently or sequentially, and how many parallel branches your tests will take.
• Test Script Logic: The intelligence in your test runner script is crucial. It needs to accurately measure and report the duration of the test execution on each executor.

The surprising thing that most people miss is how much the specific dependencies and file I/O patterns of your test suite can influence executor performance. A test suite that heavily relies on disk-intensive operations might perform dramatically differently on a Docker executor (which has layered filesystems) versus a machine executor with a fast SSD. Similarly, tests that involve significant network calls might benefit from the dedicated network interfaces of a machine executor. You might find that a cheaper Docker executor is perfectly adequate, or that the premium cost of a machine executor is easily justified by a 50% reduction in CI time.

Once you’ve identified the fastest executor for your primary test suite, the next logical step is to explore how to shard your tests within that optimal executor for even greater parallelism.