You can SSH into a CircleCI job to debug failures live.

Here’s a CircleCI job running a simple Python test suite:

version: 2.1

jobs:
  build:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: pip install -r requirements.txt
      - run:
          name: Run tests
          command: pytest

And here’s the requirements.txt:

pytest
requests

And test_example.py:

import requests

def test_example_site_up():
    response = requests.get("https://example.com")
    assert response.status_code == 200

def test_broken_link():
    response = requests.get("https://httpbin.org/status/404")
    assert response.status_code == 200 # This will fail

When you run this, the test_broken_link will fail because example.com returns a 200, but httpbin.org/status/404 returns a 404, and the test asserts it should be 200.

To debug this live, you’ll need to modify the job to include SSH capabilities. This involves adding a debug section to your job configuration.

Here’s the modified job:

version: 2.1

jobs:
  build:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: pip install -r requirements.txt
      - run:
          name: Run tests
          command: pytest
      - run:
          name: Debug SSH
          command: |
            echo "SSH debugging enabled. Job will pause here."
            echo "To connect, run: ssh -p 2222 root@circleci-ssh.circleci.com"
            echo "Your SSH key is: $(cat ~/.ssh/id_rsa.pub)"
            sleep infinity # Keep the job running indefinitely
    debug:
      docker:
        - image: cimg/python:3.9
      steps:
        - checkout
        - run:
            name: Install dependencies
            command: pip install -r requirements.txt
        - run:
            name: Run tests
            command: pytest
        - run:
            name: Debug SSH
            command: |
              echo "SSH debugging enabled. Job will pause here."
              echo "To connect, run: ssh -p 2222 root@circleci-ssh.circleci.com"
              echo "Your SSH key is: $(cat ~/.ssh/id_rsa.pub)"
              sleep infinity # Keep the job running indefinitely

The key addition here is the debug section. This tells CircleCI to provision an SSH-accessible environment for this job. Notice that the debug section essentially mirrors the jobs section, ensuring the same environment and steps are available. The sleep infinity command is crucial; it prevents the job from exiting after the pytest command, allowing you time to connect.

Once you push this change and the job starts, you’ll see a new "Debug" tab appear in the CircleCI UI for that specific job run. Click on it. You’ll find instructions, including a command like ssh -p 2222 root@circleci-ssh.circleci.com. This command is specific to your job run. You’ll also see your public SSH key printed out.

You need to add this public key to your CircleCI user settings. Go to your CircleCI profile, find the "SSH Keys" section, and add the public key you see in the job output.

After adding the key, execute the provided SSH command in your local terminal. You should now be connected to the running container for your CircleCI job. You can navigate the filesystem, inspect files, and even manually run commands like pytest to see the failure in real-time.

The sleep infinity command is a common pattern for debugging. It keeps the container alive and accessible via SSH. When you’re done debugging, you can simply cancel the job in the CircleCI UI, which will terminate the container and end the sleep process.

The debug section in CircleCI is distinct from the main jobs section. While they often contain identical steps, the debug block is specifically for configuring the SSH debugging environment. This separation allows you to enable debugging without altering the core logic of your job for regular runs. It’s a way to "opt-in" to a debug-enabled execution.

When you’re done, remember to remove the debug section from your configuration to avoid unnecessary resource usage and to prevent accidental debugging in production builds. The next step in debugging complex CI/CD issues often involves understanding how to use environment variables effectively to pass configuration or secrets between steps and jobs.

Want structured learning?

Take the full Circleci course →