The Docker executor is being deprecated, and you need to migrate your CI/CD pipelines to a newer, more robust solution.

The most common and recommended replacement is the Kubernetes executor. It offers better resource isolation, scalability, and integration with modern container orchestration platforms.

Here’s how to approach the migration:

1. Understand the Problem

The Docker executor, while simple, has limitations. It often runs builds directly on the CI/CD runner host, leading to resource contention, security risks, and difficulty in managing build environments consistently. Deprecation means it will eventually be removed, forcing a migration.

2. Why Kubernetes Executor?

The Kubernetes executor spins up each build as a separate pod in a Kubernetes cluster. This provides:

  • True Isolation: Each build gets its own isolated environment, preventing interference between builds.
  • Scalability: Leverages Kubernetes’ auto-scaling capabilities to handle fluctuating workloads.
  • Flexibility: Define complex build environments using standard Kubernetes resource definitions (Deployments, Pods, etc.).
  • Modern Standard: Aligns with current industry best practices for containerized applications.

3. Setting Up Kubernetes Executor

First, ensure you have a Kubernetes cluster accessible to your CI/CD system. Then, configure the CI/CD runner to use the Kubernetes executor.

Example Configuration (Conceptual - specific to your CI/CD tool, e.g., GitLab Runner):

In your config.toml file, you’d replace your Docker executor configuration with something like this:

[[runners]]
  name = "my-k8s-runner"
  url = "https://your-ci-cd-server.com/"
  token = "YOUR_RUNNER_TOKEN"
  executor = "kubernetes"
  [runners.kubernetes]
    namespace = "gitlab-runner" # Or your preferred namespace
    image = "ubuntu:latest"     # Default image for builds
    cpu_request = "500m"
    memory_request = "512Mi"
    cpu_limit = "1000m"
    memory_limit = "1Gi"
    poll_interval = 5
    kube_context = "" # Optional: if not using default context

Explanation:

  • executor = "kubernetes": This is the key change.
  • namespace: The Kubernetes namespace where build pods will be created.
  • image: The default container image to use if a build job doesn’t specify one.
  • cpu_request, memory_request, cpu_limit, memory_limit: These define the resource requests and limits for each build pod, ensuring predictable resource allocation.

4. Migrating Your .gitlab-ci.yml (or equivalent)

Your pipeline configuration files will need adjustments. The primary change is how you define the Docker image for your jobs.

Before (Docker Executor):

build-job:
  stage: build
  image: docker:latest
  services:
    - docker:dind
  script:
    - docker build -t my-app .
    - docker push my-app

After (Kubernetes Executor):

build-job:
  stage: build
  image: docker:latest # This specifies the image for the build pod itself
  services: # If you need a Docker-in-Docker service, it will be a separate pod
    - name: docker:dind
      alias: docker
  script:
    - docker build -t my-app . # 'docker' command now refers to the dind service
    - docker push my-app

Key Changes:

  • The image directive now refers to the image that will be used to launch the build pod.
  • If you need Docker-in-Docker (dind) for building images, you’ll configure it as a Kubernetes service within the pod. The alias: docker makes the docker command available inside your build container, pointing to the dind service.

5. Handling Docker-in-Docker (dind)

Running docker:dind as a service within a Kubernetes pod requires careful configuration.

Kubernetes Service Definition (Conceptual):

When the CI/CD runner creates the build pod, it will instruct Kubernetes to also create a docker:dind container as a service. This service container needs to be accessible. The CI/CD tool typically handles the networking between the main build container and the service container.

Important Considerations for dind:

  • Privileged Mode: docker:dind often requires privileged mode to interact with the Docker daemon. Ensure your Kubernetes cluster and runner configuration allow for this if necessary.
  • Volumes: You might need to mount volumes for docker:dind to persist its data or for sharing Docker socket/storage between containers if not using the default setup.

6. Advanced Kubernetes Features

You can leverage more advanced Kubernetes features directly in your CI/CD configuration:

  • Custom Resource Definitions (CRDs): Define specific Kubernetes resources (e.g., Deployments, StatefulSets) that your CI/CD pipeline should create or manage.
  • Node Selectors/Taints/Tolerations: Guide where your build pods land in your Kubernetes cluster based on node labels or taints.
  • Service Accounts: Grant specific permissions to your build pods using Kubernetes Service Accounts.

Example: Using Node Selectors

build-job:
  stage: build
  image: node:lts
  tags:
    - kubernetes # Tag to ensure this job runs on a K8s runner
  script:
    - echo "Building on a specific node pool!"
  kubernetes: # This section is specific to how your CI/CD tool exposes K8s options
    node_selector:
      disktype: ssd

This tells the Kubernetes executor to only schedule the build pod on Kubernetes nodes that have the label disktype: ssd.

7. Testing and Verification

Thoroughly test your migrated pipelines. Run them with various scenarios:

  • Simple builds.
  • Docker image builds.
  • Pipelines with multiple stages and complex dependencies.
  • Pipelines that deploy to Kubernetes.

Monitor your Kubernetes cluster during builds to observe pod creation, resource utilization, and potential errors.

8. The Next Hurdle: Resource Quotas and Limits

After migrating, you’ll likely encounter Kubernetes ResourceQuota or LimitRange errors if your build pods are requesting more resources than allowed by the namespace’s policies. You’ll need to adjust these Kubernetes cluster-level configurations to accommodate your build workloads.

Want structured learning?

Take the full Argo-workflows course →