The Docker executor is being deprecated, and you need to migrate your CI/CD pipelines to a newer, more robust solution.
The most common and recommended replacement is the Kubernetes executor. It offers better resource isolation, scalability, and integration with modern container orchestration platforms.
Here’s how to approach the migration:
1. Understand the Problem
The Docker executor, while simple, has limitations. It often runs builds directly on the CI/CD runner host, leading to resource contention, security risks, and difficulty in managing build environments consistently. Deprecation means it will eventually be removed, forcing a migration.
2. Why Kubernetes Executor?
The Kubernetes executor spins up each build as a separate pod in a Kubernetes cluster. This provides:
- True Isolation: Each build gets its own isolated environment, preventing interference between builds.
- Scalability: Leverages Kubernetes’ auto-scaling capabilities to handle fluctuating workloads.
- Flexibility: Define complex build environments using standard Kubernetes resource definitions (Deployments, Pods, etc.).
- Modern Standard: Aligns with current industry best practices for containerized applications.
3. Setting Up Kubernetes Executor
First, ensure you have a Kubernetes cluster accessible to your CI/CD system. Then, configure the CI/CD runner to use the Kubernetes executor.
Example Configuration (Conceptual - specific to your CI/CD tool, e.g., GitLab Runner):
In your config.toml file, you’d replace your Docker executor configuration with something like this:
[[runners]]
name = "my-k8s-runner"
url = "https://your-ci-cd-server.com/"
token = "YOUR_RUNNER_TOKEN"
executor = "kubernetes"
[runners.kubernetes]
namespace = "gitlab-runner" # Or your preferred namespace
image = "ubuntu:latest" # Default image for builds
cpu_request = "500m"
memory_request = "512Mi"
cpu_limit = "1000m"
memory_limit = "1Gi"
poll_interval = 5
kube_context = "" # Optional: if not using default context
Explanation:
executor = "kubernetes": This is the key change.namespace: The Kubernetes namespace where build pods will be created.image: The default container image to use if a build job doesn’t specify one.cpu_request,memory_request,cpu_limit,memory_limit: These define the resource requests and limits for each build pod, ensuring predictable resource allocation.
4. Migrating Your .gitlab-ci.yml (or equivalent)
Your pipeline configuration files will need adjustments. The primary change is how you define the Docker image for your jobs.
Before (Docker Executor):
build-job:
stage: build
image: docker:latest
services:
- docker:dind
script:
- docker build -t my-app .
- docker push my-app
After (Kubernetes Executor):
build-job:
stage: build
image: docker:latest # This specifies the image for the build pod itself
services: # If you need a Docker-in-Docker service, it will be a separate pod
- name: docker:dind
alias: docker
script:
- docker build -t my-app . # 'docker' command now refers to the dind service
- docker push my-app
Key Changes:
- The
imagedirective now refers to the image that will be used to launch the build pod. - If you need Docker-in-Docker (dind) for building images, you’ll configure it as a Kubernetes service within the pod. The
alias: dockermakes thedockercommand available inside your build container, pointing to the dind service.
5. Handling Docker-in-Docker (dind)
Running docker:dind as a service within a Kubernetes pod requires careful configuration.
Kubernetes Service Definition (Conceptual):
When the CI/CD runner creates the build pod, it will instruct Kubernetes to also create a docker:dind container as a service. This service container needs to be accessible. The CI/CD tool typically handles the networking between the main build container and the service container.
Important Considerations for dind:
- Privileged Mode:
docker:dindoften requires privileged mode to interact with the Docker daemon. Ensure your Kubernetes cluster and runner configuration allow for this if necessary. - Volumes: You might need to mount volumes for
docker:dindto persist its data or for sharing Docker socket/storage between containers if not using the default setup.
6. Advanced Kubernetes Features
You can leverage more advanced Kubernetes features directly in your CI/CD configuration:
- Custom Resource Definitions (CRDs): Define specific Kubernetes resources (e.g., Deployments, StatefulSets) that your CI/CD pipeline should create or manage.
- Node Selectors/Taints/Tolerations: Guide where your build pods land in your Kubernetes cluster based on node labels or taints.
- Service Accounts: Grant specific permissions to your build pods using Kubernetes Service Accounts.
Example: Using Node Selectors
build-job:
stage: build
image: node:lts
tags:
- kubernetes # Tag to ensure this job runs on a K8s runner
script:
- echo "Building on a specific node pool!"
kubernetes: # This section is specific to how your CI/CD tool exposes K8s options
node_selector:
disktype: ssd
This tells the Kubernetes executor to only schedule the build pod on Kubernetes nodes that have the label disktype: ssd.
7. Testing and Verification
Thoroughly test your migrated pipelines. Run them with various scenarios:
- Simple builds.
- Docker image builds.
- Pipelines with multiple stages and complex dependencies.
- Pipelines that deploy to Kubernetes.
Monitor your Kubernetes cluster during builds to observe pod creation, resource utilization, and potential errors.
8. The Next Hurdle: Resource Quotas and Limits
After migrating, you’ll likely encounter Kubernetes ResourceQuota or LimitRange errors if your build pods are requesting more resources than allowed by the namespace’s policies. You’ll need to adjust these Kubernetes cluster-level configurations to accommodate your build workloads.