Your Docker container isn’t shutting down cleanly because the process inside isn’t receiving the termination signals sent by Docker.
Here’s what’s likely happening and how to fix it:
1. Your Application Ignores SIGTERM
- Diagnosis: Run
docker exec <container_id> ps aux. Look for your main application process. Then, rundocker kill -s SIGTERM <container_id>. Immediately rundocker exec <container_id> ps auxagain. If your process is still running and doesn’t show signs of shutting down, it’s ignoringSIGTERM. - Fix: Modify your application’s code to catch the
SIGTERMsignal and initiate a graceful shutdown. For example, in Node.js:
In Python:process.on('SIGTERM', () => { console.log('SIGTERM signal received: closing HTTP server'); server.close(() => { console.log('HTTP server closed'); process.exit(0); // Exit gracefully }); });import signal import sys def graceful_shutdown(signum, frame): print(f"Received signal {signum}, shutting down gracefully...") # Add your cleanup logic here (e.g., close database connections) sys.exit(0) signal.signal(signal.SIGTERM, graceful_shutdown) - Why it works:
SIGTERMis the standard signal Docker sends to tell a process to stop. If the process doesn’t have a handler for it, it will be unceremoniously killed bySIGKILLafter a timeout, which doesn’t allow for cleanup.
2. Your Entrypoint Script Doesn’t Forward Signals
- Diagnosis: If you’re using a shell script as your
ENTRYPOINT(e.g.,docker run --entrypoint /bin/sh myimage /app/start.sh), and your application is a child process of that script, the script might be consuming the signals instead of passing them down. Check yourENTRYPOINTscript for signal handling. - Fix: If your
ENTRYPOINTis a shell script, useexec "$@"at the end to replace the shell process with your application process. This ensures that your application becomes PID 1 and receives signals directly.
If you’re using#!/bin/sh # ... other setup ... exec /app/your_application --arg1 --arg2tiniordumb-initas your entrypoint (which is recommended), ensure they are configured correctly to forward signals. - Why it works: When a shell script starts another process, the shell script itself is PID 1. Signals are often sent to PID 1. If the shell script doesn’t explicitly forward
SIGTERMto its child process, the child process never sees it.execreplaces the shell process with your application, making your application PID 1 and directly responsible for signal handling.
3. Your Application is a Child Process of sh or bash (and Not PID 1)
- Diagnosis: Use
docker exec -it <container_id> ps auxto inspect the process tree. If your application is not the first process (PID 1) and is instead a child of a shell (like/bin/sh -c ...or/bin/bash), it might not be receiving signals. - Fix: Use an init system like
tiniordumb-initas your container’s entrypoint. These are lightweight processes designed to handle signal forwarding and reaping zombie processes.- Install
tini: Add this to your Dockerfile:RUN apt-get update && apt-get install -y tini && rm -rf /var/lib/apt/lists/* ENTRYPOINT ["/usr/bin/tini", "--"] CMD ["/app/your_application"] - Install
dumb-init: Add this to your Dockerfile:RUN apt-get update && apt-get install -y dumb-init && rm -rf /var/lib/apt/lists/* ENTRYPOINT ["/usr/bin/dumb-init", "--"] CMD ["/app/your_application"]
CMDorENTRYPOINTin your Dockerfile points to your actual application. - Install
- Why it works:
tinianddumb-initare specifically designed to be PID 1 and correctly handle signals, forwarding them to the child processes they manage, and also cleaning up any orphaned child processes.
4. Docker Daemon is Sending SIGKILL Too Soon
- Diagnosis: By default, Docker waits 10 seconds after sending
SIGTERMbefore sendingSIGKILL. If your shutdown process takes longer than this, you’ll see unclean shutdowns. Check yourdocker logs <container_id>for messages indicating a long shutdown time before the container stops. - Fix: Increase the container stop timeout using the
STOPSIGNALandstop_timeoutoptions in yourdocker-compose.ymlor the--stop-timeoutflag withdocker stop.docker-compose.yml:services: your_service: image: your_image stop_signal: SIGTERM stop_grace_period: 30s # Increase this valuedocker stopcommand:docker stop --time 30 <container_id>
- Why it works: This explicitly tells Docker to wait longer for your application to shut down gracefully after
SIGTERMis sent, giving your application more time to complete its cleanup tasks before being forcefully killed.
5. Your Application is Forking and Not Handling Signals Correctly After Fork
- Diagnosis: If your application spawns child processes using
fork()and doesn’t properly set up signal handlers in those children, the signals might not reach the intended process. Usepstree -p <container_id>to visualize the process tree and identify complex forking. - Fix: Ensure that signal handlers are set up in all relevant child processes, or redesign your application to avoid complex forking if possible, relying on threads or a more managed process management strategy. If using a shell script entrypoint, ensure
set -mis not used if you intend to forward signals, or useexecto avoid the shell becoming the parent. - Why it works: Signals are delivered to specific process IDs. If a process forks, the parent process might continue to handle signals while the child process needs its own signal handlers or needs to inherit them correctly.
execis often the simplest way to ensure the main application process becomes PID 1 and handles signals directly.
6. The STOPSIGNAL is Not SIGTERM
- Diagnosis: By default,
docker stopsendsSIGTERM. However, if yourDockerfilehasSTOPSIGNALset to something else, or if you’re manually sending a different signal withdocker kill -s <signal>, your application might not be listening for that specific signal. - Fix: Ensure your
DockerfilehasSTOPSIGNAL SIGTERM(or that you are explicitly sendingSIGTERMwithdocker stopordocker kill). If you must use a different signal, update your application to handle that specific signal.# In your Dockerfile STOPSIGNAL SIGTERM - Why it works: This explicitly sets the signal Docker will send when
docker stopis executed.SIGTERMis the conventional signal for graceful shutdown, and most applications are designed to handle it.
After implementing these fixes, the next likely issue you’ll encounter is a different type of resource leak or a more complex dependency deadlock during shutdown, which will require delving into your application’s specific cleanup logic.