RabbitMQ doesn’t actually persist messages in the way you might expect; it relies on the underlying operating system’s file system for durability.
Let’s see how this works with a simple producer and consumer.
# producer.py
import pika
import time
connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
channel.queue_declare(queue='durable_queue', durable=True) # Key: durable=True
message = "Hello, durable world!"
channel.basic_publish(exchange='',
routing_key='durable_queue',
body=message,
properties=pika.BasicProperties(
delivery_mode=2, # Make message persistent
))
print(f" [x] Sent '{message}'")
connection.close()
# consumer.py
import pika
import time
connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
channel.queue_declare(queue='durable_queue', durable=True)
def callback(ch, method, properties, body):
print(f" [x] Received {body}")
ch.basic_ack(delivery_tag=method.delivery_tag) # Acknowledge the message
channel.basic_consume(queue='durable_queue',
on_message_callback=callback,
auto_ack=False) # Important: we'll ack manually
print(' [*] Waiting for messages. To exit press CTRL+C')
channel.start_consuming()
If you run the producer, then stop the consumer and restart RabbitMQ, the message will still be there when the consumer starts again. The magic happens with durable=True on the queue_declare and delivery_mode=2 on basic_publish.
The core problem RabbitMQ solves here is ensuring that messages aren’t lost if the broker itself crashes or restarts. Without durability, messages are held only in RAM. When the broker goes down, so do those messages. By marking a queue as durable, you’re telling RabbitMQ, "This queue needs to survive a broker restart." When a message is published to a durable queue with a persistent delivery mode, RabbitMQ writes it to disk. If the broker crashes, upon restart, RabbitMQ will load the durable queue definitions from disk and then load any messages that were persisted to disk for that queue.
The durable=True flag on queue_declare ensures the queue definition is persisted. The delivery_mode=2 on basic_publish ensures the message content is persisted. You need both for true durability. If you mark the queue as durable but don’t set delivery_mode=2, messages sent before a restart will be lost. If you send persistent messages to a non-durable queue, the messages themselves will be lost when the queue is recreated after a restart.
Here’s a crucial detail often overlooked: even with durable queues and persistent messages, there’s a small window where a message can be lost. When a message is published with delivery_mode=2, RabbitMQ writes it to its internal logs and then acknowledges receipt to the publisher. However, it doesn’t immediately flush that log to disk. There’s a brief period between writing to the log buffer and flushing to disk. If RabbitMQ crashes during that tiny window, the message might be lost. To guarantee absolute persistence, you need to use publisher confirms.
The next step in ensuring message delivery without loss involves understanding publisher confirms and acknowledgements.