Claude’s API treats every API call as a fresh start, but your conversation history is what gives Claude context. Sending the entire chat log with every prompt is the most straightforward way to maintain continuity, but it quickly becomes inefficient and expensive as conversations grow.
Let’s see this in action. Imagine a simple back-and-forth:
import anthropic
client = anthropic.Anthropic(
api_key="YOUR_ANTHROPIC_API_KEY",
)
# First turn
response = client.messages.create(
model="claude-3-opus-20240229",
max_tokens=1000,
messages=[
{"role": "user", "content": "What's the capital of France?"}
]
)
print(response.content)
# Second turn, maintaining history
conversation_history = [{"role": "user", "content": "What's the capital of France?"}]
conversation_history.append({"role": "assistant", "content": "The capital of France is Paris."})
response = client.messages.create(
model="claude-3-opus-20240229",
max_tokens=1000,
messages=conversation_history + [{"role": "user", "content": "And what is its population?"}]
)
print(response.content)
The messages array in the API call is where the magic happens. It’s an ordered list of message objects, alternating between user and assistant roles. When you send this list, Claude processes it as the complete dialogue so far.
The core problem is that this list grows with every turn. If a conversation spans 50 turns, you’re sending 100 message objects (50 user, 50 assistant) plus your new prompt. This means more data to transmit, longer processing times for Claude, and ultimately, higher costs because you’re paying for token usage on the entire history.
To manage this efficiently, you need to implement a strategy for summarizing or pruning the history. The most common approach is to keep a rolling window of recent messages and condense older ones.
One effective technique is to periodically use Claude itself to summarize older parts of the conversation. You can take a chunk of the history, send it to Claude with a prompt like "Summarize the key points and decisions made in the following conversation excerpt:", and then replace that chunk with the summary. This drastically reduces the token count while retaining the essential context.
Consider a scenario where you have a long, detailed technical discussion. Instead of sending 100 messages about debugging a specific piece of code, you might use an intermediate Claude call to distill it down to: "User and assistant discussed a NullPointerException in the UserService module. The root cause was identified as an uninitialized UserRepository, and the fix involved adding a null check in the constructor." This summary, just one or two messages, replaces dozens of previous ones.
The levers you control are:
- History Window Size: How many recent turns to keep in full.
- Summarization Frequency: How often you trigger the summarization process (e.g., every 10 turns, or when token count exceeds a threshold).
- Summarization Prompt: The instructions you give Claude to create the summary. This can be tailored to extract specific types of information (e.g., action items, decisions, factual statements).
- Model Choice: Using a cheaper, faster model for summarization tasks can be more cost-effective than using your primary model for everything.
The most surprising thing about managing conversation history is that Claude can be your best tool for managing its own context. By treating the conversation history as data that can be processed and compressed, you can maintain sophisticated, long-running interactions without prohibitive costs or latency.
A subtle but critical detail is how you handle the order of messages when combining full history with summaries. The messages array must always be chronologically ordered, so a summary generated from turns 5-10 must be placed after turn 4 and before turn 11 in the messages array sent to Claude.
The next challenge you’ll face is dealing with the potential for hallucination or loss of nuance during the summarization process.