Choosing your DynamoDB partition key is the single most impactful decision you’ll make for performance and cost.
Let’s see it in action. Imagine a simple Users table. We want to store user profiles and retrieve them by their username. A common first thought is to use username as the partition key:
{
"TableName": "Users",
"KeySchema": [
{ "AttributeName": "username", "KeyType": "HASH" } // Partition Key
],
"AttributeDefinitions": [
{ "AttributeName": "username", "AttributeType": "S" }
],
"ProvisionedThroughput": {
"ReadCapacityUnits": 5,
"WriteCapacityUnits": 5
}
}
If we have 100,000 users and each read/write operation is relatively small, we might provision 5 RCU and 5 WCU. This seems fine. But what happens when "admin" or "support" or even a popular celebrity’s username is used? All traffic for that user, no matter how frequent, hits the exact same partition. DynamoDB partitions are physical storage units. If one partition is overloaded with requests, it becomes a bottleneck, and all requests to that table will slow down or fail with throttling errors, even if other partitions are sitting idle.
The problem DynamoDB partition keys solve is distributing data and traffic across multiple physical partitions. When you pick a good partition key, DynamoDB can spread your data and requests across many nodes. A bad partition key, however, funnels all activity to a single node, creating a "hot partition." This wastes provisioned throughput on other nodes and leads to throttling.
Here’s how it works internally: DynamoDB uses a hash function on your partition key value to determine which physical partition a given item belongs to. When you query or put an item, DynamoDB calculates this hash and directs the request to the appropriate partition. If all your partition key values hash to the same or a small set of partitions, you’ve got a hot partition problem.
The goal is uniform distribution. This means that across all possible values of your partition key, the data is spread out evenly, and the read/write requests are also spread out evenly. It’s not just about the number of items; it’s about the frequency of access to those items.
Consider an e-commerce order system. Using orderId as a partition key might seem natural. But if orders are processed chronologically and you’re always inserting new orders, the partition for the most recent orders will get hammered. Instead, a better approach might be to use a composite partition key, or even a randomly generated UUID for the partition key and store orderId as a sort key if you need to retrieve orders by that attribute.
Let’s say we have a Sessions table tracking user activity. If we use userId as the partition key, and some users are much more active than others (e.g., bots or power users), those userIds will create hot partitions. A more robust solution might involve a composite key like userId#sessionId or even a derived attribute that distributes load better, like a timestamp modulo some number, or a hash of the userId.
The levers you control are the choice of attribute(s) for your partition key and, if using composite keys, the order of attributes. You can also use DynamoDB’s auto-scaling to adjust provisioned throughput, but this won’t fix a fundamental hot partition issue – it just throws more resources at the problem, which can become expensive.
The single most surprising true thing about DynamoDB partition keys is that the frequency of access to an item is often more critical than the sheer number of items when determining if a key will cause a hot partition. An attribute that looks unique and evenly distributed by count might still be a terrible partition key if a small subset of its values are accessed orders of magnitude more frequently than others.
If you encounter throttling errors after implementing these recommendations, the next problem you’ll likely face is understanding the nuances of DynamoDB Streams and how they interact with your application logic.