Cosmos DB is a globally distributed, multi-model database service that offers a variety of APIs to interact with your data. The Python SDK for Cosmos DB allows you to programmatically manage and query your data from Python applications.
Let’s see Cosmos DB in action with a simple Python script that creates a container, adds an item, and then queries it.
import azure.cosmos.cosmos_client as cosmos_client
import azure.cosmos.exceptions as exceptions
import os
# Replace with your actual Cosmos DB endpoint and key
HOST = os.environ.get('COSMOS_HOST', 'https://your-cosmos-db-account.documents.azure.com:443/')
MASTER_KEY = os.environ.get('COSMOS_KEY', 'your-master-key')
# Initialize the Cosmos client
client = cosmos_client.CosmosClient(HOST, {'masterKey': MASTER_KEY})
# Database and container names
DATABASE_ID = 'MyDatabase'
CONTAINER_ID = 'MyContainer'
try:
# Create database if it doesn't exist
database = client.create_database_if_not_exists(id=DATABASE_ID)
print(f"Database '{DATABASE_ID}' created or already exists.")
# Create container if it doesn't exist
container = database.create_container_if_not_exists(
id=CONTAINER_ID,
partition_key={'path': '/category'},
offer_throughput=400
)
print(f"Container '{CONTAINER_ID}' created or already exists.")
# Add an item to the container
item_body = {
'id': 'item1',
'name': 'My First Item',
'category': 'example',
'description': 'This is a sample item.'
}
container.create_item(body=item_body)
print("Item 'item1' created.")
# Query items from the container
print("Querying items:")
items = list(container.query_items(
query='SELECT * FROM root r WHERE r.category = "example"',
enable_cross_partition_query=False # Set to True if your query spans partitions
))
for item in items:
print(f"- {item}")
except exceptions.CosmosHttpResponseError as e:
print(f"An error occurred: {e.status_code} - {e.reason}")
except Exception as e:
print(f"An unexpected error occurred: {e}")
This script demonstrates the core operations: establishing a connection, creating a database and container (if they don’t exist), inserting a document (item), and performing a query. The partition_key is crucial for performance and scalability in Cosmos DB. The offer_throughput (measured in Request Units per second, or RU/s) dictates the performance tier of your container.
The fundamental problem Cosmos DB solves is providing a globally distributed, highly available, and scalable database that can support various data models. It achieves this through a distributed architecture where data is partitioned across multiple physical servers. When you interact with Cosmos DB, your requests are routed to the appropriate partition based on the partition key of the item you’re accessing. The SDK handles this routing, abstracting away the complexity of the distributed system.
You control Cosmos DB’s behavior through several key configuration points:
- Endpoint and Master Key: These are your credentials for accessing the Cosmos DB account.
- Database and Container IDs: These logically organize your data. A database is a logical grouping of containers, and containers hold your items.
- Partition Key: This is the most critical element for performance. Choosing a good partition key ensures that your data is evenly distributed and that queries are efficient by targeting specific partitions. A bad partition key can lead to "hot partitions" where one partition receives a disproportionate amount of traffic, degrading performance.
- Offer Throughput (RU/s): This determines the provisioned performance for a container. You can scale this up or down based on your application’s needs. For development, you might use autoscale or a lower fixed throughput, while production workloads often require higher, provisioned throughput.
- Indexing Policy: By default, Cosmos DB indexes all properties of your items. You can customize this to exclude paths or change the indexing mode (consistent, lazy, none) to optimize for read or write workloads.
When performing a query, the enable_cross_partition_query flag is important. If your query doesn’t specify the partition key, or if it needs to scan data across multiple partitions, setting this to True will execute a cross-partition query. However, these are generally less performant and consume more Request Units than queries that target a single partition, so it’s best practice to design your queries to leverage the partition key whenever possible.
The underlying mechanism for distributing data and requests is a sophisticated sharding system managed by Cosmos DB. When you write data, the partition key is hashed, and this hash determines which physical partition the data resides in. When you read data, the same hashing process is used to locate the correct partition. The SDK communicates with the Cosmos DB service, which then directs the operation to the appropriate storage nodes.
The concept of Request Units (RUs) is central to understanding Cosmos DB’s pricing and performance. Every database operation, from a simple read to a complex query, consumes RUs. The amount of RUs consumed depends on factors like the size of the data, the type of operation, and the efficiency of your query (e.g., whether it uses the partition key). Provisioning throughput means reserving a certain number of RUs per second for your container. If your application exceeds this limit, you’ll experience throttling.
The Python SDK also supports various consistency levels, allowing you to trade off consistency for performance and availability. The default is Strong, but you can opt for BoundedStaleness, Session, or Eventual consistency depending on your application’s requirements.
Understanding how Cosmos DB partitions data based on your chosen partition key is the next crucial step in optimizing your application’s performance and cost.