Cosmos DB’s Gremlin API lets you model your data as a graph, but most people don’t realize it’s actually running a fully managed Apache TinkerPop graph database under the hood.
Let’s see a graph in action. Imagine we’re modeling a social network. We have person vertices and knows edges.
// Vertex: Person
{
"id": "person1",
"label": "person",
"properties": {
"name": "Alice",
"age": 30
}
}
// Vertex: Person
{
"id": "person2",
"label": "person",
"properties": {
"name": "Bob",
"age": 35
}
}
// Edge: Knows
{
"id": "edge1",
"outV": "person1",
"inV": "person2",
"label": "knows",
"properties": {
"since": 2020
}
}
We can insert this data using the Gremlin console or SDKs. Here’s a sample Gremlin query to add a person and then have them know another person:
g.addV('person').property('name', 'Alice').property('age', 30)
g.addV('person').property('name', 'Bob').property('age', 35)
g.V().has('name', 'Alice').addE('knows').to(g.V().has('name', 'Bob')).property('since', 2020)
Now, let’s query this graph. To find everyone Alice knows:
g.V().has('name', 'Alice').out('knows').values('name')
This would return ["Bob"].
To find people who know Alice:
g.V().has('name', 'Alice').in('knows').values('name')
This would also return ["Bob"] in this simple case.
The core problem Cosmos DB’s Gremlin API solves is providing a scalable, globally distributed, managed graph database. Traditional graph databases often require self-management, scaling complexities, and a less robust global distribution story. Cosmos DB handles all of that for you.
Internally, Cosmos DB uses a distributed log-structured merge-tree (LSM-tree) storage engine. When you add vertices and edges, they are written to this distributed storage. The Gremlin execution engine translates your Gremlin traversal steps into efficient queries against this underlying storage. For example, out('knows') isn’t just a simple pointer lookup; it’s an optimized read operation that leverages internal indexing.
The key levers you control are:
- Throughput (RUs): Request Units are how you provision performance. Higher RUs mean more throughput for reads and writes. You can scale this up or down.
- Indexing Policy: Cosmos DB automatically indexes all properties by default. You can customize this to exclude certain properties or specify composite indexes for specific query patterns, which can significantly improve performance for complex traversals.
- Partitioning: Cosmos DB automatically partitions your graph data based on a partition key. For Gremlin, this is typically derived from the vertex
id. Proper partition key selection is crucial for performance and scalability, ensuring your graph data is distributed evenly. - API Choice: While we’re discussing Gremlin, Cosmos DB also offers other APIs (SQL, MongoDB, Cassandra, Table). The Gremlin API is specifically for graph workloads.
When you execute a traversal like g.V().hasLabel('person').has('age', gt(30)), the Gremlin server on Cosmos DB parses this. It identifies the hasLabel and has steps as filters. These filters are translated into efficient reads against the underlying storage, potentially using pre-built indexes for label and age properties. The results are then streamed back. The gt(30) predicate means "greater than 30," and Cosmos DB’s engine efficiently applies this to the age property of the person vertices.
The actual execution of a Gremlin traversal on Cosmos DB involves a distributed query planner. This planner breaks down your traversal into smaller operations that can be executed in parallel across the distributed partitions of your graph. It determines the most efficient way to fetch vertices and edges, join them, and apply filters, all while minimizing network hops and maximizing resource utilization. The g.V().outE().inV() pattern, for instance, is optimized to fetch outgoing edges and then immediately fetch the target vertices, rather than fetching all outgoing edges first and then performing a separate lookup for each destination vertex.
To achieve high performance and low latency for graph traversals, especially those involving multiple hops or complex filtering, you need to understand how Cosmos DB’s distributed nature interacts with your Gremlin queries. For example, if you have a query that frequently traverses edges between vertices that are on different physical partitions, the cost in terms of network overhead and coordination can be significant. This is why ensuring your vertex IDs are well-distributed and that your queries align with partition boundaries, where possible, is important. If your graph data is highly skewed, with a few vertices having an enormous number of connections, you might encounter performance bottlenecks.
The most surprising thing about Gremlin’s graph traversal language is how its seemingly simple steps can be composed into incredibly powerful and expressive queries that are optimized by the underlying distributed database engine. You’re not just writing code; you’re describing a path through a network, and the engine figures out the fastest way to walk it across potentially thousands of machines.
The next logical step after mastering basic traversals is to explore how to optimize complex, multi-hop queries for performance on a distributed graph.