MergeTree settings are surprisingly malleable, and the most impactful tuning often involves reducing the frequency of merges, not increasing it.

Let’s watch a MergeTree table come alive. Imagine we have a table events designed to store time-series data, like website clicks.

CREATE TABLE events (
    event_time DateTime,
    user_id UInt64,
    event_type String,
    page_url String
) ENGINE = MergeTree()
ORDER BY (user_id, event_time)
PARTITION BY toYYYYMM(event_time);

Now, let’s insert some data.

INSERT INTO events VALUES
('2023-10-27 10:00:00', 101, 'click', '/home'),
('2023-10-27 10:01:00', 102, 'view', '/products'),
('2023-10-27 10:00:30', 101, 'click', '/about');

ClickHouse, by default, will start background processes to merge these small data parts. If we query system.parts for our table, we’d see entries for each INSERT.

SELECT name, size_rows, modification_time FROM system.parts WHERE table = 'events' AND active;

The ORDER BY clause (user_id, event_time) defines the sort order within each data part. This is crucial for query performance, as ClickHouse can then use sorted data for efficient filtering and aggregation. The PARTITION BY toYYYYMM(event_time) directive divides the data into monthly chunks, simplifying data management and improving query performance when filtering by month.

The magic of MergeTree is its background merging process. Small data parts generated by inserts are periodically combined into larger ones. This reduces the number of files ClickHouse needs to read for a query and improves compression. The MergeTree engine has several settings that control this behavior, primarily related to how and when merges happen.

Here are the key levers you can pull:

  • index_granularity: This setting defines how many rows are in each "granule" of the primary index. A smaller index_granularity (e.g., 8192) means more index entries per part, potentially speeding up queries that seek to specific rows but can also increase memory usage for the index. The default is 8192.

    -- Example: Setting index_granularity to 4096 for potentially faster seeks
    CREATE TABLE events_tuned (
        event_time DateTime,
        user_id UInt64,
        event_type String,
        page_url String
    ) ENGINE = MergeTree()
    ORDER BY (user_id, event_time)
    PARTITION BY toYYYYMM(event_time)
    SETTINGS index_granularity = 4096;
    

    This works by reducing the number of rows between index marks, allowing ClickHouse to jump more precisely to the data it needs.

  • merge_tree.max_bytes_to_merge_at_max_space_in_pool: This setting limits the total size of data parts that can be merged in a single background task when the merge pool is not constrained by disk space. A larger value (e.g., 100GB) allows larger merges, which are generally more efficient but can consume more I/O. The default is often around 10GB.

    -- Example: Allowing larger merges to improve compression efficiency
    ALTER TABLE events MODIFY SETTING merge_tree.max_bytes_to_merge_at_max_space_in_pool = 100 * 1024 * 1024 * 1024; -- 100 GB
    

    By increasing this, you allow ClickHouse to combine more data at once, leading to fewer, larger parts with better overall compression ratios.

  • merge_tree.max_bytes_to_merge_at_min_space_in_pool: Similar to the above, but this applies when disk space is more constrained. A smaller value here (e.g., 1GB) prevents massive merges that could fill up the disk. The default is often around 1GB.

    -- Example: Being cautious with merges when disk is filling up
    ALTER TABLE events MODIFY SETTING merge_tree.max_bytes_to_merge_at_min_space_in_pool = 1 * 1024 * 1024 * 1024; -- 1 GB
    

    This acts as a safety valve, ensuring that even under disk pressure, merges are kept small enough not to exhaust available space.

  • merge_tree.number_of_free_entries_in_pool_to_lower_max_size_of_merge: This setting influences when ClickHouse starts scaling down the size of merges. If there are fewer free slots in the merge pool than this number (default 50), ClickHouse reduces the maximum size of merges it will perform. Adjusting this can make merges more aggressive or conservative.

    -- Example: Making merges more aggressive by allowing them to start sooner
    ALTER TABLE events MODIFY SETTING merge_tree.number_of_free_entries_in_pool_to_lower_max_size_of_merge = 20;
    

    A lower value means ClickHouse waits for more merge slots to become free before it starts reducing merge sizes, allowing larger merges to persist for longer.

  • background_pool_size: This is a server-wide setting that determines the number of threads dedicated to background merge operations. Increasing this (e.g., to 16) can speed up merges on multi-core machines, but be mindful of I/O contention. The default is often 16.

    -- Example: Dedicating more threads to background merges on a powerful server
    -- This is a server-level setting, typically in config.xml
    <background_pool_size>16</background_pool_size>
    

    More threads mean more concurrent merge operations can be processed, leading to faster consolidation of data parts, assuming sufficient I/O bandwidth.

  • background_schedule_pool_size: This setting controls the number of threads for scheduling background tasks, including merges. A higher value (e.g., 16) can help ensure merges are scheduled promptly, especially with many tables. Default is 16.

    -- Example: Ensuring merges are scheduled quickly even with many tables
    -- This is a server-level setting, typically in config.xml
    <background_schedule_pool_size>16</background_schedule_pool_size>
    

    This ensures that the system has enough capacity to initiate merge operations efficiently, preventing a backlog of pending merges.

The most common mistake is assuming more merges are always better. In reality, for high-insert workloads, too frequent small merges can create more overhead (CPU, I/O) than they save. You want merges to be large enough to be efficient. For read-heavy workloads with infrequent inserts, you might want to ensure merges happen frequently enough to keep part counts low.

The next challenge you’ll face is optimizing the ORDER BY key for your specific query patterns.

Want structured learning?

Take the full Clickhouse course →