Every database you use for write-heavy workloads is probably running an LSM tree under the hood — RocksDB, Cassandra, ScyllaDB, TiKV. The compaction strategy is the most consequential configuration decision you'll make, and the defaults are optimized for a workload that probably isn't yours.
Building tiny-lsm from scratch gave me the clarity to understand why.
What compaction is actually doing
LSM trees solve the write amplification problem of B-trees by buffering writes in memory (the memtable) and flushing sorted immutable files to disk (SSTables). Reads are expensive because you might need to check multiple SSTables for a single key. Compaction merges SSTables to reduce read amplification — but it introduces write amplification (you're rewriting data that hasn't changed).
Every compaction strategy is a different answer to the question: where do you want the amplification to go?
Leveled compaction
RocksDB's default. SSTables are organized into levels (L0, L1, L2, ...) with exponentially increasing size budgets. L0 accepts flushes directly; compaction moves data from L0 → L1 → L2, merging with the target level as it goes.
Read amplification: Low. A key exists in at most one SSTable per level (except L0). Worst case: O(levels) SSTable reads.
Write amplification: High. Moving data from L0 to Lmax rewrites it ~10x per level. For a 5-level tree, that's 10^5 = 100,000x write amplification in the worst case (in practice, much less due to key distribution).
Good for: Read-heavy workloads, point lookups, latency-sensitive queries. Bad for write-heavy workloads where you're paying 10-30x write amplification constantly.
Tiered (universal) compaction
Fewer, larger SSTables. Instead of leveled organization, SSTables are sorted by size and merged when the size ratio between adjacent tiers exceeds a threshold.
Read amplification: Higher. A key might exist in any tier, so worst-case reads are proportional to the number of SSTables.
Write amplification: Lower. You're rewriting data less frequently because merges happen less often.
Good for: Write-heavy workloads where you can tolerate slightly slower reads. Common in time-series databases where you mostly append.
Time-window compaction (TWCS)
The one Cassandra uses for time-series data, and the one I'm implementing in tiny-lsm.
SSTables are bucketed by their write time. Within a time window, tiered compaction applies. Across time windows, nothing is compacted — old data is never rewritten unless explicitly triggered.
Read amplification: Low for recent data, potentially high for old data (many small SSTables per window if windows are granular).
Write amplification: Very low. Old data is written once and never touched again.
Good for: Time-series, event logs, append-only workloads where data is queried by time range and old data is rarely updated.
Why this matters for tiny-lsm
The Mini-LSM course implements leveled compaction, which is correct for a general-purpose engine. My extension goal is TWCS because the workload I care about is time-ordered events — telemetry, audit logs, game move histories.
The key insight I got from building the compaction layer: the merge iterator is the primitive that makes all of these possible. A compaction strategy is just a policy for which SSTables to feed into the merge iterator and when. Once the iterator is correct, the strategy is policy code on top.
The hard part isn't the compaction logic — it's the manifest. Every compaction must be atomic from the manifest's perspective: the old SSTables and the new SSTable must swap atomically, or a crash leaves you with either duplicated or missing data. Getting this right required understanding MDBX-style copy-on-write semantics well enough to implement something equivalent in my simpler storage layer.
Practical takeaway
If you're running RocksDB for a write-heavy workload and you haven't changed the compaction strategy, you're probably paying 10-30x more write amplification than you need to. options.compaction_style = kUniversal is the first thing to try. Profile your write amplification before and after — rocksdb.compaction-write-key-num and rocksdb.bytes-written tell you where the I/O is going.