How Writes Work: Commit Log, Memtables, SSTables

Cassandra is known for very fast writes. The reason is in what a node does, and does not do, when your INSERT arrives.

The write path

On each replica, a write goes to two places:

          INSERT
            |
            v
  1) commit log  (added to a file on disk)
  2) memtable    (sorted table in memory)
            |
            v
          done!   (no reading, no searching)
  1. Commit log: the write is added to the end of a log file on disk. Adding to the end of a file is one of the fastest things a disk can do. If the node crashes, Cassandra reads this log when it starts again, so the write is not lost.
  2. Memtable: the write is put into a table in memory (RAM), kept sorted by key.

Then the node says "done". Look at what did not happen. The node did not read the old row. It did not look for the row on disk to change it. It did not check if the row exists. Remember, every write is an upsert. It just stores the new values.

SSTables

When a memtable gets full, Cassandra writes all of it to disk in one go, as a new file called an SSTable (Sorted String Table).

SSTables never change after they are written. An update does not edit an old SSTable. The new value goes into the memtable and later into a newer SSTable. So one row can have pieces in several SSTables. When you read the row, Cassandra checks them all and uses the value with the latest timestamp.

A delete works the same way. It writes a tombstone, a marker that says "this data is deleted".

Compaction

Over time, SSTables pile up. In the background, Cassandra merges several SSTables into one new SSTable. This is called compaction. For each row, it keeps only the latest values. It throws away old values and old tombstones, and the space is freed. This is the moment when deleted data really leaves the disk.

 SSTable 1     SSTable 2     SSTable 3
 alice v1      alice v2      bob (tombstone)
     \             |             /
      '------ compaction -------'
                   |
                   v
             new SSTable
             alice v2

Writes go quickly to the end of files, reads pick the latest version, and compaction cleans up later. That is the design behind every query you have run in this course.

In the exercise, choose what a node does before it says a write is done.