Book: Designing Data-Intensive Applications by Martin Kleppmann — Chapter 12: The Future of Data Systems.

Chapter 11 closed with batch and stream processing being two answers to the same question. This last chapter takes a step back from any single technique and asks something bigger: now that we’ve seen replication, partitioning, transactions, consensus, batch and stream processing — how do all these pieces actually fit together in a real system, and what do we, as the people building them, owe the people who end up depending on them?

Unbundling the database: small tools instead of one big one

The first big idea ties directly back to something I wrote about twice already. Chapter 10 opened with the Unix philosophy — small tools, each doing one thing, chained together with pipes. Chapter 11 showed change data capture turning every database write into an event that independent systems could subscribe to. This chapter puts the two together into a genuinely appealing idea: instead of one database trying to be good at everything — transactions, full-text search, caching, analytics, all bundled into a single product — you can unbundle it into several small, specialized tools, each excellent at one job, kept in sync by a shared stream of events.

A single “batteries included” database like Postgres has to make one set of tradeoffs and hope they’re good enough for every use case its users throw at it. An unbundled setup instead lets you pick the best specialized tool for each job — Elasticsearch for full-text search, Redis for a fast cache, a dedicated analytics database for reporting — and feed all of them from the same underlying stream of events, so they never drift out of sync with each other or with the source of truth. The tradeoff is real too: you’re now operating several systems instead of one, and you’ve traded a single vendor’s tested defaults for your own responsibility to wire them together correctly.

Correctness doesn’t have to mean waiting for everyone to agree

The second idea revisits something Chapter 9 already showed was expensive: getting every node to agree before proceeding. This chapter makes a subtler point — you don’t always need that level of agreement to be correct, if you design the ends of the system carefully instead of relying entirely on the middle. This is the end-to-end argument: a guarantee is only as good as the point where it’s actually checked, so instead of trying to make every layer of infrastructure in between perfectly reliable, you put the real correctness check at the boundary that actually matters — often right where a request enters or leaves your system.

A concrete version of this: instead of relying purely on the network or the database to prevent a duplicate request (say, a user double-clicking “buy now”), the client can generate a unique request ID up front and the server can simply refuse to process the same ID twice — a small, cheap check at the true edge of the system, rather than an expensive guarantee threaded through every layer underneath it. It’s a genuinely freeing idea after a whole book about how unreliable networks and clocks are: you don’t have to fix every layer, you just have to make sure the one check that actually matters is in the right place.

The part of the chapter that isn’t really about engineering at all

The last section is the one I found most unexpected, because it isn’t about mechanics at all — it’s about responsibility. The book is blunt that the same techniques covered in this whole book — collecting data, deriving insights from it, predicting behavior from it — can be used to build a genuinely useful recommendation system, or to build a surveillance tool, or a system that quietly discriminates against people based on patterns in data that reflect historical unfairness rather than anything about a specific individual. A credit-scoring or hiring algorithm trained on biased historical data doesn’t magically become neutral just because it’s “just following the data” — it can bake in and even amplify exactly the same bias, at a scale and speed a human process never could.

The book’s phrase for this that stuck with me: data, once collected, doesn’t just sit there neutrally — it becomes a liability, something that can be subpoenaed, breached, repurposed for things the people it describes never agreed to. The practical takeaway isn’t “don’t build data systems,” it’s “the engineer who decides what gets logged, retained, and fed into a model is making a real decision with real consequences for real people, whether or not they think of it that way.”

Closing the loop on the whole book

Looking back across everything from reliability and scalability through this final chapter, the shape of the whole book finally makes sense as one connected argument rather than twelve separate topics: data models and storage engines decide how a single machine holds data honestly; replication and partitioning spread that honesty across many machines; transactions, distributed-systems trouble, and consensus keep that honesty intact when things fail; and batch and stream processing turn that honest data into something useful. This chapter’s real closing point is that getting all of that technically right is necessary, but it was never sufficient on its own — the last question always has to be what you’re actually building it for, and who has to live with the answer.

Book: Designing Data-Intensive Applications by Martin Kleppmann — Chapter 11: Stream Processing.

Chapter 10 ended on a cliffhanger of sorts: batch processing is great when you’re happy to wait for a full day’s (or year’s) data to pile up before computing anything from it. This chapter is about the other half of that pair — what happens when you genuinely can’t wait, because the whole point is reacting the instant something happens: flagging a fraudulent card swipe before it clears, updating a live dashboard, refreshing a search index the moment new content is published.

An event is just a fact that a record used to be — but now it never stops arriving

The mental shift the book asks for is small but important: a stream is just an unbounded, never-ending sequence of events, where an event is a small, immutable record of something that happened, tagged with when it happened. The unbounded part is the whole difference from batch processing — a batch job reads a file that has a definite end; a stream processor reads something that, by design, is never “done.”

Where do these events actually come from? The book’s answer connects directly back to two things I’d already run into: message queues and databases. A message broker like Kafka is built specifically to carry a continuous flow of events from producers to consumers, durably and in order — but the more interesting source, and the one that genuinely surprised me, is the database itself, through something called change data capture.

Change data capture: turning “the database changed” into an event

Change data capture (CDC) takes every insert, update, and delete happening inside a database and turns each one into an event on a stream, in the exact order they happened. Instead of every other part of the system polling the database or querying it directly to notice changes, they simply subscribe to this stream of “here’s what changed” events.

What clicked for me here is how naturally this solves a problem I’d actually run into without having a name for it: keeping a search index, a cache, and a database all in sync with each other, without wiring every single one of them to talk directly to every other one. With CDC, the database is the single source of truth, its changes flow out as one ordered stream, and anything that needs to stay in sync — a search index at one company, a fraud-detection system at another, a cache at a third — just subscribes independently, reading at its own pace, without ever needing to know the others exist. This is exactly the role Kafka plays at companies like LinkedIn (where it originated) — one durable, replayable log that many independent systems can read from concurrently.

Windows: how do you “add up” something that never ends?

A batch job can compute “total sales for the day” by reading the whole day’s data and summing it, because the input has a clear boundary. A stream never has a clear boundary, so the book introduces windows — an artificial boundary you draw yourself, like “the last 5 minutes” or “this specific hour” — to make aggregate questions (“how many logins in the last 5 minutes?”) answerable at all over an endless stream.

The genuinely tricky part, which I hadn’t appreciated before, is that events don’t always arrive in the order they actually happened. A phone can lose signal and send a batch of delayed events all at once; a network hiccup can make one event arrive seconds after events that happened after it. So a stream processor has to make a real decision: does it wait a little while for stragglers before finalizing a window’s answer (more correct, but slower to produce a result), or does it finalize quickly and risk being wrong when a late event shows up afterward? There’s no universally correct answer — it’s a real tradeoff between latency and completeness that every stream processing system has to pick a stance on.

Exactly-once: the promise that’s harder than it sounds

The last idea that stuck with me connects straight back to Chapter 8’s honest inventory of distributed-systems trouble: what happens if a stream processor crashes partway through handling an event, and has to restart? If it simply reprocesses everything it’s unsure about, some events might get counted twice. The guarantee everyone actually wants is exactly-once semantics — each event affects the final result exactly once, no matter how many times the underlying machinery has to retry after a crash. Real systems (Kafka’s own transactional features, Flink’s checkpointing) achieve this not by making failures impossible, but by making the effect of retried work idempotent — safe to apply more than once without changing the outcome — which is a much more achievable goal than pretending crashes won’t happen.

Looking at chapters 10 and 11 together, the real lesson is that batch and stream processing aren’t competitors — they’re the same underlying question (“how do I compute derived data from a big pile of events?”) answered under two different constraints: how much you’re willing to wait, and whether you can define a clean beginning and end to your input at all.

Book: Designing Data-Intensive Applications by Martin Kleppmann — Chapter 10: Batch Processing.

Everything from replication through consensus was about keeping data safe and correct across multiple machines. This chapter starts a new part of the book, and the question changes: given a huge pile of data that’s already sitting there safely, how do you actually compute something useful from it — a search index, a set of recommendations, a report — without waiting forever? The book calls this kind of computed output derived data, and batch processing is the first of two ways it covers for producing it.

The Unix philosophy, scaled up

The chapter’s opening example is almost embarrassingly simple, and that’s exactly the point: counting how often each word appears in a log file using nothing but classic Unix command-line tools — cat to read the file, sort to group identical words next to each other, uniq -c to count how many times each one repeats. Each of these tools does exactly one small thing, and they’re chained together with pipes, where the output of one becomes the input of the next.

What makes this example matter for the rest of the chapter is the design principle behind it: every one of these tools reads from a plain stream of text and writes to a plain stream of text, with no knowledge of what tool comes before or after it in the chain. Because the interface is so simple and uniform, you can rearrange, swap, or insert new tools into the pipeline freely. This turns out to be the exact same idea that lets modern batch processing systems combine dozens of small operations into one large computation without every piece needing to understand every other piece.

MapReduce: the same idea, spread across a thousand machines

A single Unix pipeline runs on one machine. MapReduce — the framework Google introduced and popularized, and which Hadoop later made available outside Google — takes that same “read input, transform it, group it, write output” pattern and spreads it across a whole cluster of machines, so it can chew through datasets far too large for any single machine to hold.

The map step takes the input, split across many machines, and transforms each individual record independently — in the word-count example, turning each line of text into a series of (word, 1) pairs. Then comes the shuffle: all the pairs with the same word get routed to the same machine, so that machine sees every occurrence of that one word together. Finally the reduce step takes each group and combines it into a final answer — adding up all the 1s for a given word to get its total count. Input and output at every stage lives on a distributed filesystem (Hadoop’s version is called HDFS), which is what lets the whole thing scale to datasets spread across hundreds of machines in the first place.

Where MapReduce gets genuinely clever: batch joins

Beyond simple counting, the chapter spends real time on something I hadn’t thought about before: how do you join two enormous datasets — say, a log of user activity and a separate table of user profiles — when both are too big to fit on one machine and you can’t just run a database-style join? The book covers a couple of approaches, and the tradeoff between them was a genuinely useful thing to internalize:

A sort-merge join sorts both datasets by the shared key (say, user ID) and streams through them together, matching up records as it goes — very similar in spirit to how sort groups identical words together in the Unix example, just done at a much larger scale across a cluster. A broadcast hash join instead recognizes that one of the two datasets (the smaller one, like the user-profile table) can fit entirely in memory on every machine, so it gets copied out to all of them, and each machine can then look up matches locally without needing to sort or shuffle the giant dataset at all. Which one is better depends entirely on the actual size difference between the two datasets — exactly the kind of “know your actual workload” lesson that showed up back in the Twitter timeline example too.

Why the diagram matters: not every batch system pays the same disk cost

The one design choice that most shaped how batch processing evolved after the original MapReduce is exactly what the diagram above shows. Classic MapReduce chains jobs together by having each one write its complete output to the distributed filesystem before the next job is even allowed to start reading it — safe and simple, but it means a computation with five sequential steps pays for five full round-trips to disk, even though the earlier Unix pipe example never needed to touch disk at all between commands. Newer dataflow engines — Spark, Flink, Tez — fix exactly this: they keep intermediate results in memory and pipeline data directly from one processing stage to the next wherever possible, only touching disk when they genuinely have to (running out of memory, or writing the final result). This is a big part of why a Spark job doing the same work as an old-style MapReduce job can run dramatically faster: not a smarter algorithm, just far less unnecessary disk I/O between the steps.

Looking ahead, this chapter is really the “first half” of a pair — batch processing works great when you’re happy to wait for all the input to be collected before computing anything (a nightly report, reprocessing a year of logs). The next chapter tackles the opposite case: producing derived data continuously, as new events arrive one at a time, instead of waiting for a full batch to accumulate.

Book: Designing Data-Intensive Applications by Martin Kleppmann — Chapter 9: Consistency and Consensus.

This is the chapter that replication kept gesturing toward every time it needed to safely pick a new leader, and that the previous chapter set up by explaining exactly how unreliable the network and clocks really are. Given all that unreliability, this chapter asks the big question directly: how do you get a group of machines to agree on one single thing — one value, one leader, one order of events — when any of them might be slow, might have crashed, or might simply be unreachable for a while?

Linearizability: the illusion of “just one copy”

The chapter opens with a specific, very strong consistency guarantee called linearizability. The plain-language version: even though your data might actually be spread across several replicated copies, the system behaves as if there were only ever one single copy, and every operation happens at one specific, real instant in time, visible to everyone immediately. Once you write something, literally everyone who asks afterward sees that new value — no stale reads from a lagging replica, no confusing “it’s there for me but not for you” moment.

This sounds like exactly what everyone would obviously want, all the time — so the natural question is why it isn’t just the universal default. The honest answer: linearizability is expensive, because guaranteeing it typically means every read and write has to coordinate with enough of the other replicas to be sure nothing has changed elsewhere first, which adds real latency, especially across long distances (a request that has to round-trip to a data center on another continent before it can complete is never going to be instant). Systems that need to survive a network partition without pausing everything have to give up linearizability to keep working at all — this tradeoff even has a name, the CAP theorem, though the book is careful to point out CAP is narrower and more specific than the internet’s popularized version of it.

Ordering events without a trustworthy clock

Since the previous chapter already established that physical clocks can’t be trusted to put events in the correct order, this chapter introduces a different tool: instead of asking “what time did this happen,” you ask “what happened before what.” This is captured by something called a logical clock — instead of real time, you track cause-and-effect relationships directly (if event B was created in response to event A, B is recorded as happening “after” A, regardless of what either machine’s physical clock said).

The strongest, most useful version of this ordering is total order broadcast: a guarantee that every node in the system delivers the exact same sequence of messages, in the exact same order, no exceptions. This sounds abstract until you realize it’s secretly the same problem as replication itself — if every replica applies the exact same writes in the exact same order, they’ll all end up in the same final state. Total order broadcast and single-leader replication are, in a very real sense, describing the same underlying problem from two different angles.

Consensus: agreeing on one thing despite failures

This is the chapter’s actual centerpiece, and the definition is precise: consensus means getting a set of nodes to agree on a single value, with a guarantee that once a decision is made, it can never be silently reversed or contradicted later — even if some nodes crash, restart, or temporarily can’t communicate with the rest.

The mechanism nearly every real consensus algorithm relies on is the majority vote: as long as more than half the nodes agree, the decision is considered final and safe, even if the remaining nodes are down or unreachable. This is precisely why consensus systems are almost always deployed with an odd number of nodes (3, 5, 7) — it guarantees there’s always a clear majority possible, and it’s also why a majority-based system can keep working even while some minority of nodes has failed, which is the entire point of building it this way in the first place.

The book names the well-known real implementations of this idea, and I found it genuinely satisfying to connect algorithm names to tools I already recognized: Paxos (the original, notoriously hard to fully understand), Raft (designed specifically to be more understandable, and the algorithm behind etcd, the coordination store at the heart of Kubernetes), and ZAB (the protocol underneath Apache ZooKeeper, itself referenced back in the partitioning chapter as the kind of coordination service that tracks which machine currently owns which piece of data). Every one of these is, underneath the specific implementation details, solving the exact same majority-agreement problem.

Why consensus isn’t used for absolutely everything

Given how powerful and safe consensus is, the natural question is why entire databases aren’t built as one giant consensus system for every single write. The honest answer, which echoes the linearizability tradeoff from earlier in the chapter: consensus requires real network round-trips between a majority of nodes for every single decision, which makes it noticeably slower than a system that doesn’t need everyone to agree before proceeding. In practice, consensus gets reserved for the decisions that are genuinely worth paying that cost for — electing a leader, agreeing on cluster membership, committing a distributed transaction — rather than for every single ordinary read and write flowing through a system.

Closing the loop on the whole distributed-systems arc

Looking back across replication, partitioning, the unreliability of networks and clocks, and now this chapter, the shape of the whole argument finally clicks into place for me: distributed systems are hard specifically because the network and clocks can’t be trusted (Chapter 8), which makes safely picking a leader or ordering events genuinely difficult (Chapter 5’s failover problem), which is exactly the problem consensus algorithms exist to solve properly (this chapter) — using nothing more clever than getting a plain majority of honest nodes to agree. It’s a satisfying amount of the book to have now connected into one coherent picture, built from a handful of genuinely simple underlying ideas.

Book: Designing Data-Intensive Applications by Martin Kleppmann — Chapter 8: The Trouble with Distributed Systems.

After transactions, the book takes what felt like a step back — but is actually the most important chapter so far for understanding why everything before it (replication, partitioning, isolation) is as complicated as it is. This chapter doesn’t introduce a new technique. It’s an honest, occasionally unsettling inventory of everything that can quietly go wrong the moment your system spans more than one machine, and it left me with a healthy amount of paranoia about two things I used to take completely for granted: the network, and the clock.

The network lies to you, and it lies by omission

On a single machine, if a function call fails, you usually get a clear error immediately. Across a network, the book’s central, uncomfortable point is that you often get nothing — no error, no confirmation, just silence — and silence is fundamentally ambiguous. Did the request never arrive? Did it arrive and get processed, but the response got lost on the way back? Is the other machine just extremely slow right now? From the sending machine’s point of view, all three of these look absolutely identical: you sent something, and you haven’t heard back.

This is called a network partition, and real ones happen at real companies running real infrastructure — a misconfigured router, a severed undersea cable, a firewall rule change that accidentally blocks the wrong traffic. The book’s blunt conclusion: the only tool you actually have to distinguish “slow” from “dead” is a timeout, and a timeout is fundamentally a guess dressed up as a decision. Set it too short, and you’ll declare healthy, slightly-slow machines dead constantly, triggering unnecessary and disruptive failovers. Set it too long, and a genuinely dead machine keeps everyone else waiting far longer than necessary. There is no timeout value that is simultaneously always correct and always fast — you’re just picking where on that spectrum you’d rather be wrong.

Clocks don’t tell the truth either

This was the section that genuinely surprised me, because I’d always assumed “just check the timestamp” was a safe, boring way to figure out which of two events happened first. It isn’t. Every machine has its own physical clock, and these clocks drift — they run very slightly fast or slow compared to real time, and compared to each other, purely due to hardware imperfections. Systems try to correct for this using NTP (Network Time Protocol), periodically syncing to a reference clock over the network — but that correction itself travels over the same unreliable network described above, and can be delayed, or occasionally jump the clock backward or forward to fix a large drift.

The practical consequence the book hammers on: if two different machines each timestamp an event locally, and you later try to sort those events by comparing timestamps, you can get the order wrong, because the two clocks were never perfectly in sync to begin with. This is why “last write wins,” a strategy that sounds perfectly reasonable at first, is quietly dangerous in a distributed system — “last” according to whose clock? This exact problem is serious enough that Google built a specialized piece of infrastructure, TrueTime, specifically for their Spanner database — instead of pretending clocks are perfectly accurate, TrueTime reports a time as an honest range (“the real time is somewhere between X and Y”), and the system is designed to wait out that uncertainty window rather than gamble on a single, possibly-wrong timestamp.

Process pauses: your code can stop without you knowing

The third source of trouble is one I’d genuinely never considered: even a program that isn’t crashed and isn’t stuck in a network wait can simply stop running for an unpredictable stretch of time, for reasons entirely outside its own control. The book’s main example is garbage collection (GC) pauses — many programming languages periodically pause your entire program to clean up unused memory, and this pause can occasionally last much longer than expected under load. From every other machine’s perspective, a process that’s paused for a long GC cycle looks exactly like a process that’s crashed or a network that’s down — because, again, all they can observe is “I haven’t heard from it in a while.”

The unsettling implication: a distributed system has to be designed to tolerate a node going silent for an unpredictable amount of time and then coming back to life and resuming exactly where it left off, possibly still believing it holds a lock or a leadership role it actually lost while it was paused. This is why real systems use techniques like fencing tokens — a monotonically increasing number handed out with each lease or lock, so that if a paused node wakes up and tries to act on stale authority, the system can recognize its token is outdated and reject the action.

The honest lesson: assume nothing, verify everything

The chapter’s real conclusion isn’t a specific fix — it’s a change in posture. In a single-machine program, you can generally trust that if a function returns, it actually ran, and it ran once. In a distributed system, none of that is safe to assume: messages can be delayed, duplicated, or lost; clocks can’t be fully trusted to order events; and a node that seems dead might just be paused, and might wake back up believing something that’s no longer true. Every protocol covered in the chapters that follow — consensus and consistency especially — exists specifically to build reliable guarantees on top of this genuinely unreliable foundation, not to pretend the foundation is more solid than it is.

Book: Designing Data-Intensive Applications by Martin Kleppmann — Chapter 7: Transactions.

This chapter opens Part II properly, and it’s about a word I’d been using casually for years without ever needing to define it precisely: transaction. Loosely, it means “a group of operations that the database treats as one single unit” — but the whole chapter is really about what that promise is actually worth, because it turns out different databases, and even the same database configured differently, promise wildly different things while all technically calling it a “transaction.”

ACID is four separate promises, not one word

Most people (myself included, until reading this properly) treat “ACID” as one blob meaning “the database is safe.” The book insists on splitting it into four genuinely distinct guarantees, and I found it useful to think about which specific bug each one actually prevents:

• Atomicity — a transaction either fully happens or fully doesn’t; there’s no “half-finished” state visible to anyone. If your code transfers money by subtracting from one account and adding to another, atomicity is what guarantees you never end up in a world where the money left one account but never arrived at the other, even if the server crashes halfway through.
• Consistency — here the book makes an important, easy-to-miss point: this is actually an application-level property, not something the database can enforce all on its own. The database can enforce specific rules you define (like “balances can’t go negative”), but “consistency” itself just means your data always satisfies whatever invariants your application cares about — the database is a tool that helps you keep that promise, not something that magically knows what your invariants are.
• Isolation — concurrent transactions shouldn’t be able to see each other’s half-finished work. This is the guarantee that gets the most attention in the chapter, because it’s the one with the most ways to go subtly, dangerously wrong.
• Durability — once a transaction says “committed,” it stays committed, even if the machine loses power one second later.

Where isolation actually breaks down

The chapter’s real substance is a tour of specific ways concurrent transactions can step on each other, and honestly, several of these were bugs I would not have predicted on my own:

Dirty reads — one transaction sees a change from another transaction that hasn’t been committed yet, and might get rolled back. Imagine seeing your bank balance already reflect a transfer that then gets cancelled a moment later — you briefly saw money that was never actually real.

Lost updates — two transactions read the same value, both decide what to write based on that value, and one write silently overwrites the other, throwing away one of the two updates. The classic version of this, which the book uses and which I found genuinely clarifying:

Two customers both check a product page at nearly the same instant, both see “1 in stock,” both click buy. If the database naively lets each transaction read the stock count, decide to subtract one, and write the new value back — without anything stopping the second write from blindly overwriting the first — you end up having sold the same last unit twice. Nobody’s individual code was wrong; the interleaving was the bug.

Write skew — a subtler cousin of lost updates, where two transactions read overlapping data, each makes a decision that’s individually fine given what it read, but the combination violates a rule neither transaction technically broke on its own. The book’s example: a hospital requires at least one doctor on call at all times; two doctors, seeing “there are currently two of us on call,” each independently decide it’s safe for them to go off call — and now there are zero.

Isolation levels: how much protection you’re actually buying

Because preventing every one of these perfectly, all the time, is expensive, real databases offer a menu of isolation levels, and the honest, slightly uncomfortable truth the book states plainly: many databases don’t default to the strongest one, because the strongest one is slower.

• Read committed — the weakest level covered in depth; it just guarantees you’ll never see another transaction’s uncommitted changes (no dirty reads). It does nothing to stop lost updates or write skew.
• Snapshot isolation — each transaction sees a consistent snapshot of the database as it looked at the moment the transaction started, so it’s immune to dirty reads and most read-related weirdness. It’s a very popular default (Postgres calls its version of this “repeatable read”) because it’s much faster than full serializability, but it still doesn’t fully protect against write skew.
• Serializable — the strongest level: the database guarantees the result is as if every transaction ran one at a time, in some order, with zero overlap, even though under the hood they might actually run concurrently for performance. This is the only level that genuinely closes every one of the bugs above, and it’s also the most expensive, because achieving that guarantee under real concurrent load takes real work — either literally running things one at a time, or detecting and aborting/retrying transactions that would have conflicted.

The lesson that stuck with me: “my database uses transactions” tells you almost nothing on its own. The actual question that matters is “which isolation level, specifically, and did anyone deliberately choose it, or is it just whatever the database defaulted to?”

Why this chapter sets up everything after it

Reading this right after storage engines and right before the more distributed-systems-heavy chapters made the sequencing click for me: this chapter is entirely about correctness on a single machine (or a single database, at least) under concurrent access. The much harder version of this same problem — transactions that span multiple machines, where a network can fail in the middle — is exactly what makes the next couple of chapters (the trouble with distributed systems and consistency and consensus) so much harder than this one. You can’t really appreciate why distributed transactions are painful until you’ve seen how much careful engineering it already takes to get transactions right on one honest, reliable machine.

Book: Designing Data-Intensive Applications by Martin Kleppmann — Chapter 6: Partitioning.

Chapter 5 was about replication — keeping copies of the same data on multiple machines so the system survives one of them dying. This chapter is about something that sounds similar but is actually a completely separate problem: what do you do when your data is simply too big, or gets too much traffic, for any single machine to handle at all, even a perfectly healthy one? The answer is partitioning (also called sharding): splitting your data into smaller pieces and spreading those pieces across many machines.

It’s worth sitting with why these are different problems before going further, because I originally assumed “replication” and “partitioning” were basically the same idea. They’re not. Replication answers “what if this machine dies?” Partitioning answers “what if this machine simply can’t fit or serve all the data by itself, dead or alive?” In a real system you almost always need both at once: split the data into partitions to spread the load, and replicate each partition so any one machine holding a piece of it can still die without losing data. Chapter 5 was about the replication half. This chapter is about the splitting half.

The core decision: how do you decide what goes where?

Say you have a huge table of user data and ten machines to spread it across. The obvious first idea is “put user A’s data on machine 1, user B’s on machine 2,” and so on. But how do you decide, systematically, which machine any given piece of data belongs on? DDIA covers two main strategies, and the tradeoff between them is the whole chapter in miniature.

Partitioning by key range is like splitting a printed encyclopedia into volumes: Volume 1 covers entries A through D, Volume 2 covers E through H, and so on. Any entry has an obvious, predictable home based on where it falls alphabetically. The nice side effect: if you want to look up everything between “Da” and “De,” you know exactly which single volume to grab, and you can even scan through it in sorted order efficiently. The problem: if your encyclopedia happens to be about celebrities, and suddenly everyone is looking up entries starting with “S” this week for some reason, the poor volume covering “S” gets hammered with traffic while every other volume sits idle. This uneven load is called a hot spot, and it’s the classic weakness of range-based partitioning. A very real-world version of this: if you partition by timestamp (because timestamps happen to sort nicely), all new data being written right now lands on whichever single partition owns “today,” no matter how many machines you have.

Partitioning by hash of the key fixes the hot-spot problem by deliberately throwing away the nice ordering. Instead of “wherever this key alphabetically belongs,” you run the key through a hash function (a formula that turns any input into a number that looks essentially random and evenly spread out) and use that number to pick a machine. Now the data is scattered evenly and unpredictably across all your machines, so no single machine gets stuck holding “today’s” data or “everything starting with S.” The tradeoff: you’ve lost the ability to efficiently ask for a range of keys (like “everything from Monday to Friday”), because a range of real keys no longer corresponds to a nearby range of hash values — they’re scattered randomly across every machine, so a range query now has to ask every single machine instead of just one.

Neither approach is “better” in general — it genuinely depends on whether your application needs efficient range queries more than it needs even load distribution, and that’s a real design decision, not something a database can automatically get right for you.

The celebrity problem

Even hash partitioning, which is supposed to spread load evenly, can still get ambushed by a single extremely popular piece of data — DDIA calls this a skewed workload. Imagine a social media platform where load is partitioned by user ID, which is normally a perfectly reasonable, evenly-distributed choice. Then a celebrity with a hundred million followers posts something, and suddenly one single user ID (the celebrity’s) is generating a wildly disproportionate amount of read and write traffic compared to literally every other user in the system — hashing didn’t help, because the problem was never that user IDs were unevenly spread out, it’s that a small number of specific keys are just genuinely, unavoidably way “hotter” than the rest. One practical trick the book mentions: for a small number of known hot keys, you can append a random suffix (splitting the celebrity’s data across, say, 20 sub-keys instead of one) purely to spread that one key’s load across multiple machines — at the cost of now having to gather and combine results from all 20 sub-keys whenever you need “the real” data for that key.

Searching by something other than the partition key gets messy

Partitioning works cleanly as long as you’re always looking things up by the exact key you partitioned on. Real applications also want to search by other attributes — “find all cars that are red,” when your data is partitioned by car ID, not by color. This is where secondary indexes (an index on a column other than the main key) run into trouble, and DDIA describes two approaches, each with a real cost:

• Local indexes (partitioned by document): each machine keeps an index covering only the data that already physically lives on it — like each branch library keeping an index of only the books in that specific building. Writing new data is simple (you only ever touch the one machine that owns it), but a search like “find all red cars” now has to ask every single machine to check its own local index and then combine all the answers together — a pattern called scatter/gather. It works, but it’s slow and its cost grows with the number of machines you have.
• Global indexes (partitioned by term): instead, you maintain one shared index — say, split up by color instead of by car ID — covering the entire dataset regardless of which machine each car physically lives on. Now “find all red cars” is fast, because you only have to check the one part of the index that covers “red.” The cost shows up on the write side instead: adding one new car might now require updating an index partition that lives on a completely different machine from where the car’s own data lives, and keeping that update honest and consistent across machines is genuinely harder to get right.

I found this a really clean example of a pattern that shows up everywhere in system design: you can’t make both reads and writes maximally cheap and simple at the same time — you’re choosing which side absorbs the complexity, based on which one your application actually does more of.

Rebalancing without a truck full of movers

Systems grow. You’ll eventually add more machines, and existing data has to get reshuffled across the new, larger set of machines — this is called rebalancing. The naive approach — literally computing hash(key) mod number_of_machines to decide where something lives — sounds reasonable until you realize that adding just one extra machine changes the divisor, which changes the answer for almost every single key in the entire database simultaneously. That’s the data equivalent of renumbering every house on every street in a city just because one new street got added — a genuinely enormous, unnecessary amount of data movement for a small change in capacity.

Real systems avoid this in one of a couple of ways:

• Start with far more partitions than you currently have machines — say, always keep exactly 1,000 fixed partitions in existence, even on day one with just a handful of machines. Growing the cluster then just means moving some of those already-existing partitions onto the new machines, rather than recomputing where every single piece of data belongs from scratch.
• Split partitions dynamically as they grow, the way a very popular Wikipedia article’s edit history might eventually get split into “part 1” and “part 2” once it gets unmanageably long — a partition that grows past a size threshold gets automatically split in two, and one of the two halves can then be handed off to a different, less busy machine.

Either way, the goal is the same: change where a chunk of data lives without needing to change how the data is chunked in the first place every time the cluster’s size changes even slightly.

How does a client even know which machine to ask?

If data is scattered across dozens of machines and can be reshuffled at any time as the cluster grows, a client application obviously can’t just hardcode “user data lives on machine 7.” Something in the system has to track, in real time, the current answer to “which machine currently owns this piece of data,” and make that answer available to whoever’s asking. DDIA describes a few shapes this can take — a client might ask any machine and get redirected to the right one, or there might be a dedicated routing layer in front of everything, or the client itself might keep track directly — but in practice, most real systems solve this by leaning on a separate, small, extremely reliable coordination service (ZooKeeper is the most commonly cited example) that acts like a shared, trusted bulletin board: every machine posts “here’s what I currently own” to this bulletin board, and anyone who needs to route a request — the client, a routing tier, or another machine — just checks the board instead of needing to independently track every change themselves.

Tying this back to Chapter 5

The thing I keep coming back to after finishing this chapter: replication and partitioning solve genuinely different problems, but almost no real system uses just one of them. You partition to spread out load and size across many machines, and you replicate each partition so that any one of those machines dying doesn’t cost you the data it was holding. Chapter 5 explained how to keep multiple copies of one dataset in sync. This chapter explained how to split one enormous dataset into pieces small enough for a single machine to reasonably own in the first place. Put together, that’s most of what “distributed database” actually means in practice — everything else is refinements on top of these two decisions.

Book: Designing Data-Intensive Applications by Martin Kleppmann — Chapter 5: Replication.

This chapter answers a question I’d always taken for granted without really thinking through: how does a website or app keep working when one of the machines behind it just… dies? Physical machines fail — hard disks wear out, power supplies blow, someone in a data center trips over the wrong cable. The obvious fix is to keep more than one copy of your data on more than one machine, so if one dies, another one already has the data ready to go. That’s replication, and it sounds simple in one sentence, but the chapter spends its length on everything that makes it genuinely tricky to do safely — and once I worked through it, a bunch of things I used to lump together as “just high availability stuff” turned out to be distinct, specific problems.

Failure isn’t just “on or off”

The first thing that reframed this chapter for me: I used to picture “a machine failing” as a light switch — it’s either fine or it’s dead. Real failures are messier than that. A machine can be slow instead of dead. A network cable can be flaky, dropping some messages but not all. A machine can be perfectly healthy but unreachable from certain other machines due to a network hiccup, while still being reachable from others — this is called a partial failure, and it’s genuinely confusing because from any single other machine’s point of view, “that server is slow to respond” and “that server is dead” look identical until you wait long enough to be sure.

This matters because a system that only knows how to handle the clean, easy case — “the machine crashed and restarted” — falls apart the first time it hits one of these messier, more ambiguous situations. So the real goal of “high availability” isn’t “prevent every possible failure” (you can’t — hardware breaks, that’s just physics), it’s “design the system so that when something breaks, the rest of it can keep going anyway.” Which immediately implies two rules: no single machine should be the only copy of any piece of data, and no single machine should be the only one allowed to accept new writes forever.

Three ways to organize your copies

Given that you’re going to keep several copies of the data, the next question is: who’s allowed to write to which copy, and how do the copies stay in sync? The book walks through three common setups, and I found it much easier to remember them by thinking about who’s “in charge” of writes in each one:

• Single-leader — think of a classroom with one teacher writing notes on the board, and every student just copying down whatever the teacher writes. One machine (the “leader”) is the only one allowed to accept new writes; every other machine (a “follower”) just receives a copy of everything the leader does, in order. This is easy to reason about — there’s never any doubt about which copy is the “real,” most current one — but it has an obvious weak point: if the teacher (the leader) suddenly leaves, someone has to step up and take over, and until that happens, nobody can write anything new.
• Multi-leader — instead of one teacher, imagine several classrooms in different cities, each with its own teacher, and the teachers periodically compare notes with each other to stay in sync. Several machines can all accept writes at the same time (often used when you have offices or users in different countries, so each region can write to a nearby copy instead of one far away). The catch: if two teachers in two different cities write conflicting things onto their boards before comparing notes, someone has to decide which version wins, or how to merge them.
• Leaderless — there’s no teacher at all; any student can just shout out an update, and everyone tries to make sure enough of the group has heard the latest version. Concretely, the client talks directly to several copies at once for both reads and writes, and as long as enough of them agree (a “quorum” — think of it as “enough votes to be confident”), the system considers the operation successful. This avoids ever having a single leader that can fail, but it pushes the hard problem onto figuring out, after the fact, when different copies have quietly drifted out of sync with each other.

What I found genuinely useful here isn’t memorizing the three names — it’s noticing that none of them “solves” the hard problem, they just move it somewhere else. Single-leader moves it to “how do we safely hand off leadership when the leader dies.” Multi-leader moves it to “how do we merge two conflicting versions of the truth.” Leaderless moves it to “how does anyone know if they just read a stale copy.”

When the leader dies, theory meets a genuinely messy reality

This was the part of the chapter that felt the least like abstract theory and the most like real, hard-won engineering experience. “The leader dies, so promote a follower” sounds like a one-line fix, but every step of actually doing that safely has a sharp edge hiding in it:

1. Figuring out the leader is actually dead, and not just briefly slow or temporarily unreachable — usually done by waiting for some timeout period with no response. Wait too short a time and you’ll trigger a false alarm every time the leader is just momentarily busy, kicking off an unnecessary and disruptive handoff. Wait too long and real outages last longer than they need to.
2. Picking which follower becomes the new leader — ideally the one with the most up-to-date copy of the data, not one that’s lagging behind and would silently lose recent writes. Choosing correctly, safely, with multiple machines possibly disagreeing about who should win, is exactly the kind of problem that needs a proper “everyone agrees on one answer” protocol — which is the entire subject of the consensus algorithms covered later, in Chapter 9.
3. Telling every client and every other follower about the new leader, so everyone starts sending writes to the right place going forward.

The genuinely scary failure mode here is called split brain: the old leader comes back online after being unreachable for a bit, doesn’t realize a new leader has already been promoted in its absence, and keeps right on accepting writes as if nothing happened — meanwhile so does the new leader. Now you have two machines both convinced they’re in charge, both accepting different writes, and no clean way to know which version of events is the “real” one without manually untangling the mess afterward. Kleppmann’s blunt point here, which stuck with me: there is no timeout value that is simultaneously perfectly safe and perfectly fast. You’re always picking a point on that tradeoff, not making the tradeoff disappear.

Even when nothing “fails,” copies can still lag behind

Here’s a subtler problem that has nothing to do with anything actually breaking. Followers usually replicate asynchronously — meaning the leader doesn’t wait around for every follower to confirm before telling the original writer “done, saved” — because waiting for every single copy on every single write would make everything painfully slow. But this means followers are always at least a little bit behind the leader, and that small lag can cause genuinely confusing bugs. Picture posting a comment on a website, having it save successfully, and then immediately refreshing the page — only to see the comment has vanished, because your refresh happened to read from a follower that hadn’t caught up yet. Nothing broke. The comment is safe on the leader. But from the user’s point of view, it looks exactly like data loss.

The book names a few specific promises a system can choose to make, to prevent exactly this kind of confusion:

• Read-your-writes — you should always be able to see your own recent changes immediately, even if other people’s reads are still served from a slightly-behind copy.
• Monotonic reads — once you’ve seen a piece of data, you should never see an older version of it later, which can otherwise happen if two of your reads happen to land on two different followers that aren’t equally caught up.
• Consistent prefix reads — if one event genuinely happened before another in reality, nobody should ever see them in the reverse order.

None of these come for free — each one is a specific thing an engineering team has to deliberately build (send a user’s own reads to the leader right after they write something, keep a user’s session pinned to the same follower, track which version of the data a client has already seen). Skip one, and you get a very specific, very reproducible bug that some real user will eventually stumble into.

Why this keeps pointing at Chapter 9

The thread running underneath all of this — choosing a new leader safely, avoiding split brain, agreeing on which of several conflicting writes actually “won” — is really the exact same underlying problem wearing different clothes: getting a group of machines to agree on one answer, even when some of them might be slow, crashed, or unable to talk to each other. That underlying problem has a name, consensus, and it’s serious enough to deserve its own full chapter later in the book (Chapter 9). So “high availability” isn’t really a separate topic from consensus at all — it’s consensus, applied to the one specific question that matters most when you’re keeping a system alive: who is allowed to accept writes right now?

What actually changed in how I think about this

Before this chapter, “high availability” was a vague, feel-good phrase to me — something you got by “adding more servers.” After it, I see it as a set of very concrete, very deliberate tradeoffs: how long to wait before declaring a leader dead, which specific staleness guarantees you’re willing to promise your users, and how you’ll detect and untangle conflicting writes when they happen. None of that comes from the failure-free, everything-working case — it all comes from deliberately planning for the mess.

Book: Designing Data-Intensive Applications by Martin Kleppmann — Chapter 4: Encoding and Evolution.

This chapter closes out Part I of the book, and it’s about a problem that sounds almost too obvious to write a whole chapter on: how do you turn an in-memory object — a struct, a class instance, whatever your programming language calls it — into bytes you can send over a network or write to disk, and then turn it back into an object later? That’s called encoding (or serialization). The reason it deserves a full chapter is the second half of the title: evolution — what happens when the shape of that data needs to change, but you can’t update every single reader and writer of it at exactly the same instant.

Why “just update everything at once” is never actually an option

This is the part of the chapter that reframed the whole topic for me. In any system beyond a toy project, you basically never get to update all your code and all your data at the same instant:

• Server-side rolling deploys — when you deploy new code to a fleet of, say, a hundred servers, you don’t take them all down at once (that would mean an outage). You update them a few at a time, which means for some window of time, old code and new code are both running simultaneously, and they might both be reading and writing the exact same data or exchanging messages with each other.
• Client-side apps — a user might not update their mobile app for months. The new server has to keep working correctly with old app versions still in the wild, sometimes for years.
• Data outlives code — a row written to a database five years ago, by code that no longer exists, still has to be readable by whatever code is running today.

Once I saw it framed this way, I realized “backward compatibility” isn’t some extra nice-to-have feature — it’s a mechanical necessity of how real deployments physically work. You can’t will your way out of the fact that old and new code overlap in time; you can only design your encoding so that the overlap doesn’t break anything.

Two compatibility directions, and why you need both

The book defines two distinct kinds of compatibility, and I found it useful to think of them as two separate questions rather than one blurry concept:

• Backward compatibility — can newer code read data that was written by older code? This is usually the easier one: newer code typically knows about old fields and can just keep supporting them.
• Forward compatibility — can older code read data written by newer code? This is the trickier direction, because old code, by definition, doesn’t know about fields that didn’t exist yet when it was written. The old code has to be written defensively enough to just ignore fields it doesn’t recognize, rather than crashing on them.

During a rolling deploy, you genuinely need both at once, in both directions, because at any given moment some machines are old and some are new, and they’re all talking to each other simultaneously.

Text formats vs. binary formats

The chapter walks through a few families of encoding formats, and the tradeoff pattern is one I now recognize everywhere:

Text-based formats — JSON, XML, CSV. Their big advantage is that they’re human-readable — you can open one in a plain text editor and understand it, which is genuinely valuable for debugging. Their downside is that they’re verbose (field names get repeated in every single record) and somewhat ambiguous about types (JSON, for instance, famously doesn’t distinguish integers from floating-point numbers, and has no native way to represent large numbers precisely — a real, documented source of bugs when large IDs get silently rounded).

Binary formats with a schema — Protocol Buffers (Google), Thrift (originally Facebook), Avro (Apache, heavily used inside the Kafka/LinkedIn ecosystem). Instead of repeating field names in every message, you define a schema once — “field 1 is name, a string; field 2 is age, an integer” — and the encoded bytes just contain the values in a known order, referencing fields by a short numeric tag instead of spelling out the name every time. This is dramatically smaller on the wire and faster to parse, at the cost of needing that schema definition to make sense of the raw bytes at all.

What made schemas click for me as more than “a way to save bytes”: a schema is also a contract. It’s the thing that lets Protobuf and Avro define precise, checkable rules for what counts as a backward- or forward-compatible change — for example, you’re generally allowed to add a new optional field (old readers just skip it, satisfying forward compatibility) but you’re generally not allowed to change a field’s type or reuse someone else’s old field number, because that silently corrupts data for whichever version doesn’t expect it.

Dataflow: how encoded data actually travels

The last part of the chapter looks at the different paths encoded data takes through a real system, and each path has its own compatibility expectations:

• Through a database — you write a row today, and it might be read years from now by a completely different version of your application. This is backward compatibility stretched out over a very long timescale, which is why database schema migrations are treated so carefully in real teams — you’re not just changing today’s code, you’re committing to how every future version of your code will need to interpret today’s write.
• Through service calls (REST, RPC) — one service calls another over the network, and the two services are very likely running different code versions at any given time, especially in a company running dozens of independently-deployed microservices. Both directions of compatibility matter constantly, because you rarely control exactly when every other team deploys their side.
• Through a message queue (like Kafka) — a message gets written once by a producer and might be read later by several different consumers, each potentially running different code versions, and the message itself might sit in the queue for a while before anyone reads it. This is arguably the trickiest case, because the producer often doesn’t even know who all the eventual readers are, or what version of the code they’ll be running when they finally read it.

Why this closes out Part I so well

Looking back at the four chapters so far — reliability and scalability, data models, storage engines, and now encoding — I can see why the book groups them as “foundations.” They’re all about a single machine’s honest relationship with its own data: how to survive faults, how to shape data, how to store it, and now, how to let that shape safely change over time. Everything from here on — replication, partitioning, transactions, consensus — is about coordinating multiple machines, and none of that works if a single machine can’t even reliably read data that a slightly different version of itself wrote yesterday.

Book: Designing Data-Intensive Applications by Martin Kleppmann — Chapter 3: Storage and Retrieval.

After data models, the book goes one level deeper: forget how your data looks to your application for a moment — how does the database actually store it on an actual physical disk, in a way that lets it find any given piece of it back quickly? This is the chapter where I finally understood why Postgres and Cassandra genuinely behave like different animals under load, instead of just “both being databases.”

The simplest possible database, and why it’s bad

The book opens with a deliberately silly toy database: a script that appends every write as a new line to a text file, and looks things up by scanning the entire file from the top every single time. Writes are blazing fast (just append a line — nothing to search for, nothing to reorganize). Reads are painfully slow once the file gets big, because every single lookup means scanning potentially millions of lines. This toy example matters because it sets up the entire chapter’s central tension: fast writes and fast reads pull the storage engine’s design in opposite directions, and every real storage engine is really just a specific, clever compromise between the two.

The first real improvement the book introduces is an index — a separate, smaller structure that tells you exactly where to look, so reads don’t need a full scan. But an index isn’t free: every index you add makes writes slightly slower, because now every write has to update the index too, not just append the raw data. This is a genuinely useful mental model I didn’t have before: every index is a bet that you’ll read that field often enough to be worth paying a small write-cost on every single write, forever.

The two families: B-trees and LSM-trees

The chapter’s real payoff is a head-to-head comparison of the two dominant storage engine designs used in real production databases today, and once I saw them side by side, a lot of “why does this database behave this way” questions I’d had for years just resolved themselves:

B-trees — used by PostgreSQL, MySQL/InnoDB, and Oracle, among many others — organize data on disk into fixed-size pages arranged in a tree, where each write finds the exact page a key belongs to and updates it directly in place. This is the same basic idea as an old-school library card catalog: everything is kept in strict, browsable order, so both “find this one book” and “find every book between these two call numbers” are fast and predictable. The cost is that every single write means finding a specific spot on the physical disk and modifying it there, which — especially on spinning disks, and even on SSDs to a lesser degree — is a comparatively expensive operation called a seek, and it happens on every write.

LSM-trees (log-structured merge-trees) — used by Cassandra, RocksDB, LevelDB, and HBase — take the toy append-only log idea from the start of the chapter and make it genuinely production-grade. New writes first land in a small, fast, in-memory structure (a memtable). Once that fills up, it gets flushed to disk as an immutable file (an SSTable), and a background process periodically merges older SSTables together, discarding overwritten or deleted values along the way — a process called compaction. Because writes never modify anything in place — they only ever append — write throughput can be dramatically higher than a B-tree’s. The cost shows up on reads: a single lookup might, in the worst case, have to check the memtable and several SSTables before it finds the answer (or confirms it doesn’t exist), though clever tricks like Bloom filters — a small, probabilistic structure that can quickly say “this key is definitely not in this file” without actually reading the file — claw a lot of that read cost back.

The everyday analogy that stuck with me: a B-tree is like keeping one perfectly alphabetized filing cabinet, where every new document gets carefully inserted into its exact correct spot right away — precise, but slow to file each new piece of paper. An LSM-tree is like tossing every new document onto an always-growing “inbox” pile, and only periodically, in the background, sorting and merging those piles into neater ones — fast to receive new paper, but you might have to check a few piles before you’re sure you’ve found (or ruled out) a given document.

Neither is “better” — it comes back to the same tension from the very start of the chapter. Workloads that write constantly and read less often (logging systems, time-series data, write-heavy analytics pipelines) tend to love LSM-trees. Workloads that need consistently fast, predictable reads and can tolerate somewhat slower writes tend to reach for B-trees. This is exactly why Cassandra (LSM-tree) is the go-to choice for something like storing a constant stream of IoT sensor data, while Postgres (B-tree) is the go-to choice for a banking system’s account balances, where you’re reading a specific account far more often than you’re bulk-writing.

OLTP vs OLAP: two completely different jobs wearing the same word, “database”

The last part of the chapter draws a line I’d always sort of sensed but never had clean vocabulary for: OLTP (online transaction processing) versus OLAP (online analytic processing). They’re both “databases,” but they’re optimized for almost opposite access patterns:

• OLTP is what runs your actual application in real time — a user checking their bank balance, an app adding an item to a cart. Each query touches a small number of rows, but there are a huge number of concurrent queries happening every second, from many different users, each looking at their own tiny slice of the data.
• OLAP is what a business analyst runs at the end of the quarter to answer something like “what was our total revenue by region, across the last two years?” Each query might scan millions of rows, but there are relatively few such queries, usually run by internal analysts rather than end users in real time.

This split explains a design choice I’d seen without understanding why: column-oriented storage, used by systems like ClickHouse (which I wrote a separate paper note on — its whole architecture is basically a deep dive into exactly this idea). A normal row-oriented database stores an entire row together on disk, which is efficient for OLTP (“give me everything about this one order”) but wasteful for OLAP (“give me just the revenue column, summed, across 50 million orders”) — you’d end up reading every other column of every row just to throw it away. Column-oriented storage flips this: it stores each column together on disk instead, so an analytical query that only cares about revenue and region can skip reading every other column entirely, often making a scan over millions of rows dramatically faster.

Why this chapter earns its place before replication and partitioning

Reading this chapter clarified something for me about the replication and partitioning chapters I’d already written notes on: those chapters are about where copies of your data live and how it’s split across machines, but this chapter is about what’s actually happening on a single machine’s disk underneath all of that. A partitioned, replicated cluster of machines each running a badly-suited storage engine for the workload is still going to be slow — partitioning and replication solve “too much data/traffic for one machine,” not “the wrong data structure for this access pattern.” Both problems are real, and both need solving, but they’re genuinely separate layers of the same system.