change en docs anchor

This commit is contained in:
Feng Ruohang 2025-08-10 09:16:43 +08:00
parent 4f025dd1e9
commit 95781d99ba
11 changed files with 285 additions and 102 deletions

View File

@ -438,7 +438,7 @@ cloud or on-premises—for example, whether you use an orchestration framework s
However, choice of deployment tooling is out of scope of this book, since other factors have a
greater influence on the architecture of data systems.
### Pros and Cons of Cloud Services {#pros-and-cons-of-cloud-services}
### Pros and Cons of Cloud Services {#sec_introduction_cloud_tradeoffs}
Using a cloud service, rather than running comparable software yourself, essentially outsources the
operation of that software to the cloud provider. There are good arguments for and against cloud
@ -832,7 +832,7 @@ a metered billing model, the serverless approach is bringing metered billing to
only pay for the time that your application code is actually running, rather than having to
provision resources in advance.
### Cloud Computing versus Supercomputing {#cloud-computing-versus-supercomputing}
### Cloud Computing versus Supercomputing {#id17}
Cloud computing is not the only way of building large-scale computing systems; an alternative is
*high-performance computing* (HPC), also known as *supercomputing*. Although there are overlaps, HPC
@ -973,7 +973,7 @@ data is being processed—an aspect that many engineers are prone to ignoring. H
requirements into technical implementations is not yet well understood, but its important to keep
this question in mind as we move through the rest of this book.
### References {#references}
### References
[^1]: Richard T. Kouzes, Gordon A. Anderson, Stephen T. Elbert, Ian Gorton, and Deborah K. Gracio. [The Changing Paradigm of Data-Intensive Computing](http://www2.ic.uff.br/~boeres/slides_AP/papers/TheChanginParadigmDataIntensiveComputing_2009.pdf). *IEEE Computer*, volume 42, issue 1, January 2009. [doi:10.1109/MC.2009.26](https://doi.org/10.1109/MC.2009.26)
[^2]: Martin Kleppmann, Adam Wiggins, Peter van Hardenberg, and Mark McGranaghan. [Local-first software: you own your data, in spite of the cloud](https://www.inkandswitch.com/local-first/). At *2019 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software* (Onward!), October 2019. [doi:10.1145/3359591.3359737](https://doi.org/10.1145/3359591.3359737)

View File

@ -99,7 +99,7 @@ his query) *after* he heard Aaliyah exclaim the final score, and therefore he ex
result to be at least as recent as Aaliyahs. The fact that his query returned a stale result is a
violation of linearizability.
### What Makes a System Linearizable? {#what-makes-a-system-linearizable}
### What Makes a System Linearizable? {#sec_consistency_lin_definition}
In order to understand linearizability better, lets look at some more examples.
[Figure 10-2](/en/ch10#fig_consistency_linearizability_1) shows three clients concurrently reading and writing the same
@ -262,7 +262,7 @@ largely independently from each other [^15] [^16].
--------
### Relying on Linearizability {#relying-on-linearizability}
### Relying on Linearizability {#sec_consistency_linearizability_usage}
In what circumstances is linearizability useful? Viewing the final score of a sporting match is
perhaps a frivolous example: a result that is outdated by a few seconds is unlikely to cause any
@ -377,7 +377,7 @@ queue, but not in the case of Aaliyah and Bryce), you can use alternative approa
we discussed in [“Reading Your Own Writes”](/en/ch6#sec_replication_ryw), at the cost of additional complexity.
### Implementing Linearizable Systems {#implementing-linearizable-systems}
### Implementing Linearizable Systems {#sec_consistency_implementing_linearizable}
Now that weve looked at a few examples in which linearizability is useful, lets think about how we
might implement a system that offers linearizable semantics.
@ -464,7 +464,7 @@ linearizable compare-and-set operation cannot, because it requires a consensus a
In summary, it is safest to assume that a leaderless system with Dynamo-style replication does not
provide linearizability, even with quorum reads and writes.
### The Cost of Linearizability {#the-cost-of-linearizability}
### The Cost of Linearizability {#sec_linearizability_cost}
As some replication methods can provide linearizability and others cannot, it is interesting to
explore the pros and cons of linearizability in more depth.
@ -689,7 +689,7 @@ using atomic clocks or GPS receivers. But it would also be nice to be able to ge
unique and correctly ordered without relying on special hardware. Thats what *logical clocks* are
about.
### Logical Clocks {#logical-clocks}
### Logical Clocks {#sec_consistency_timestamps}
In [“Unreliable Clocks”](/en/ch9#sec_distributed_clocks) we discussed time-of-day clocks and monotonic clocks. Both of these
are *physical clocks*: they measure the passing of seconds (or milliseconds, microseconds, etc.).
@ -896,7 +896,7 @@ stronger than logical clocks or ID generators: we need consensus.
## Consensus {#consensus}
## Consensus {#sec_consistency_consensus}
In this chapter we have seen several examples of things that are easy when you have only a single
node, but which get a lot harder if you want fault tolerance:
@ -953,7 +953,7 @@ importance, distributed systems can usually achieve consensus in practice.
--------
### The Many Faces of Consensus {#the-many-faces-of-consensus}
### The Many Faces of Consensus {#sec_consistency_faces}
Consensus can be expressed in several different ways:
@ -1223,7 +1223,7 @@ consensus; if atomic commit aborts, the proposing node retries with a new transa
This shows that atomic commit and consensus are also equivalent to each other.
### Consensus in Practice {#consensus-in-practice}
### Consensus in Practice {#sec_consistency_total_order}
We have seen that single-value consensus, CAS, shared logs, and atomic commitment are all equivalent
to each other: you can convert a solution to one of them into a solution to any of the others. That
@ -1416,7 +1416,7 @@ generally dont offer linearizability, but for applications that dont need
### Coordination Services {#coordination-services}
### Coordination Services {#sec_consistency_coordination}
Consensus algorithms are useful in any distributed database that wants to offer linearizable
operations, and many modern distributed databases use consensus algorithms for replication. But one
@ -1625,7 +1625,7 @@ availability and better performance. In these cases, it is common to use leaderl
replication, which we previously discussed in [Chapter 6](/en/ch6#ch_replication). The logical clocks that we
discussed in this chapter are helpful in that context.
### References {#references}
### References
[^1]: Maurice P. Herlihy and Jeannette M. Wing. [Linearizability: A Correctness Condition for Concurrent Objects](https://cs.brown.edu/~mph/HerlihyW90/p463-herlihy.pdf). *ACM Transactions on Programming Languages and Systems* (TOPLAS), volume 12, issue 3, pages 463492, July 1990. [doi:10.1145/78969.78972](https://doi.org/10.1145/78969.78972)
[^2]: Leslie Lamport. [On interprocess communication](https://www.microsoft.com/en-us/research/publication/interprocess-communication-part-basic-formalism-part-ii-algorithms/). *Distributed Computing*, volume 1, issue 2, pages 77101, June 1986. [doi:10.1007/BF01786228](https://doi.org/10.1007/BF01786228)

View File

@ -51,7 +51,7 @@ Lets also assume that the average user follows 200 people and has 200 followe
a very wide range: most people have only a handful of followers, and a few celebrities such as
Barack Obama have over 100 million followers).
### Representing Users, Posts, and Follows {#representing-users-posts-and-follows}
### Representing Users, Posts, and Follows {#id20}
Imagine we keep all of the data in a relational database as shown in [Figure 2-1](/en/ch2#fig_twitter_relational). We
have one table for users, one table for posts, and one table for follow relationships.
@ -201,7 +201,7 @@ the current hardware can handle, the capacity needs to be expanded; a system is
In this section we will focus primarily on response times, and we will return to throughput and
scalability in [“Scalability”](/en/ch2#sec_introduction_scalability).
### Latency and Response Time {#latency-and-response-time}
### Latency and Response Time {#id23}
“Latency” and “response time” are sometimes used interchangeably, but in this book we will use the
terms in a specific way (illustrated in [Figure 2-4](/en/ch2#fig_response_time)):
@ -237,7 +237,7 @@ service times, the client will see a slow overall response time due to the time
prior request to complete. The queueing delay is not part of the service time, and for this reason
it is important to measure response times on the client side.
### Average, Median, and Percentiles {#average-median-and-percentiles}
### Average, Median, and Percentiles {#id24}
Because the response time varies from one request to the next, we need to think of it not as a
single number, but as a *distribution* of values that you can measure. In [Figure 2-5](/en/ch2#fig_lognormal), each
@ -382,7 +382,7 @@ However, if the system youre talking about contains many hard drives, then th
hard drive is only a fault from the point of view of the bigger system, and the bigger system might
be able to tolerate that fault by having a copy of the data on another hard drive.
### Fault Tolerance {#fault-tolerance}
### Fault Tolerance {#id27}
We call a system *fault-tolerant* if it continues providing the required service to the user in
spite of certain faults occurring. If a system cannot tolerate a certain part becoming faulty, we
@ -512,7 +512,7 @@ help: carefully thinking about assumptions and interactions in the system; thoro
isolation; allowing processes to crash and restart; avoiding feedback loops such as retry storms
(see [“When an overloaded system wont recover”](/en/ch2#sidebar_metastable)); measuring, monitoring, and analyzing system behavior in production.
### Humans and Reliability {#humans-and-reliability}
### Humans and Reliability {#id31}
Humans design and build software systems, and the operators who keep the systems running are also
human. Unlike machines, humans dont just follow rules; their strength is being creative and
@ -620,7 +620,7 @@ you will learn where your performance bottlenecks lie, and therefore you will kn
dimensions you need to scale. At that point its time to start worrying about techniques for
scalability.
### Describing Load {#describing-load}
### Describing Load {#id33}
First, we need to succinctly describe the current load on the system; only then can we discuss
growth questions (what happens if our load doubles?). Often this will be a measure of throughput:
@ -699,7 +699,7 @@ scalability problems of older systems: instead of providing a filesystem (NAS) o
abstraction, the storage service offers a specialized API that is designed for the specific needs of
the database [^83].
### Principles for Scalability {#principles-for-scalability}
### Principles for Scalability {#id35}
The architecture of systems that operate at large scale is usually highly specific to the
application—there is no such thing as a generic, one-size-fits-all scalable architecture
@ -767,7 +767,7 @@ Evolvability
it for unanticipated use cases as requirements change.
### Operability: Making Life Easy for Operations {#operability-making-life-easy-for-operations}
### Operability: Making Life Easy for Operations {#id37}
We previously discussed the role of operations in [“Operations in the Cloud Era”](/en/ch1#sec_introduction_operations), and we saw that
human processes are at least as important for reliable operations as software tools. In fact, it has
@ -790,8 +790,7 @@ on high-value activities. Data systems can do various things to make routine tas
* Allowing monitoring tools to check the systems key metrics, and supporting observability tools
(see [“Problems with Distributed Systems”](/en/ch1#sec_introduction_dist_sys_problems)) to give insights into the systems runtime behavior.
A variety of commercial and open source tools can help here
[^90].
A variety of commercial and open source tools can help here [^90].
* Avoiding dependency on individual machines (allowing machines to be taken down for maintenance
while the system as a whole continues running uninterrupted)
* Providing good documentation and an easy-to-understand operational model (“If I do X, Y will happen”)
@ -799,7 +798,7 @@ on high-value activities. Data systems can do various things to make routine tas
* Self-healing where appropriate, but also giving administrators manual control over the system state when needed
* Exhibiting predictable behavior, minimizing surprises
### Simplicity: Managing Complexity {#simplicity-managing-complexity}
### Simplicity: Managing Complexity {#id38}
Small software projects can have delightfully simple and expressive code, but as projects get
larger, they often become very complex and difficult to understand. This complexity slows down
@ -896,7 +895,7 @@ There are no easy answers on how to achieve these things, but one thing that can
applications using well-understood building blocks that provide useful abstractions. The rest of
this book will cover a selection of building blocks that have proved to be valuable in practice.
### References {#references}
### References
[^1]: Mike Cvet. [How We Learned to Stop Worrying and Love Fan-In at Twitter](https://www.youtube.com/watch?v=WEgCjwyXvwc). At *QCon San Francisco*, December 2016.
[^2]: Raffi Krikorian. [Timelines at Scale](https://www.infoq.com/presentations/Twitter-Timeline-Scalability/). At *QCon San Francisco*, November 2012. Archived at [perma.cc/V9G5-KLYK](https://perma.cc/V9G5-KLYK)

View File

@ -98,7 +98,7 @@ documents are thought to be more flexible.
The pros and cons of document and relational data have been debated extensively; lets examine some
of the key points of that debate.
### The Object-Relational Mismatch {#the-object-relational-mismatch}
### The Object-Relational Mismatch {#sec_datamodels_document}
Much application development today is done in object-oriented programming languages, which leads to
a common criticism of the SQL data model: if data is stored in relational tables, an awkward
@ -693,7 +693,7 @@ sections where schema flexibility is beneficial. Relational-document hybrids are
--------
## Graph-Like Data Models {#graph-like-data-models}
## Graph-Like Data Models {#sec_datamodels_graph}
We saw earlier that the type of relationships is an important distinguishing feature between
different data models. If your application has mostly one-to-many relationships (tree-structured
@ -761,7 +761,7 @@ in graph databases, but difficult in other models.
{{< figure src="/fig/ddia_0306.png" id="fig_datamodels_graph" caption="Figure 3-6. Example of graph-structured data (boxes represent vertices, arrows represent edges)." class="w-full my-4" >}}
### Property Graphs {#property-graphs}
### Property Graphs {#id56}
In the *property graph* (also known as *labeled property graph*) model, each vertex consists of:
@ -850,7 +850,7 @@ substances. Then you could write a query to find out what is safe for each perso
Graphs are good for evolvability: as you add features to your application, a graph can easily be
extended to accommodate changes in your applications data structures.
### The Cypher Query Language {#the-cypher-query-language}
### The Cypher Query Language {#id57}
*Cypher* is a query language for property graphs, originally created for the Neo4j graph database,
and later developed into an open standard as *openCypher* [^38]. Besides Neo4j, Cypher is supported by Memgraph, KùzuDB [^35],
@ -919,7 +919,7 @@ Europe. Then you can proceed to find all locations (states, regions, cities, etc
Europe respectively by following all incoming `WITHIN` edges. Finally, you can look for people who
can be found through an incoming `BORN_IN` or `LIVES_IN` edge at one of the location vertices.
### Graph Queries in SQL {#graph-queries-in-sql}
### Graph Queries in SQL {#id58}
[Example 3-3](/en/ch3#fig_graph_sql_schema) suggested that graph data can be represented in a relational database. But
if we put graph data in a relational structure, can we also query it using SQL?
@ -1018,7 +1018,7 @@ Oracle has a different SQL extension for recursive queries, which it calls *hier
However, the situation may be improving: at the time of writing, there are plans to add a graph
query language called GQL to the SQL standard [^42] [^43], which will provide a syntax inspired by Cypher, GSQL [^44], and PGQL [^45].
### Triple-Stores and SPARQL {#triple-stores-and-sparql}
### Triple-Stores and SPARQL {#id59}
The triple-store model is mostly equivalent to the property graph model, using different words to
describe the same ideas. It is nevertheless worth discussing, because there are various tools and
@ -1203,7 +1203,7 @@ bound to any vertex that has a `name` property whose value is the string `"Unite
SPARQL is supported by Amazon Neptune, AllegroGraph, Blazegraph, OpenLink Virtuoso, Apache Jena, and
various other triple stores [^36].
### Datalog: Recursive Relational Queries {#datalog-recursive-relational-queries}
### Datalog: Recursive Relational Queries {#id62}
Datalog is a much older language than SPARQL or Cypher: it arose from academic research in the 1980s [^57] [^58] [^59].
It is less well known among software engineers and not widely supported in mainstream databases, but
@ -1307,7 +1307,7 @@ referring to other rules, similarly to the way that you break down code into fun
each other. Just like functions can be recursive, Datalog rules can also invoke themselves, like
rule 2 in [Example 3-12](/en/ch3#fig_datalog_query), which enables graph traversals in Datalog queries.
### GraphQL {#graphql}
### GraphQL {#id63}
GraphQL is a query language that, by design, is much more restrictive than the other query languages
we have seen in this chapter. The purpose of GraphQL is to allow client software running on a users
@ -1676,7 +1676,7 @@ come into play when *implementing* the data models described in this chapter.
### References {#references}
### References
[^1]: Jamie Brandon. [Unexplanations: query optimization works because sql is declarative](https://www.scattered-thoughts.net/writing/unexplanations-sql-declarative/). *scattered-thoughts.net*, February 2024. Archived at [perma.cc/P6W2-WMFZ](https://perma.cc/P6W2-WMFZ)
[^2]: Joseph M. Hellerstein. [The Declarative Imperative: Experiences and Conjectures in Distributed Logic](https://www2.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-90.pdf). Tech report UCB/EECS-2010-90, Electrical Engineering and Computer Sciences, University of California at Berkeley, June 2010. Archived at [perma.cc/K56R-VVQM](https://perma.cc/K56R-VVQM)

View File

@ -37,7 +37,7 @@ Later in [“Data Storage for Analytics”](/en/ch4#sec_storage_analytics) we
analytics, and in [“Multidimensional and Full-Text Indexes”](/en/ch4#sec_storage_multidimensional) well briefly look at indexes for more advanced
queries, such as text retrieval.
## Storage and Indexing for OLTP {#storage-and-indexing-for-oltp}
## Storage and Indexing for OLTP {#sec_storage_oltp}
Consider the worlds simplest database, implemented as two Bash functions:
@ -133,7 +133,7 @@ writing the application or administering the database—to choose indexes manual
knowledge of the applications typical query patterns. You can then choose the indexes that give
your application the greatest benefit, without introducing more overhead on writes than necessary.
### Log-Structured Storage {#log-structured-storage}
### Log-Structured Storage {#sec_storage_log_structured}
To start, lets assume that you want to continue storing data in the append-only file written by
`db_set`, and you just want to speed up reads. One way you could do this is by keeping a hash map in
@ -726,7 +726,7 @@ and analytical processing (HTAP) databases (introduced in [“Data Warehousing
becoming two separate storage and query engines, which happen to be accessible through a common SQL
interface [^50] [^51] [^52] [^53].
### Cloud Data Warehouses {#cloud-data-warehouses}
### Cloud Data Warehouses {#sec_cloud_data_warehouses}
Data warehouse vendors such as Teradata, Vertica, and SAP HANA sell both on-premises warehouses
under commercial licenses and cloud-based solutions. But as many of their customers move to the
@ -779,7 +779,7 @@ Data catalog
integrated, but decoupling them has enabled data discovery and data governance systems
(discussed in [“Data Systems, Law, and Society”](/en/ch1#sec_introduction_compliance)) to access a catalogs metadata as well.
### Column-Oriented Storage {#column-oriented-storage}
### Column-Oriented Storage {#sec_storage_column}
As discussed in [“Stars and Snowflakes: Schemas for Analytics”](/en/ch3#sec_datamodels_analytics), data warehouses by convention often use a relational
schema with a big fact table that contains foreign key references into dimension tables.
@ -1018,7 +1018,7 @@ performance by taking advantages of the characteristics of modern CPUs:
* operating directly on compressed data without decoding it into a separate in-memory
representation, which saves memory allocation and copying costs.
### Materialized Views and Data Cubes {#materialized-views-and-data-cubes}
### Materialized Views and Data Cubes {#sec_storage_materialized_views}
We previously encountered *materialized views* in [“Materializing and Updating Timelines”](/en/ch2#sec_introduction_materializing):
in a relational data model, they are table-like object whose contents are the results of some
@ -1156,7 +1156,7 @@ It does this by storing the set of terms as a finite state automaton over the ch
and transforming it into a *Levenshtein automaton*, which supports efficient search for words within a given edit distance [^97].
### Vector Embeddings {#vector-embeddings}
### Vector Embeddings {#id92}
Semantic search goes beyond synonyms and typos to try and understand document concepts
and user intentions. For example, if your help pages contain a page titled “cancelling your
@ -1288,7 +1288,7 @@ documentation for the database of your choice.
### References {#references}
### References
[^1]: Nikolay Samokhvalov. [How partial, covering, and multicolumn indexes may slow down UPDATEs in PostgreSQL](https://postgres.ai/blog/20211029-how-partial-and-covering-indexes-affect-update-performance-in-postgresql). *postgres.ai*, October 2021. Archived at [perma.cc/PBK3-F4G9](https://perma.cc/PBK3-F4G9)
[^2]: Goetz Graefe. [Modern B-Tree Techniques](https://w6113.github.io/files/papers/btreesurvey-graefe.pdf). *Foundations and Trends in Databases*, volume 3, issue 4, pages 203402, August 2011. [doi:10.1561/1900000028](https://doi.org/10.1561/1900000028)

View File

@ -68,7 +68,7 @@ formats are used for data storage and for communication: in databases, web servi
remote procedure calls (RPC), workflow engines, and event-driven systems such as actors and
message queues.
## Formats for Encoding Data {#formats-for-encoding-data}
## Formats for Encoding Data {#sec_encoding_formats}
Programs usually work with data in (at least) two different representations:
@ -104,7 +104,7 @@ However, most systems need to convert between in-memory objects and flat byte se
such a common problem, there are a myriad different libraries and encoding formats to choose from.
Lets do a brief overview.
### Language-Specific Formats {#language-specific-formats}
### Language-Specific Formats {#id96}
Many programming languages come with built-in support for encoding in-memory objects into byte
sequences. For example, Java has `java.io.Serializable`, Python has `pickle`, Ruby has `Marshal`,
@ -273,7 +273,7 @@ In the following sections we will see how we can do much better, and encode the
{{< figure link="#fig_encoding_json" src="/fig/ddia_0502.png" id="fig_encoding_messagepack" caption="Figure 5-2. Example record Example 5-2 encoded using MessagePack." class="w-full my-4" >}}
### Protocol Buffers {#protocol-buffers}
### Protocol Buffers {#sec_encoding_protobuf}
Protocol Buffers (protobuf) is a binary encoding library developed at Google.
It is similar to Apache Thrift, which was originally developed by Facebook [^13];
@ -363,7 +363,7 @@ because the parser can fill in any missing bits with zeros. However, if old code
by new code, the old code is still using a 32-bit variable to hold the value. If the decoded 64-bit
value wont fit in 32 bits, it will be truncated.
### Avro {#avro}
### Avro {#sec_encoding_avro}
Apache Avro is another binary encoding format that is interestingly different from Protocol Buffers.
It was started in 2009 as a subproject of Hadoop, as a result of Protocol Buffers not being a good
@ -541,7 +541,7 @@ automate this, but the schema generator would have to be very careful to not ass
field tags.) This kind of dynamically generated schema simply wasnt a design goal of Protocol
Buffers, whereas it was for Avro.
### The Merits of Schemas {#the-merits-of-schemas}
### The Merits of Schemas {#sec_encoding_schemas}
As we saw, Protocol Buffers and Avro both use a schema to describe a binary encoding format. Their
schema languages are much simpler than XML Schema or JSON Schema, which support much more detailed
@ -580,7 +580,7 @@ In summary, schema evolution allows the same kind of flexibility as schemaless/s
databases provide (see [“Schema flexibility in the document model”](/en/ch3#sec_datamodels_schema_flexibility)), while also providing better
guarantees about your data and better tooling.
## Modes of Dataflow {#modes-of-dataflow}
## Modes of Dataflow {#sec_encoding_dataflow}
At the beginning of this chapter we said that whenever you want to send some data to another process
with which you dont share memory—for example, whenever you want to send data over the network or
@ -1160,7 +1160,7 @@ quite achievable. May your applications evolution be rapid and your deploymen
### References {#references}
### References
[^1]: [CWE-502: Deserialization of Untrusted Data](https://cwe.mitre.org/data/definitions/502.html). Common Weakness Enumeration, *cwe.mitre.org*, July 2006. Archived at [perma.cc/26EU-UK9Y](https://perma.cc/26EU-UK9Y)
[^2]: Steve Breen. [What Do WebLogic, WebSphere, JBoss, Jenkins, OpenNMS, and Your Application Have in Common? This Vulnerability](https://foxglovesecurity.com/2015/11/06/what-do-weblogic-websphere-jboss-jenkins-opennms-and-your-application-have-in-common-this-vulnerability/). *foxglovesecurity.com*, November 2015. Archived at [perma.cc/9U97-UVVD](https://perma.cc/9U97-UVVD)

View File

@ -63,7 +63,7 @@ current state of the database in primary storage.
--------
## Single-Leader Replication {#single-leader-replication}
## Single-Leader Replication {#sec_replication_leader}
Each node that stores a copy of the database is called a *replica*. With multiple replicas, a
question inevitably arises: how do we ensure that all the data ends up on all the replicas?
@ -107,7 +107,7 @@ elect a new leader if the old one fails (we will discuss consensus in more detai
--------
### Synchronous Versus Asynchronous Replication {#synchronous-versus-asynchronous-replication}
### Synchronous Versus Asynchronous Replication {#sec_replication_sync_async}
An important detail of a replicated system is whether the replication happens *synchronously* or
*asynchronously*. (In relational databases, this is often a configurable option; other systems are
@ -589,7 +589,7 @@ zones/datacenters in a single geographic location.
--------
### Monotonic Reads {#monotonic-reads}
### Monotonic Reads {#sec_replication_monotonic_reads}
Our second example of an anomaly that can occur when reading from asynchronous followers is that its
possible for a user to see things *moving backward in time*.
@ -662,7 +662,7 @@ the same shard—but in some applications that cannot be done efficiently. There
that explicitly keep track of causal dependencies, a topic that we will return to in
[“The “happens-before” relation and concurrency”](/en/ch6#sec_replication_happens_before).
### Solutions for Replication Lag {#solutions-for-replication-lag}
### Solutions for Replication Lag {#id131}
When working with an eventually consistent system, it is worth thinking about how the application
behaves if the replication lag increases to several minutes or even hours. If the answer is “no
@ -1210,7 +1210,7 @@ in others, a coordinator node does this on behalf of the client. However, unlike
that coordinator does not enforce a particular ordering of writes. As we shall see, this difference in design has
profound consequences for the way the database is used.
### Writing to the Database When a Node Is Down {#writing-to-the-database-when-a-node-is-down}
### Writing to the Database When a Node Is Down {#id287}
Imagine you have a database with three replicas, and one of the replicas is currently
unavailable—perhaps it is being rebooted to install a system update. In a single-leader
@ -1401,7 +1401,7 @@ Eventual consistency is a deliberately vague guarantee, but for operability it
able to quantify “eventual.”
### Single-Leader vs. Leaderless Replication Performance {#single-leader-vs-leaderless-replication-performance}
### Single-Leader vs. Leaderless Replication Performance {#sec_replication_leaderless_perf}
A replication system based on a single leader can provide strong consistency guarantees that are
difficult or impossible to achieve in a leaderless system. However, as we have seen in
@ -1750,7 +1750,7 @@ machine to store only a subset of the data.
### References {#references}
### References
[^1]: B. G. Lindsay, P. G. Selinger, C. Galtieri, J. N. Gray, R. A. Lorie, T. G. Price, F. Putzolu, I. L. Traiger, and B. W. Wade. [Notes on Distributed Databases](https://dominoweb.draco.res.ibm.com/reports/RJ2571.pdf). IBM Research, Research Report RJ2571(33471), July 1979. Archived at [perma.cc/EPZ3-MHDD](https://perma.cc/EPZ3-MHDD)

View File

@ -64,7 +64,7 @@ the network between nodes. We will discuss such faults in [Chapter 9](/en/ch9#c
--------
## Pros and Cons of Sharding {#pros-and-cons-of-sharding}
## Pros and Cons of Sharding {#sec_sharding_reasons}
The primary reason for sharding a database is *scalability*: its a solution if the volume of data
or the write throughput has become too great for a single node to handle, as it allows you to spread
@ -108,7 +108,7 @@ access* (NUMA) architecture in which some banks of memory are closer to one CPU
For example, Redis, VoltDB, and FoundationDB use one process per core, and rely on sharding to
spread load across CPU cores in the same machine [^6].
### Sharding for Multitenancy {#sharding-for-multitenancy}
### Sharding for Multitenancy {#sec_sharding_multitenancy}
Software as a Service (SaaS) products and cloud services are often *multitenant*, where each tenant
is a customer. Multiple users may have logins on the same tenant, but each tenant has a
@ -171,7 +171,7 @@ The main challenges around using sharding for multitenancy are:
## Sharding of Key-Value Data {#sharding-of-key-value-data}
## Sharding of Key-Value Data {#sec_sharding_key_value}
Say you have a large amount of data, and you want to shard it. How do you decide which records to
store on which nodes?
@ -195,7 +195,7 @@ necessarily its primary key). That algorithm needs to be amenable to rebalancing
hot spots.
### Sharding by Key Range {#sharding-by-key-range}
### Sharding by Key Range {#sec_sharding_key_range}
One way of sharding is to assign a contiguous range of partition keys (from some minimum to some
maximum) to each shard, like the volumes of a paper encyclopedia, as illustrated in
@ -267,7 +267,7 @@ all of its data to be rewritten into new files, similarly to a compaction in a l
storage engine. A shard that needs splitting is often also one that is under high load, and the cost
of splitting can exacerbate that load, risking it becoming overloaded.
### Sharding by Hash of Key {#sharding-by-hash-of-key}
### Sharding by Hash of Key {#sec_sharding_hash}
Key-range sharding is useful if you want records with nearby (but different) partition keys to be
grouped into the same shard; for example, this might be the case with timestamps. If you dont care
@ -592,7 +592,7 @@ search engines such as Solr and Elasticsearch. The problem with secondary indexe
map neatly to shards. There are two main approaches to sharding a database with secondary indexes:
local and global indexes.
### Local Secondary Indexes {#local-secondary-indexes}
### Local Secondary Indexes {#id166}
For example, imagine you are operating a website for selling used cars (illustrated in
[Figure 7-9](/en/ch7#fig_sharding_local_secondary)). Each listing has a unique ID, and you use that ID as partition
@ -642,7 +642,7 @@ process every query anyway.
Nevertheless, local secondary indexes are widely used [^31]: for example, MongoDB, Riak, Cassandra [^32], Elasticsearch [^33],
SolrCloud, and VoltDB [^34] all use local secondary indexes.
### Global Secondary Indexes {#global-secondary-indexes}
### Global Secondary Indexes {#id167}
Rather than each shard having its own, local secondary index, we can construct a *global index* that
covers data in all shards. However, we cant just store that index on one node, since it would
@ -744,7 +744,7 @@ that question in the following chapters.
### References {#references}
### References
[^1]: Claire Giordano. [Understanding partitioning and sharding in Postgres and Citus](https://www.citusdata.com/blog/2023/08/04/understanding-partitioning-and-sharding-in-postgres-and-citus/). *citusdata.com*, August 2023. Archived at [perma.cc/8BTK-8959](https://perma.cc/8BTK-8959)
[^2]: Brandur Leach. [Partitioning in Postgres, 2022 edition](https://brandur.org/fragments/postgres-partitioning-2022). *brandur.org*, October 2022. Archived at [perma.cc/Z5LE-6AKX](https://perma.cc/Z5LE-6AKX)

View File

@ -63,7 +63,7 @@ Concurrency control is relevant for both single-node and distributed databases.
chapter, in [“Distributed Transactions”](/en/ch8#sec_transactions_distributed), we will examine the *two-phase commit* protocol and
the challenge of achieving atomicity in a distributed transaction.
## What Exactly Is a Transaction? {#what-exactly-is-a-transaction}
## What Exactly Is a Transaction? {#sec_transactions_overview}
Almost all relational databases today, and some nonrelational databases, support transactions. Most
of them follow the style that was introduced in 1975 by IBM System R, the first SQL database [^2] [^3] [^4].
@ -91,7 +91,7 @@ technical design choice, transactions have advantages and limitations. In order
trade-offs, lets go into the details of the guarantees that transactions can provide—both in normal
operation and in various extreme (but realistic) circumstances.
## #The Meaning of ACID {#the-meaning-of-acid}
### The Meaning of ACID {#sec_transactions_acid}
The safety guarantees provided by transactions are often described by the well-known acronym *ACID*,
which stands for *Atomicity*, *Consistency*, *Isolation*, and *Durability*. It was coined in 1983 by
@ -272,7 +272,7 @@ backups—and they can and should be used together. As always, its wise to
--------
### Single-Object and Multi-Object Operations {#single-object-and-multi-object-operations}
### Single-Object and Multi-Object Operations {#sec_transactions_multi_object}
To recap, in ACID, atomicity and isolation describe what the database should do if a client makes
several writes within the same transaction:
@ -1911,7 +1911,7 @@ cannot guarantee atomicity.
A better solution in practice is to replace the single-node coordinator with a fault-tolerant
consensus protocol. We will see how to do this in [Chapter 10](/en/ch10#ch_consistency).
### Distributed Transactions Across Different Systems {#distributed-transactions-across-different-systems}
### Distributed Transactions Across Different Systems {#sec_transactions_xa}
Distributed transactions and two-phase commit have a mixed reputation. On the one hand, they are
seen as providing an important safety guarantee that would be hard to achieve otherwise; on the
@ -2086,7 +2086,7 @@ However, keeping several heterogeneous data systems consistent with each other i
important problem, so we need to find a different solution to it. This can be done, as we will see
in the next section and in [Link to Come].
### Database-internal Distributed Transactions {#database-internal-distributed-transactions}
### Database-internal Distributed Transactions {#sec_transactions_internal}
As explained previously, there is a big difference between distributed transactions that span
multiple heterogeneous storage technologies, and those that are internal to a system—i.e., where all
@ -2260,7 +2260,7 @@ The examples in this chapter used a relational data model. However, as discussed
### References {#references}
### References
[^1]: Steven J. Murdoch. [What went wrong with Horizon: learning from the Post Office Trial](https://www.benthamsgaze.org/2021/07/15/what-went-wrong-with-horizon-learning-from-the-post-office-trial/). *benthamsgaze.org*, July 2021. Archived at [perma.cc/CNM4-553F](https://perma.cc/CNM4-553F)
[^2]: Donald D. Chamberlin, Morton M. Astrahan, Michael W. Blasgen, James N. Gray, W. Frank King, Bruce G. Lindsay, Raymond Lorie, James W. Mehl, Thomas G. Price, Franco Putzolu, Patricia Griffiths Selinger, Mario Schkolnick, Donald R. Slutz, Irving L. Traiger, Bradford W. Wade, and Robert A. Yost. [A History and Evaluation of System R](https://dsf.berkeley.edu/cs262/2005/SystemR.pdf). *Communications of the ACM*, volume 24, issue 10, pages 632646, October 1981. [doi:10.1145/358769.358784](https://doi.org/10.1145/358769.358784)

View File

@ -34,7 +34,7 @@ explore how to think about the state of a distributed system and how to reason a
have happened ([“Knowledge, Truth, and Lies”](/en/ch9#sec_distributed_truth)). Later, in [Chapter 10](/en/ch10#ch_consistency), we will look at some
examples of how we can achieve fault tolerance in the face of those faults.
## Faults and Partial Failures {#faults-and-partial-failures}
## Faults and Partial Failures {#sec_distributed_partial_failure}
When you are writing a program on a single computer, it normally behaves in a fairly predictable
way: either it works or it doesnt. Buggy software may give the appearance that the computer is
@ -129,7 +129,7 @@ the response is not going to arrive. However, when a timeout occurs, you still d
the remote node got your request or not (and if the request is still queued somewhere, it may still
be delivered to the recipient, even if the sender has given up on it).
### The Limitations of TCP {#the-limitations-of-tcp}
### The Limitations of TCP {#sec_distributed_tcp}
Network packets have a maximum size (generally a few kilobytes), but many applications need to send
messages (requests, responses) that are too big to fit in one packet. These applications most often
@ -238,7 +238,7 @@ and ensure that the system can recover from them.
It may make sense to deliberately trigger network problems and test the systems response (this is
known as *fault injection*; see [“Fault injection”](/en/ch9#sec_fault_injection)).
### Detecting Faults {#detecting-faults}
### Detecting Faults {#id307}
Many systems need to automatically detect faulty nodes. For example:
@ -377,7 +377,7 @@ observed response time distribution. The Phi Accrual failure detector [^32],
which is used for example in Akka and Cassandra [^33]
is one way of doing this. TCP retransmission timeouts also work similarly [^5].
### Synchronous Versus Asynchronous Networks {#synchronous-versus-asynchronous-networks}
### Synchronous Versus Asynchronous Networks {#sec_distributed_sync_networks}
Distributed systems would be a lot simpler if we could rely on the network to deliver packets with
some fixed maximum delay, and not to drop packets. Why cant we solve this at the hardware level
@ -519,7 +519,7 @@ synchronize clocks to some degree: the most commonly used mechanism is the Netwo
allows the computer clock to be adjusted according to the time reported by a group of servers [^39].
The servers in turn get their time from a more accurate time source, such as a GPS receiver.
### Monotonic Versus Time-of-Day Clocks {#monotonic-versus-time-of-day-clocks}
### Monotonic Versus Time-of-Day Clocks {#sec_distributed_monotonic_timeofday}
Modern computers have at least two different kinds of clocks: a *time-of-day clock* and a *monotonic
clock*. Although they both measure time, it is important to distinguish the two, since they serve
@ -580,7 +580,7 @@ In a distributed system, using a monotonic clock for measuring elapsed time (e.g
usually fine, because it doesnt assume any synchronization between different nodes clocks and is
not sensitive to slight inaccuracies of measurement.
### Clock Synchronization and Accuracy {#clock-synchronization-and-accuracy}
### Clock Synchronization and Accuracy {#sec_distributed_clock_accuracy}
Monotonic clocks dont need synchronization, but time-of-day clocks need to be set according to an
NTP server or other external time source in order to be useful. Unfortunately, our methods for
@ -1033,7 +1033,7 @@ we can make and the guarantees we may want to provide. In [Chapter 10](/en/ch10
look at some examples of distributed algorithms that provide particular guarantees under particular
assumptions.
### The Majority Rules {#the-majority-rules}
### The Majority Rules {#sec_distributed_majority}
Imagine a network with an asymmetric fault: a node is able to receive all messages sent to it, but
any outgoing messages from that node are dropped or delayed [^22]. Even though that node is working
@ -1483,7 +1483,7 @@ They are incredibly helpful for distilling down the complexity of real systems t
of faults that we can reason about, so that we can understand the problem and try to solve it
systematically.
### Formal Methods and Randomized Testing {#formal-methods-and-randomized-testing}
### Formal Methods and Randomized Testing {#sec_distributed_formal}
How do we know that an algorithm satisfies the required properties? Due to concurrency, partial
failures, and network delays there are a huge number of potential states. We need to guarantee
@ -1702,7 +1702,7 @@ problems in distributed systems.
### References {#references}
### References
[^1]: Mark Cavage. [Theres Just No Getting Around It: Youre Building a Distributed System](https://queue.acm.org/detail.cfm?id=2482856). *ACM Queue*, volume 11, issue 4, pages 80-89, April 2013. [doi:10.1145/2466486.2482856](https://doi.org/10.1145/2466486.2482856)
[^2]: Jay Kreps. [Getting Real About Distributed System Reliability](https://blog.empathybox.com/post/19574936361/getting-real-about-distributed-system-reliability). *blog.empathybox.com*, March 2012. Archived at [perma.cc/9B5Q-AEBW](https://perma.cc/9B5Q-AEBW)

View File

@ -6,30 +6,214 @@ breadcrumbs: false
---
### [Preface](/en/preface)
### [Part I: Foundations of Data Systems](/en/part-i)
- [1. Tradeoffs in Data Systems Architecture](/en/ch1)
- [2. Defining NonFunctional Requirements](/en/ch2)
- [3. Data Models and Query Languages](/en/ch3)
- [4. Storage and Retrieval](/en/ch4)
- [5. Encoding and Evolution](/en/ch5)
### [Part II: Distributed Data](/en/part-ii)
- [6. Replication](/en/ch6)
- [7. Partitioning](/en/ch7)
- [8. Transactions](/en/ch8)
- [9. The Trouble with Distributed Systems](/en/ch9)
- [10. Consistency and Consensus](/en/ch10)
### [Part III: Derived Data](/en/part-iii)
- [11. Batch Processing](/en/ch11) (WIP)
- [12. Stream Processing](/en/ch12) (WIP)
- [13. Doing the Right Thing](/en/ch13) (WIP)
### [Glossary](/en/glossary)
### [Colophon](/en/colophon)
![](/title.jpg)
![](/title.jpg)
## [Preface](/en/preface)
## [1. Trade-offs in Data Systems Architecture](/en/ch1)
- [Analytical versus Operational Systems](/en/ch1#sec_introduction_analytics)
- [Characterizing Transaction Processing and Analytics](/en/ch1#sec_introduction_oltp)
- [Data Warehousing](/en/ch1#sec_introduction_dwh)
- [Systems of Record and Derived Data](/en/ch1#sec_introduction_derived)
- [Cloud versus Self-Hosting](/en/ch1#sec_introduction_cloud)
- [Pros and Cons of Cloud Services](/en/ch1#sec_introduction_cloud_tradeoffs)
- [Cloud-Native System Architecture](/en/ch1#sec_introduction_cloud_native)
- [Operations in the Cloud Era](/en/ch1#sec_introduction_operations)
- [Distributed versus Single-Node Systems](/en/ch1#sec_introduction_distributed)
- [Problems with Distributed Systems](/en/ch1#sec_introduction_dist_sys_problems)
- [Microservices and Serverless](/en/ch1#sec_introduction_microservices)
- [Cloud Computing versus Supercomputing](/en/ch1#id17)
- [Data Systems, Law, and Society](/en/ch1#sec_introduction_compliance)
- [Summary](/en/ch1#summary)
## [2. Defining Nonfunctional Requirements](/en/ch2)
- [Case Study: Social Network Home Timelines](/en/ch2#sec_introduction_twitter)
- [Representing Users, Posts, and Follows](/en/ch2#id20)
- [Materializing and Updating Timelines](/en/ch2#sec_introduction_materializing)
- [Describing Performance](/en/ch2#sec_introduction_percentiles)
- [Latency and Response Time](/en/ch2#id23)
- [Average, Median, and Percentiles](/en/ch2#id24)
- [Use of Response Time Metrics](/en/ch2#sec_introduction_slo_sla)
- [Reliability and Fault Tolerance](/en/ch2#sec_introduction_reliability)
- [Fault Tolerance](/en/ch2#id27)
- [Hardware and Software Faults](/en/ch2#sec_introduction_hardware_faults)
- [Humans and Reliability](/en/ch2#id31)
- [Scalability](/en/ch2#sec_introduction_scalability)
- [Describing Load](/en/ch2#id33)
- [Shared-Memory, Shared-Disk, and Shared-Nothing Architecture](/en/ch2#sec_introduction_shared_nothing)
- [Principles for Scalability](/en/ch2#id35)
- [Maintainability](/en/ch2#sec_introduction_maintainability)
- [Operability: Making Life Easy for Operations](/en/ch2#id37)
- [Simplicity: Managing Complexity](/en/ch2#id38)
- [Evolvability: Making Change Easy](/en/ch2#sec_introduction_evolvability)
- [Summary](/en/ch2#summary)
## [3. Data Models and Query Languages](/en/ch3)
- [Relational Model versus Document Model](/en/ch3#sec_datamodels_history)
- [The Object-Relational Mismatch](/en/ch3#sec_datamodels_document)
- [Normalization, Denormalization, and Joins](/en/ch3#sec_datamodels_normalization)
- [Many-to-One and Many-to-Many Relationships](/en/ch3#sec_datamodels_many_to_many)
- [Stars and Snowflakes: Schemas for Analytics](/en/ch3#sec_datamodels_analytics)
- [When to Use Which Model](/en/ch3#sec_datamodels_document_summary)
- [Graph-Like Data Models](/en/ch3#sec_datamodels_graph)
- [Property Graphs](/en/ch3#id56)
- [The Cypher Query Language](/en/ch3#id57)
- [Graph Queries in SQL](/en/ch3#id58)
- [Triple-Stores and SPARQL](/en/ch3#id59)
- [Datalog: Recursive Relational Queries](/en/ch3#id62)
- [GraphQL](/en/ch3#id63)
- [Event Sourcing and CQRS](/en/ch3#sec_datamodels_events)
- [Dataframes, Matrices, and Arrays](/en/ch3#sec_datamodels_dataframes)
- [Summary](/en/ch3#summary)
## [4. Storage and Retrieval](/en/ch4)
- [Storage and Indexing for OLTP](/en/ch4#sec_storage_oltp)
- [Log-Structured Storage](/en/ch4#sec_storage_log_structured)
- [B-Trees](/en/ch4#sec_storage_b_trees)
- [Comparing B-Trees and LSM-Trees](/en/ch4#sec_storage_btree_lsm_comparison)
- [Multi-Column and Secondary Indexes](/en/ch4#sec_storage_index_multicolumn)
- [Keeping everything in memory](/en/ch4#sec_storage_inmemory)
- [Data Storage for Analytics](/en/ch4#sec_storage_analytics)
- [Cloud Data Warehouses](/en/ch4#sec_cloud_data_warehouses)
- [Column-Oriented Storage](/en/ch4#sec_storage_column)
- [Query Execution: Compilation and Vectorization](/en/ch4#sec_storage_vectorized)
- [Materialized Views and Data Cubes](/en/ch4#sec_storage_materialized_views)
- [Multidimensional and Full-Text Indexes](/en/ch4#sec_storage_multidimensional)
- [Full-Text Search](/en/ch4#sec_storage_full_text)
- [Vector Embeddings](/en/ch4#id92)
- [Summary](/en/ch4#summary)
## [5. Encoding and Evolution](/en/ch5)
- [Formats for Encoding Data](/en/ch5#sec_encoding_formats)
- [Language-Specific Formats](/en/ch5#id96)
- [JSON, XML, and Binary Variants](/en/ch5#sec_encoding_json)
- [Protocol Buffers](/en/ch5#sec_encoding_protobuf)
- [Avro](/en/ch5#sec_encoding_avro)
- [The Merits of Schemas](/en/ch5#sec_encoding_schemas)
- [Modes of Dataflow](/en/ch5#sec_encoding_dataflow)
- [Dataflow Through Databases](/en/ch5#sec_encoding_dataflow_db)
- [Dataflow Through Services: REST and RPC](/en/ch5#sec_encoding_dataflow_rpc)
- [Durable Execution and Workflows](/en/ch5#sec_encoding_dataflow_workflows)
- [Event-Driven Architectures](/en/ch5#sec_encoding_dataflow_msg)
- [Summary](/en/ch5#summary)
## [6. Replication](/en/ch6)
- [Single-Leader Replication](/en/ch6#sec_replication_leader)
- [Synchronous Versus Asynchronous Replication](/en/ch6#sec_replication_sync_async)
- [Setting Up New Followers](/en/ch6#sec_replication_new_replica)
- [Handling Node Outages](/en/ch6#sec_replication_failover)
- [Implementation of Replication Logs](/en/ch6#sec_replication_implementation)
- [Problems with Replication Lag](/en/ch6#sec_replication_lag)
- [Reading Your Own Writes](/en/ch6#sec_replication_ryw)
- [Monotonic Reads](/en/ch6#sec_replication_monotonic_reads)
- [Consistent Prefix Reads](/en/ch6#sec_replication_consistent_prefix)
- [Solutions for Replication Lag](/en/ch6#id131)
- [Multi-Leader Replication](/en/ch6#sec_replication_multi_leader)
- [Geographically Distributed Operation](/en/ch6#sec_replication_multi_dc)
- [Sync Engines and Local-First Software](/en/ch6#sec_replication_offline_clients)
- [Dealing with Conflicting Writes](/en/ch6#sec_replication_write_conflicts)
- [CRDTs and Operational Transformation](/en/ch6#sec_replication_crdts)
- [Leaderless Replication](/en/ch6#sec_replication_leaderless)
- [Writing to the Database When a Node Is Down](/en/ch6#id287)
- [Limitations of Quorum Consistency](/en/ch6#sec_replication_quorum_limitations)
- [Single-Leader vs. Leaderless Replication Performance](/en/ch6#sec_replication_leaderless_perf)
- [Detecting Concurrent Writes](/en/ch6#sec_replication_concurrent)
- [Summary](/en/ch6#summary)
## [7. Sharding](/en/ch7)
- [Pros and Cons of Sharding](/en/ch7#sec_sharding_reasons)
- [Sharding for Multitenancy](/en/ch7#sec_sharding_multitenancy)
- [Sharding of Key-Value Data](/en/ch7#sec_sharding_key_value)
- [Sharding by Key Range](/en/ch7#sec_sharding_key_range)
- [Sharding by Hash of Key](/en/ch7#sec_sharding_hash)
- [Skewed Workloads and Relieving Hot Spots](/en/ch7#sec_sharding_skew)
- [Operations: Automatic or Manual Rebalancing](/en/ch7#sec_sharding_operations)
- [Request Routing](/en/ch7#sec_sharding_routing)
- [Sharding and Secondary Indexes](/en/ch7#sec_sharding_secondary_indexes)
- [Local Secondary Indexes](/en/ch7#id166)
- [Global Secondary Indexes](/en/ch7#id167)
- [Summary](/en/ch7#summary)
## [8. Transactions](/en/ch8)
- [What Exactly Is a Transaction?](/en/ch8#sec_transactions_overview)
- [The Meaning of ACID](/en/ch8#sec_transactions_acid)
- [Single-Object and Multi-Object Operations](/en/ch8#sec_transactions_multi_object)
- [Weak Isolation Levels](/en/ch8#sec_transactions_isolation_levels)
- [Read Committed](/en/ch8#sec_transactions_read_committed)
- [Snapshot Isolation and Repeatable Read](/en/ch8#sec_transactions_snapshot_isolation)
- [Preventing Lost Updates](/en/ch8#sec_transactions_lost_update)
- [Write Skew and Phantoms](/en/ch8#sec_transactions_write_skew)
- [Serializability](/en/ch8#sec_transactions_serializability)
- [Actual Serial Execution](/en/ch8#sec_transactions_serial)
- [Two-Phase Locking (2PL)](/en/ch8#sec_transactions_2pl)
- [Serializable Snapshot Isolation (SSI)](/en/ch8#sec_transactions_ssi)
- [Distributed Transactions](/en/ch8#sec_transactions_distributed)
- [Two-Phase Commit (2PC)](/en/ch8#sec_transactions_2pc)
- [Distributed Transactions Across Different Systems](/en/ch8#sec_transactions_xa)
- [Database-internal Distributed Transactions](/en/ch8#sec_transactions_internal)
- [Summary](/en/ch8#summary)
## [9. The Trouble with Distributed Systems](/en/ch9)
- [Faults and Partial Failures](/en/ch9#sec_distributed_partial_failure)
- [Unreliable Networks](/en/ch9#sec_distributed_networks)
- [The Limitations of TCP](/en/ch9#sec_distributed_tcp)
- [Network Faults in Practice](/en/ch9#sec_distributed_network_faults)
- [Detecting Faults](/en/ch9#id307)
- [Timeouts and Unbounded Delays](/en/ch9#sec_distributed_queueing)
- [Synchronous Versus Asynchronous Networks](/en/ch9#sec_distributed_sync_networks)
- [Unreliable Clocks](/en/ch9#sec_distributed_clocks)
- [Monotonic Versus Time-of-Day Clocks](/en/ch9#sec_distributed_monotonic_timeofday)
- [Clock Synchronization and Accuracy](/en/ch9#sec_distributed_clock_accuracy)
- [Relying on Synchronized Clocks](/en/ch9#sec_distributed_clocks_relying)
- [Process Pauses](/en/ch9#sec_distributed_clocks_pauses)
- [Knowledge, Truth, and Lies](/en/ch9#sec_distributed_truth)
- [The Majority Rules](/en/ch9#sec_distributed_majority)
- [Distributed Locks and Leases](/en/ch9#sec_distributed_lock_fencing)
- [Byzantine Faults](/en/ch9#sec_distributed_byzantine)
- [System Model and Reality](/en/ch9#sec_distributed_system_model)
- [Formal Methods and Randomized Testing](/en/ch9#sec_distributed_formal)
- [Summary](/en/ch9#summary)
## [10. Consistency and Consensus](/en/ch10)
- [Linearizability](/en/ch10#sec_consistency_linearizability)
- [What Makes a System Linearizable?](/en/ch10#sec_consistency_lin_definition)
- [Relying on Linearizability](/en/ch10#sec_consistency_linearizability_usage)
- [Implementing Linearizable Systems](/en/ch10#sec_consistency_implementing_linearizable)
- [The Cost of Linearizability](/en/ch10#sec_linearizability_cost)
- [ID Generators and Logical Clocks](/en/ch10#sec_consistency_logical)
- [Logical Clocks](/en/ch10#sec_consistency_timestamps)
- [Linearizable ID Generators](/en/ch10#sec_consistency_linearizable_id)
- [Consensus](/en/ch10#sec_consistency_consensus)
- [The Many Faces of Consensus](/en/ch10#sec_consistency_faces)
- [Consensus in Practice](/en/ch10#sec_consistency_total_order)
- [Coordination Services](/en/ch10#sec_consistency_coordination)
- [Summary](/en/ch10#summary)
## [11. Batch Processing](/en/ch11) (WIP)
## [12. Stream Processing](/en/ch12) (WIP)
## [13. Doing the Right Thing](/en/ch13) (WIP)
## [Glossary](/en/glossary)
## [Colophon](/en/colophon)