by Larry Harris · 2 Jan 2003 · 1,164pp · 309,327 words
to recover from service interruptions. Markets—as well as brokers and dealers—employ many of the following processes to create reliable trading systems: • They use fault-tolerant computer hardware. • They build redundant computer systems. • They build redundant network connections. * * * ▶ Some Examples of the Risks of Trading Through Unreliable Data Networks • A trader
by Irene Aldridge · 1 Dec 2009 · 354pp · 26,550 words
risk management applications ensure that the system stays within prespecified behavioral and P&L bounds. Such applications may also be known as system-monitoring and fault-tolerance software. 26 HIGH-FREQUENCY TRADING r Mobile applications suitable for monitoring performance of highfrequency trading systems alert administration of any issues. r Real-time third
by Eric Redmond, Jim Wilson and Jim R. Wilson · 7 May 2012 · 713pp · 93,944 words
, or XML to images or video clips—all accessible through a simple HTTP interface. Whatever data you have, Riak can store it. Riak is also fault-tolerant. Servers can go up or down at any moment with no single point of failure. Your cluster continues humming along as servers are added, removed
by P. W. Singer and Allan Friedman · 3 Jan 2014 · 587pp · 117,894 words
decades, most major corporations have had business continuity plans for fires or natural disasters, while the electronics industry has measured what it thinks of as fault tolerance, and the communications industry has talked about reliability and redundancy in its operations. All of these fit into the idea of resilience, but most assume
by Max More and Natasha Vita-More · 4 Mar 2013 · 798pp · 240,182 words
(Koene 2011a, 2011b). Your identity, your memories can then be embodied physically in many ways. They can also be backed up and operate robustly on fault-tolerant hardware with redundancy schemes. Achieving substrate-independence will allow us to optimize the operational framework, the hardware, to challenges posed by novel circumstances and different
by Sam Newman · 25 Dec 2014 · 540pp · 103,101 words
may be discouraged from picking alternative stacks if they have to do lots more work themselves. Netflix, for example, is especially concerned with aspects like fault tolerance, to ensure that the outage of one part of its system cannot take everything down. To handle this, a large amount of work has been
…
the duplicated effort, and more about the fact that it is so easy to get this wrong. The risk of a service getting newly implemented fault tolerance wrong is high if it could impact more of the system. Netflix mitigates this by using sidecar services, which communicate locally with a JVM that
by Sam Newman · 14 Nov 2019 · 355pp · 81,788 words
anyway can have some pretty disastrous implications. 7 See Liming Chen and Algirdas Avizienis, “N-Version Programming: A Fault-Tolerance Approach to Reliability of Software Operation,” published in the Twenty-Fifth International Symposium on Fault-Tolerant Computing (1995). Chapter 4. Decomposing the Database As we’ve already explored, there are a host of ways
by Kevin Roose · 9 Mar 2021 · 208pp · 57,602 words
hand against AI. Another type of scarce work that will be hard to automate is work that involves rare or high-stakes situations with low fault tolerance. Most AI learns in an iterative way—that is, it repeats a task over and over again, getting it a little more right each time
by Robert Wright · 1 Jan 1994 · 604pp · 161,455 words
, so no one culture controls the memes (though some try in vain). This decentralization makes epic social setbacks of reliably limited duration; the system is “fault-tolerant,” as computer engineers say. While Europe fell into its slough of despond, Byzantium and southern China stayed standing, India had ups and downs, and the
by Martin Kleppmann · 17 Apr 2017
Streams Keeping Systems in Sync Change Data Capture Event Sourcing State, Streams, and Immutability Processing Streams Uses of Stream Processing Reasoning About Time Stream Joins Fault Tolerance Summary 440 441 446 451 452 454 457 459 464 465 468 472 476 479 12. The Future of Data Systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489 Data Integration
…
indi‐ vidual processes without warning. Many critical bugs are actually due to poor error handling [3]; by deliberately inducing faults, you ensure that the fault-tolerance machinery is continually exercised and tested, which can increase your confidence that faults will be handled correctly when they occur naturally. The Netflix Chaos Monkey
…
flexibility and elasticityi over single-machine reliability. Hence there is a move toward systems that can tolerate the loss of entire machines, by using software fault-tolerance techniques in preference or in addition to hardware redundancy. Such systems also have operational advantages: a single-server system requires planned downtime if you
…
typically random and uncorrelated), software (bugs are typically sys‐ tematic and hard to deal with), and humans (who inevitably make mistakes from time to time). Fault-tolerance techniques can hide certain types of faults from the end user. Scalability means having strategies for keeping performance good, even when load increases. In order
…
and Gone,” at 21st International Conference on Data Engineering (ICDE), April 2005. [2] Walter L. Heimerdinger and Charles B. Weinstock: “A Conceptual Framework for System Fault Tolerance,” Technical Report CMU/SEI-92-TR-033, Software Engi‐ neering Institute, Carnegie Mellon University, October 1992. [3] Ding Yuan, Yu Luo, Xin Zhuang, et
…
followed the path of CODASYL. Relational Versus Document Databases Today There are many differences to consider when comparing relational databases to document databases, including their fault-tolerance properties (see Chapter 5) and handling of concurrency (see Chapter 7). In this chapter, we will concentrate only on the differences in the data
…
weak forms of “lying”—for example, invalid messages due to hardware issues, software bugs, and misconfiguration. Such protec‐ tion mechanisms are not full-blown Byzantine fault tolerance, as they would not withstand a determined adversary, but they are nevertheless simple and pragmatic steps toward better reliability. For example: • Network packets do sometimes
…
a consistent state after a new leader is elected, ensuring that the safety properties are always met. These differences are key to the correctness and fault tolerance of a consensus algorithm. 368 | Chapter 9: Consistency and Consensus Limitations of consensus Consensus algorithms are a huge breakthrough for distributed systems: they bring
…
proceeds in fixed rounds: at every iteration, the framework delivers all messages sent in the previous iteration. Actors normally have no such timing guarantee. Fault tolerance The fact that vertices can only communicate by message passing (not by querying each other directly) helps improve the performance of Pregel jobs, since messages
…
message brokers write all messages to disk, they are able to achieve throughput of millions of messages per second by partitioning across multiple machines, and fault tolerance by replicating messages [22, 23]. Logs compared to traditional messaging The log-based approach trivially supports fan-out messaging, because several con‐ sumers can
…
to the present. The facilities for maintaining state as streams and replaying messages are also the basis for the techniques that enable stream joins and fault tolerance in various stream processing frameworks. We discussed several purposes of stream processing, includ‐ ing searching for event patterns (complex event processing), computing windowed aggregations (
…
to create applications and systems that are reliable, scalable, and maintainable. These themes have run through all of the chapters: for example, we discussed many fault-tolerance algo‐ rithms that help improve reliability, partitioning to improve scalability, and mecha‐ nisms for evolution and abstraction that improve maintainability. In this chapter we will
…
one-directional, asynchronous message streams rather than synchronous request/response interactions. Besides the advantages listed in “Message-Passing Dataflow” on page 136, such as better fault tolerance, dataflow systems can also achieve better performance. For example, say a customer is purchasing an item that is priced in one currency but paid for
…
“Distributed Transactions in Practice” on page 360). When we refuse to use distributed transactions because they are too expensive, we end up having to reimplement fault-tolerance mechanisms in application code. As numerous examples throughout this book have shown, reasoning about concurrency and partial failure is difficult and counterintuitive, and so I
…
without requiring coordination, while still giving strong integrity guarantees. Such coordination-avoiding data systems have a lot of appeal: they can achieve better performance and fault tolerance than systems that need to perform synchronous coordination [56]. For example, such a system could operate distributed across multiple datacenters in a multi-leader configuration
…
writes, 230 auditability, 528-533 designing for, 531 self-auditing systems, 530 through immutability, 460 tools for auditable data systems, 532 availability, 8 (see also fault tolerance) in CAP theorem, 337 in service level agreements (SLAs), 15 Avro (data format), 122-127 code generation, 127 dynamically generated schemas, 126 object container
…
, 361-364 recovery, 363 copy-on-write (B-trees), 82, 242 CORBA (Common Object Request Broker Architecture), 134 correctness, 6 auditability, 528-533 Byzantine fault tolerance, 305, 532 dealing with partial failures, 274 in log-based systems, 521-524 of algorithm within system model, 308 of compensating transactions, 355 of consensus
…
, 512 outputs of batch and stream processing, 495 through application code, 505 versus distributed transactions, 492 deterministic operations, 255, 274, 554 accidental nondeterminism, 423 and fault tolerance, 423, 426 and idempotence, 478, 492 computing derived data, 495, 526, 531 in state machine replication, 349, 452, 458 joins, 476 DevOps, 394 differential
…
introducing deliberately, 7, 280 network faults, 279-281 asymmetric faults, 300 detecting, 280 tolerance of, in multi-leader replication, 169 software errors, 8 tolerating (see fault tolerance) federated databases, 501 fence (CPU instruction), 338 fencing (preventing split brain), 158, 302-304 generating fencing tokens, 349, 370 properties of fencing tokens, 308 stream
…
tags (Thrift and Protocol Buffers), 119-121 file descriptors (Unix), 395 financial data, 460 Firebase (database), 456 Flink (processing framework), 421-423 dataflow APIs, 427 fault tolerance, 422, 477, 479 Gelly API (graph processing), 425 integration of batch and stream processing, 495, 498 machine learning, 428 query optimizer, 427 stream processing, 466
…
Helix (cluster manager), 216 heterogeneous distributed transactions, 360, 364 heuristic decisions (in 2PC), 363 Hibernate (object-relational mapper), 30 hierarchical model, 36 high availability (see fault tolerance) high-frequency trading, 290, 299 high-performance computing (HPC), 275 hinted handoff, 183 histograms, 16 Hive (query engine), 419, 427 for data warehouses, 93 HCatalog
…
in batch processors, 422 JavaScript in MapReduce querying, 46 setting element styles (example), 45 use in advanced queries, 48 Jena (RDF framework), 57 Jepsen (fault tolerance testing), 515 jitter (network delay), 284 joins, 555 by index lookup, 403 expressing as relational operators, 427 in relational and document databases, 34 MapReduce map
…
Juttle (query language), 504 K k-nearest neighbors, 429 Kafka (messaging), 137, 448 Kafka Connect (database integration), 457, 461 Kafka Streams (stream processor), 466, 467 fault tolerance, 479 leader-based replication, 153 log compaction, 456, 467 message offsets, 447, 478 request routing, 216 transaction support, 477 usage example, 4 Ketama (partitioning library
…
also evolvability) from log-based messaging, 451 request routing, 214-216 582 | Index approaches to, 214 parallel query execution, 216 resilient systems, 6 (see also fault tolerance) response time as performance metric for services, 13, 389 guarantees on, 298 latency versus, 14 mean and percentiles, 14 user experience, 15 responsibility and accountability
…
(Datalog), 61 S safety and liveness properties, 308 in consensus algorithms, 366 in transactions, 222 sagas (see compensating transactions) Samza (stream processor), 466, 467 fault tolerance, 479 streaming SQL support, 466 sandboxes, 9 SAP HANA (database), 93 scalability, 10-18, 489 approaches for coping with load, 17 defined, 22 describing load
…
record) Spanner (database) data locality, 41 snapshot isolation using clocks, 295 TrueTime API, 294 Spark (processing framework), 421-423 bytecode generation, 428 dataflow APIs, 427 fault tolerance, 422 for data warehouses, 93 GraphX API (graph processing), 425 machine learning, 428 query optimizer, 427 Spark Streaming, 466 microbatching, 477 stream processing on top
…
job, 474, 477, 478, 517 combining with batch processing lambda architecture, 497 unifying technologies, 498 comparison to batch processing, 464 complex event processing (CEP), 465 fault tolerance, 476-479 atomic commit, 477 idempotence, 478 microbatching and checkpointing, 477 rebuilding state after a failure, 478 for data integration, 494-498 586 | Index maintaining
…
sessions, 229 telemetry (see monitoring) Teradata (database), 93, 200 term-partitioned indexes, 208, 217 termination (consensus), 365 Terrapin (database), 413 Tez (dataflow engine), 421-423 fault tolerance, 422 support by higher-level tools, 427 thrashing (out of memory), 297 threads (concurrency) actor model, 138, 468 (see also message-passing) atomic operations, 223
by Rennay Dorasamy · 2 Dec 2021 · 328pp · 77,877 words
by Robert Wright · 28 Dec 2010
by Steven Levy · 12 Apr 2011 · 666pp · 181,495 words
by Betsy Beyer, Chris Jones, Jennifer Petoff and Niall Richard Murphy · 15 Apr 2016 · 719pp · 181,090 words
by Olivier Cure and Guillaume Blin · 10 Dec 2014
by Martin Kleppmann · 16 Mar 2017 · 1,237pp · 227,370 words
by Marianne Bellotti · 17 Mar 2021 · 232pp · 71,237 words
by Thomas A. Limoncelli, Strata R. Chalup and Christina J. Hogan · 27 Aug 2014 · 757pp · 193,541 words
by Saša Jurić · 30 Jan 2019
by David Birch · 14 Jun 2017 · 275pp · 84,980 words
by Robert Daigneau · 14 Sep 2011
by Michal Zalewski · 26 Nov 2011 · 570pp · 115,722 words
by Jan Kunigk, Ian Buss, Paul Wilkinson and Lars George · 8 Jan 2019 · 1,409pp · 205,237 words
by Imran Bashir · 28 Mar 2018
by Trey Grainger and Timothy Potter · 14 Sep 2014 · 1,085pp · 219,144 words
by Haym Benaroya · 12 Jan 2018 · 571pp · 124,448 words
by Casey Rosenthal and Nora Jones · 27 Apr 2020 · 419pp · 102,488 words
by Eswar S. Prasad · 27 Sep 2021 · 661pp · 185,701 words
by Will Larson · 19 May 2019 · 227pp · 63,186 words
by G. Pascal Zachary · 1 Apr 2014 · 384pp · 109,125 words
by Andy Oram · 26 Feb 2001 · 673pp · 164,804 words
by Unknown
by Michael Schrenk · 19 Aug 2009 · 371pp · 78,103 words
by George Dyson · 6 Mar 2012
by Robin Hanson · 31 Mar 2016 · 589pp · 147,053 words
by Robin Sharp · 13 Feb 2008
by Sendhil Mullainathan · 3 Sep 2014 · 305pp · 89,103 words
by Diomidis Spinellis and Georgios Gousios · 30 Dec 2008 · 680pp · 157,865 words
by Douglas Edwards · 11 Jul 2011 · 496pp · 154,363 words
by Peter Gutmann
by David N. Blank-Edelman · 16 Sep 2018
by Peter Van-Roy and Seif Haridi · 15 Feb 2004 · 931pp · 79,142 words
by Francesco Cesarini · 496pp · 70,263 words
by Andy Greenberg · 12 Sep 2012 · 461pp · 125,845 words
by Martin L. Abbott and Michael T. Fisher · 1 Dec 2009
by Bruce Schneier · 3 Sep 2018 · 448pp · 117,325 words
by Steve McConnell · 8 Jun 2004 · 1,758pp · 342,766 words
by Bruce Schneier · 2 Mar 2015 · 598pp · 134,339 words
by Tom Chivers · 6 May 2024 · 283pp · 102,484 words
by Lorne Lantz and Daniel Cawrey · 8 Dec 2020 · 434pp · 77,974 words
by Timothy Ferriss · 6 Dec 2016 · 669pp · 210,153 words
by Tom White · 29 May 2009 · 933pp · 205,691 words
by Amy Brown and Greg Wilson · 24 May 2011 · 834pp · 180,700 words
by Jamie L. Mitchell and Rex Black · 15 Feb 2015
by Bruce Schneier · 1 Jan 2000 · 470pp · 144,455 words
by Mehmed Kantardzić · 2 Jan 2003 · 721pp · 197,134 words
by John MacCormick and Chris Bishop · 27 Dec 2011 · 250pp · 73,574 words
by Bruce Schneier · 10 Nov 1993
by Vaughn Vernon · 16 Aug 2015
by Lars George · 29 Aug 2011
by Alasdair Gilchrist · 27 Jun 2016
by Nilanjan Raychaudhuri · 27 Mar 2012
by Paul Scharre · 23 Apr 2018 · 590pp · 152,595 words
by Eric Topol · 1 Jan 2019 · 424pp · 114,905 words
by Toby Segaran and Jeff Hammerbacher · 1 Jul 2009
by George Dyson · 28 Mar 2012 · 463pp · 118,936 words
by Cathy O'Neil and Rachel Schutt · 8 Oct 2013 · 523pp · 112,185 words
by Cal Newport · 2 Mar 2021 · 350pp · 90,898 words
by Bjarne Stroustrup · 2 Jan 1986 · 923pp · 516,602 words
by Tom Kleenex and Joshua Suereth · 2 Jan 2010 · 554pp · 108,035 words
by George Gilder · 16 Jul 2018 · 332pp · 93,672 words
by Jason Fried and David Heinemeier Hansson · 9 Mar 2010 · 102pp · 27,769 words
by Charles Murray and Catherine Bly Cox · 1 Jan 1989 · 619pp · 197,256 words
by Katie Hafner and Matthew Lyon · 1 Jan 1996 · 352pp · 96,532 words
by David A. Mindell · 3 Apr 2008 · 377pp · 21,687 words
by Campbell R. Harvey, Ashwin Ramachandran, Joey Santoro, Vitalik Buterin and Fred Ehrsam · 23 Aug 2021 · 179pp · 42,081 words
by Don Tapscott and Anthony D. Williams · 28 Sep 2010 · 552pp · 168,518 words
by William Mougayar · 25 Apr 2016 · 161pp · 44,488 words
by Nick Harkaway · 18 Oct 2017 · 778pp · 239,744 words
by John Ousterhout · 28 Jan 2018 · 211pp · 58,677 words
by Harihara Subramanian · 31 Jan 2019 · 422pp · 86,414 words
by Michal Zalewski · 4 Apr 2005 · 412pp · 104,864 words
by Ron Jeffries · 14 Aug 2015 · 444pp · 118,393 words
by M. Mitchell Waldrop · 14 Apr 2001
by Heather Adkins, Betsy Beyer, Paul Blankinship, Ana Oprea, Piotr Lewandowski and Adam Stubblefield · 29 Mar 2020 · 1,380pp · 190,710 words
by Neal Stephenson · 19 May 2015 · 945pp · 292,893 words
by Peter Seibel · 22 Jun 2009 · 1,201pp · 233,519 words
by Brian Christian and Tom Griffiths · 4 Apr 2016 · 523pp · 143,139 words
by Matthew A. Russell · 15 Jan 2011 · 541pp · 109,698 words
by Paul Vigna and Michael J. Casey · 27 Feb 2018 · 348pp · 97,277 words
by Kai-Fu Lee and Qiufan Chen · 13 Sep 2021
by Zigurd Mednieks, Laird Dornin, G. Blake Meike and Masumi Nakamura · 15 Jul 2011
by Pieter Hintjens · 12 Mar 2013 · 1,025pp · 150,187 words
by Ian Robinson, Jim Webber and Emil Eifrem · 13 Jun 2013 · 201pp · 63,192 words
by Bruce C. Greenwald · 31 Aug 2016 · 482pp · 125,973 words
by Drew Neil
by Drew Neil
by Drew Neil · 6 Oct 2012 · 722pp · 90,903 words
by Martin Campbell-Kelly and Nathan Ensmenger · 29 Jul 2013 · 528pp · 146,459 words
by Yasha Levine · 6 Feb 2018 · 474pp · 130,575 words
by Jeff Geerling · 9 Oct 2015 · 313pp · 75,583 words
by Rafal Kuc and Marek Rogozinski · 14 Aug 2013 · 480pp · 99,288 words
by Andy Oram and John Viega · 15 Dec 2009 · 302pp · 82,233 words
by Ben Mezrich · 20 May 2019 · 304pp · 91,566 words
by Susan Fowler · 18 Feb 2020 · 205pp · 71,872 words
by Clement Nedelcu · 18 Jul 2013 · 319pp · 72,969 words
by Stewart Brand · 1 Jan 1999 · 194pp · 49,310 words
by Unknown · 13 Jan 2012 · 470pp · 109,589 words
by Andreas Herrmann, Walter Brenner and Rupert Stadler · 25 Mar 2018
by Chris Skinner · 27 Aug 2013 · 329pp · 95,309 words
by Joseph Menn · 26 Jan 2010 · 362pp · 86,195 words
by Unknown · 2 Jan 2010 · 448pp · 71,301 words
by Michael Nielsen · 2 Oct 2011 · 400pp · 94,847 words
by Tim Berners-Lee · 8 Sep 2025 · 347pp · 100,038 words
by W. Curtis Preston · 9 Feb 2009 · 1,266pp · 278,632 words
by Dan Conway · 8 Sep 2019 · 218pp · 68,648 words
by Eric Berger · 23 Sep 2024 · 375pp · 113,230 words
by Eric Ries · 15 Mar 2017 · 406pp · 105,602 words
by Andreas M. Antonopoulos and Gavin Wood Ph. D. · 23 Dec 2018 · 960pp · 125,049 words
by Miran Lipovaca · 17 Apr 2011 · 559pp · 130,949 words
by Rob Kitchin,Tracey P. Lauriault,Gavin McArdle · 2 Aug 2017
by Stross, Charles · 1 Jan 2011
by Gourav Shah · 29 Jul 2015 · 178pp · 33,275 words
by Niall O’Higgins · 66pp · 9,247 words
by Hugh McKee · 5 Sep 2016 · 31pp · 9,168 words
by Frank J. Ohlhorst · 28 Nov 2012 · 133pp · 42,254 words
by Melanie Swan · 22 Jan 2014 · 271pp · 52,814 words
by James Turnbull · 13 Jul 2014 · 265pp · 60,880 words
by Pete Warden · 20 Sep 2011 · 58pp · 12,386 words
by Zachary Kessin · 9 May 2011 · 210pp · 42,271 words
by Ben Grynhaus, Jordan Hudgens, Rayon Hunte, Matthew Thomas Morgan and Wekoslav Stefanovski · 28 Jul 2021 · 739pp · 174,990 words
by David G. W. Birch · 14 Apr 2020 · 247pp · 60,543 words
by David Smiley and Eric Pugh · 15 Nov 2009 · 648pp · 108,814 words
by Satya Nadella, Greg Shaw and Jill Tracie Nichols · 25 Sep 2017 · 391pp · 71,600 words
by James Higginbotham · 20 Dec 2021 · 283pp · 78,705 words
by Matthew Hindman · 24 Sep 2018
by Chet Haase · 12 Aug 2021 · 580pp · 125,129 words
by Ernest Cline · 15 Feb 2011 · 458pp · 137,960 words
by Chitij Chauhan · 30 Jan 2015
by Rob Manning and William L. Simon · 20 Oct 2014 · 237pp · 76,486 words
by Leslie Sikos · 10 Jul 2015