Machine Translation
by Thierry Poibeau · 14 Sep 2017 · 174pp · 56,405 words
the Modern Corporation, James W. Cortada Intellectual Property Strategy, John Palfrey The Internet of Things, Samuel Greengard Machine Learning: The New AI, Ethem Alpaydin Machine Translation, Thierry Poibeau Memes in Digital Culture, Limor Shifman Metadata, Jeffrey Pomerantz The Mind–Body Problem, Jonathan Westphal MOOCs, Jonathan Haber Neuroplasticity, Moheb Costandi Open
…
Self-Tracking, Gina Neff and Dawn Nafus Sustainability, Kent E. Portney The Technological Singularity, Murray Shanahan Understanding Beliefs, Nils J. Nilsson Waves, Frederic Raichlen Machine Translation Thierry Poibeau The MIT Press Cambridge, Massachusetts London, England © 2017 Massachusetts Institute of Technology All rights reserved. No part of this book may be reproduced
…
Report and Its Consequences 7 Parallel Corpora and Sentence Alignment 8 Example-Based Machine Translation 9 Statistical Machine Translation and Word Alignment 10 Segment-Based Machine Translation 11 Challenges and Limitations of Statistical Machine Translation 12 Deep Learning Machine Translation 13 The Evaluation of Machine Translation Systems 14 The Machine Translation Industry: Between Professional and Mass-Market Applications 15 Conclusion: The Future of
…
Machine Translation Glossary Bibliography and Further Reading Index About Author List of Tables Table 1 Example of possible
…
from the analysis of knowledge and reasoning, which explains the interest shown by philosophers and specialists of artificial intelligence as well as cognitive sciences in [machine translation]. Machine translation involves different processes that make it at least as challenging as developing an automatic dialoguing system. The degree of “understanding” shown by the machine
…
inferred information, which is of course highly challenging and simply goes beyond the current state of the art. The Revolution of Statistical Machine Translation Systems The classification of machine translation systems provided in the previous section is challenged by new approaches that have appeared since the early 1990s. The availability of huge quantities
…
in such symbols could be translated by all who possessed the dictionary” (Descartes, letter to Mersenne on November 20, 1629). This passage greatly inspired machine translation pioneers, since Descartes’ proposal aimed to replace words with unambiguous codes (“symbols” corresponding to numerical codes that are independent from the languages considered; symbols
…
in automatic translation systems. Although these projects resulted in relatively advanced proposals with vocabularies and grammar systems, they have rarely been actively used for machine translation. Esperanto was used during the 1980s in the European Distributed Translation Language project and within the Fujitsu company in Japan, but these two projects
…
it must therefore be translated; in other words, decoded in the target language). Beginning in 1947, Weaver corresponded with the cyberneticist Norbert Wiener concerning machine translation. He proposed that translation could be considered a “decoding” problem: One naturally wonders if the problem of translation could conceivably be treated as a problem
…
more powerful than symbolic rules to resolve ambiguities). The implementation of the proposed techniques, however, required efforts that went beyond anything the pioneers of machine translation had ever imagined. In particular, the inherent ambiguity of natural languages showed that traditional encryption models were not sufficient to render the complexity of automatic
…
-quality translation in the short or medium term (FAHQT, or fully automated high-quality translation; also found as FAHQMT, or fully automated high-quality machine translation). Instead of automatic translation, Bar-Hillel recommended that researchers turn toward computer-assisted translation systems, which constitute a relatively different project, clearly less exciting scientifically
…
the other hand, with the increasing amount of translations available on the Internet, it is now possible to directly design statistical models for machine translation. This approach, known as statistical machine translation, is the most popular today. Unlike a translation memory, which can be relatively small, automatic processing presumes the availability of an
…
it would be more convenient to directly use fragments of translation that one can find in existing bilingual corpora. An Overview of Example-Based Machine Translation Example-based machine translation typically operates in three stages to translate a given sentence: The system tries to find fragments of the sentence to be translated in the
…
sparsity (it is very difficult to collect enough relevant examples at phrase level). Appeal and Limitations of Example-Based Machine Translation Example-based machine translation generated great interest during the 1980s. Rather than developing a machine translation system manually, which is long and very costly, the example-based approach allowed for optimal exploitation of large
…
information of a syntactic and semantic nature has been progressively integrated into models to compensate for the limitation of purely statistical approaches. Toward Segment-Based Machine Translation The IBM models have been subjected to numerous enhancements. The most significant improvement was to take into consideration the notion of segments (or sequences
…
have described in this section have, however, helped improve the IBM models and can still be considered currently as the state of the art in machine translation. Introduction of Linguistic Information into Statistical Models Statistical translation models, despite their increasing complexity to better fit language specificities, have not solved all the
…
techniques. Lastly, for rare languages with too few data to make it possible to develop statistical systems, rule-based systems remain the norm. Hybrid Machine Translation Systems Following the success of statistical translation systems, the majority of traditional systems (based on large lexicons and transfer rules) gradually tried to incorporate statistical
…
deep learning provides an interesting approach that seems especially fitted for the challenges involved in improving human language processing. An Overview of Deep Learning for Machine Translation Deep learning achieved its first success in image recognition. Rather than using a group of predefined characteristics, deep learning generally operates from a very
…
to let the system infer by itself the best representation from the data. A translation system based solely on deep learning (aka “deep learning machine translation” or “neural machine translation”) thus simply consists of an “encoder” (the part of the system that analyzes the training data) and a “decoder” (the part of the
…
toward the resolution of such problems, hence the great success of this technique among researchers in the domain. Current Challenges for Deep Learning Machine Translation Until recently, machine translation systems based on deep learning performed well on simple sentences but still lagged behind traditional statistical systems for more complex sentences. There were different
…
of the internal model calculated by the neural network, so as to better understand how the whole approach works. The deep learning approach to machine translation (or neural machine translation) has proven efficient, first, on short sentences in closely related languages, and more recently on long sentences as well as more diverse languages.
…
initiated research in this area. A 1994 article (White et al., 1994) reviewed the first attempts at evaluation from the beginnings of research on machine translation. The article specifically reported the various possible strategies and their limits, described below. Comprehension Evaluation To assess comprehension, professional human translators first translated English newspaper
…
articles into different languages. Machine translation systems then translated the text back into English, and human analysts answered “multiple choice questions about the content of the articles” to evaluate the automatic
…
Fluency After the previous attempts involving human experts, DARPA then resorted to two evaluation scores: adequacy and fluency. As White and colleagues described of this machine translation (MT) evaluation method: “In an adequacy evaluation, literate, monolingual English speakers make judgments determining the degree to which the information in a professional translation
…
automatically. It is, rather, the capacity of the system to provide relevant translational elements that should be evaluated. 14 The Machine Translation Industry: Between Professional and Mass-Market Applications Machine translation is a popular application because it answers a very direct and simple need. Everybody can clearly see the importance of a system
…
analysis and translation systems used by Samsung’s connected devices (cell phones, tablets, and other technological gadgets). Facebook bought out different companies specialized in machine translation (such as Jibbigo in 2013 for voice messages in particular). Apple and Google are also regularly buying startups in the communication and information technology domains
…
is actually twofold. First, improving the productivity of translators: this involves efficient systems and strategies to make the best of the output of machine translation tools. Second, improving machine translation systems directly: this means being able to dynamically reuse end-user feedback to make the system evolve and propose more accurate translations in
…
Warren Weaver and the launching of MT: Brief biographical note”) and Y. Bar-Hillel (“Yehoshua Bar-Hillel: A philosopher’s contribution to machine translation”), both in Early Years in Machine Translation (see the full reference at the beginning of this chapter). Chapter 6: The 1966 ALPAC Report and Its Consequences The ALPAC report and
…
previous website (http://www.statmt.org) is probably the best source of information for recent trends related to statistical machine translation, of which segment-based machine translation is part. Chapter 11: Challenges and Limitations of Statistical Machine Translation See http://www.statmt.org,as for chapter 10 above. Kenneth Church (2011). “A pendulum swung too
…
Journal of Translation Studies 13 (1–2): 29–70. Special issue: The teaching of computer-aided translation, ed. Chan Sin Wai. Philipp Koehn (2009). Statistical Machine Translation. Cambridge: Cambridge University Press. Jorg Tiedemann (2011). Bitext Alignment. San Rafael, CA: Morgan and Claypool Publishers. Dan Jurafsky and James H. Martin (2016). Speech
…
3: The Correspondence. Cambridge: Cambridge University Press. Umberto Eco (1997). The Search for the Perfect Language. Oxford: Wiley. John Hutchins (2004). “Two precursors of machine translation: Artsrouni and Trojanskij.” International Journal of Translation 16 (1): 11–31. Philip P. Wiener (ed., 1951). Leibniz Selections. New York: Simon and Schuster. Yehoshua Bar
…
Human Intelligence (A. Elithorn and R. Banerji, eds.). Elsevier Science Publishers, Amsterdam. Eiichiro Sumita and Hitoshi Iida (1991). “Experiments and prospects of example-based machine translation.” Proceedings of the Twenty-Ninth Conference of the Association for Computational Linguistics, 185–192. Berkeley, CA. Thomas R. Green (1979). “The necessity of syntax markers
…
: Two experiments with artificial languages.” Verbal Learning and Verbal Behavior 18: 481–496. Harold Somers (1999). “Example-based machine translation.” Machine translation 14 (2): 113–157. Nano Gough and Andy Way (2004). “Robust large-scale EBMT with marker-based segmentation.” Proceedings of the Tenth International Conference on
…
Theoretical and Methodological Issues in Machine Translation, 95–104. Baltimore, MD. Peter Brown, John Cocke, Stephen Della Pietra, Vincent Della Pietra, Frederick Jelinek, Robert Mercer, and Paul Roossin (1988). “A
…
, Yoshua Bengio and Aaron Courville (2016). Deep Learning. Cambridge, MA: MIT Press. Yonghui Wu, et al. (2016). “Google's neural machine translation system: Bridging the gap between human and machine translation.” Published online. arXiv:1609.08144. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu (2002). “BLEU: A method for automatic
…
51, 56, 61, 63, 100, 118, 164–168, 215–216, 250 Environment Canada, 87, 223 Error rate, 241. See also Evaluation Errors (in machine translation). See Typology of errors in machine translation Escher, M. C., 20 Esperanto, 28, 42, 44 Estonian, 97, 212, 213 Europarl corpus, 97, 210, 223 Europe, 36, 41, 86, 97
…
58, 60, 85, 179 Logos Corporation, 88 Machine learning, 175, 181, 183, 236. See also Deep learning Machine translation evaluation. See Evaluation Machine translation industry. See Machine translation market Machine translation market, 89, 221–246, 247–251 Machine translation quality. See Evaluation Machine translation systems Apertium, 172 Ariane-78 system, 85 Babelfish, 227, 228 Bing Translation, 33, 36, 194, 226–229,
…
Systran) TAUM Météo (see Météo) Watson, 241 Maintenance applications, 243 Maltese, 212, 213 Manual correction, 138. See also Post-edition Mass-market applications of machine translation. See Machine translation market Mass media, 239 Mathematical model of communication. See Model of communication Meaning, 8, 15, 17–21, 34, 52–55, 64–67, 70–71
…
, Igor, 69 Memorandum, Warren Weaver’s, 50, 52–59 Mercer, Robert, 93, 94, 166, 216, 258 Mersel, Jules, 76 Metal. See Machine translation systems Metaphysics, 179 Météo. See Machine translation systems Meteor. See Evaluation measure and test Michigan University, 81 Microsoft, 227–229, 240, 248–250 MIT, 60–62 Mobile application, 229, 232
…
264 System comparison (see Evaluation) maintenance, 243 quality (see Evaluation) Systran, 85–89, 171, 194, 223, 226, 227–229, 231–236 Systranet. See Machine translation systems TAUM Météo. See Machine translation systems Technical text, 4, 11, 13, 88, 223, 244 Technical translation, 92. See also Technical text Terminology, 92, 101, 119, 226, 228, 244
Machine: A White Space Novel
by Elizabeth Bear · 5 Oct 2020 · 537pp · 146,610 words
or their metabolic equivalent started working. I had to pass by Records on my way, and—with the help of my datapad and some bad machine translation—got them to upload a few ayatanas into my fox. Ox breathers of various anatomies; every little bit of information would help. Even if I
The Means of Prediction: How AI Really Works (And Who Benefits)
by Maximilian Kasy · 15 Jan 2025 · 209pp · 63,332 words
years: Large language models can predict the word that is most likely to come next given a sequence of words typed thus far. Algorithms for machine translation predict the translated version of a sentence based on the sentence its original language; they might for instance predict the English translation based on the
…
human workers on platforms such as MTurk. This is expensive, and this approach can only be scaled up to a certain point. As another example, machine translation involves predicting sentences in a target language from sentences in the original language. Creating these translated sentences is even more expensive than labeling images, since
The Codebreakers: The Comprehensive History of Secret Communication From Ancient Times to the Internet
by David Kahn · 1 Feb 1963 · 1,799pp · 532,462 words
Maya: E. V. Yevreinov, Yu. G. Kosarev, and V. A. Ustinov, three 1961 articles from different Russian sources translated and published as Foreign Developments in Machine Translation and Information Processing, No. 40, by the United States, Department of Commerce, Office of Technical Services, Joint Publications Research Service, No. 10508; and criticism by
…
-Formation and Semantics,” published in Materialy po mate-maticheskoy lingvistika i mashinnomu perevodu, II (Leningrad University, 1963), which has been translated as Foreign Developments in Machine Translation and Information Processing, No. 161, United States, Department of Commerce, Office of Technical Services, Joint Publications Research Service, No. 26209. Dr. Andreyev has proposed an
The Myth of Artificial Intelligence: Why Computers Can't Think the Way We Do
by Erik J. Larson · 5 Apr 2021
labs at MIT, Stanford, and elsewhere were encountering seemingly endless quandaries, difficulties, confusions, and outright failures. The problem, for example, of Fully Automated High-Quality Machine Translation was thought in the 1950s to be solvable, given sufficient research effort and dollars. By the 1960s, government investment in translation dried up, on the
…
achieved general intelligence, and thus be capable of thinking and acting like humans. By the 1960s, therefore, the target of AI was the task of machine translation—the fully automatic rendering of texts from one language, such as Russian, into another, such as English. AI was “all in.” NATURAL LANGUAGE UNDERSTANDING As
…
Man can do.” Marvin Minsky, too, declared in 1967 that “within a generation, the problem of creating ‘artificial intelligence’ will be substantially solved.”2 But machine translation was a different ballgame, as researchers soon discovered. Having begun with a simplistic assumption, that language could be understood by analyzing words in large texts
…
high quality. Even programs working in specific domains such as biomedical literature were not fail-proof, and the failures were often embarrassingly incorrect and mindless. Machine translation researchers, in response, expanded their approach by exploring methods for “parsing” sentences, or finding syntactic structure in them, using new and powerful “transformational” grammars developed
…
had walked into a game of three-dimensional chess thinking it was tic-tac-toe.3 The National Resource Council (NRC) was pouring millions into machine-translation work at a number of American universities by the mid-1960s, but as for actual successes in engineering systems to understand, or even to simulate
…
the depth of problems the field faced.4 The effects of his reports on the research community were seismic. He had pinpointed the exact obstacle machine translation was foundering on, and it was irritatingly “philosophical”: the dearth of so-called common sense or “world knowledge”—knowledge about the actual world. Consider a
…
the NRC, the notion that computers could be programmed with the world knowledge of humans was “utterly chimerical, and hardly deserves any further discussion.”6 Machine translation was stuck, in other words, with results that were a far cry from fully automatic, high-quality translations (and that remain so today, although the
…
to data-driven or “empirical” methods seemed to liberate AI from those early, cloudy days of work on machine translation, when seemingly endless problems with capturing meaning and context plagued engineering efforts. In fact, machine translation itself was later cracked by a group of IBM researchers using a statistical (that is, not grammar-based
…
intensive—statistical approach pioneered by IBM Research Labs. SUCCESS … OR NOT The success of contemporary systems like Google Translate on the once puzzling problem of machine translation is often touted as evidence that AI will succeed, given enough time and the right ideas. The truth is more sobering. While it turns out
…
statistical or machine learning approaches, the original concerns of Bar-Hillel and others regarding semantics (meaning) and pragmatics (context) have proven to be well-founded. Machine translation, which had seemed like a difficult natural language problem, could be adequately accomplished with simple statistical analysis, given large corpora (datasets) in different languages. (And
…
note that machine translation is still not very high quality—it is more like “good enough to be useful.”) This is not evidence of impressive growth in machines’ natural
…
language understanding intelligence, but only evidence that machine translation is a much simpler problem than it was initially perceived to be. Again, deep problems with understanding language using computers have persisted. A simple way
…
MYCIN, which provided sometimes quite good medical diagnoses, made clear that AI methods were relevant to a variety of problems normally requiring high human intelligence. Machine translation, as we’ve seen, was an initial failure, but yielded to different approaches made possible by the availability of large datasets (a precursor to many
…
. Did the bot trick some judges some of the time? Yes. Did it actually pass the Turing test in any meaningful way? No.10 Though machine translation has in recent years been tamed (somewhat) by large volumes of texts translated into different languages on the web, the Turing test remains a perpetual
…
in the pen or I loved the river. I walked to the bank.) Confusingly, work in the 1960s on so-called fully-automated high-quality machine translation began with statistical approaches, albeit simpler ones. They didn’t work very well, and soon apostates like Yehoshua Bar-Hillel concluded that automatic translation was
…
IBM researchers, and web giants like Google used similar techniques (and now deep learning) to provide decent statistically-driven translation services. Fully-automated high-quality machine translation came full circle. But Bar-Hillel’s original skepticism is still germane; translations requiring knowledge and context are still a black box to modern approaches
…
Research Council, Publication 1416, 1966. 5. Sergei Nirenburg, H. L. Somers, and Yorick Wilks, eds., Readings in Machine Translation (Cambridge, MA: MIT Press, 2003), 75. 6. For a readable discussion of early problems with machine translation, see John Haugeland, Artificial Intelligence, The Very Idea (Cambridge, MA: MIT Press, 1989). Yehoshua Bar-Hillel’s comment
…
formal systems, 284n6 Frankenstein (fictional character), 238 Frankenstein: Or, a Modern Prometheus (novel, Shelly), 238, 280 frequency assumptions, 150–154, 173 Fully Automated High-Quality Machine Translation, 48 functions, 139 Galileo, 160 gambler’s fallacy, 122 games, 125–126 Gardner, Dan, 69–70 Garland, Alex, 79, 80, 289n16 Gates, Bill, 75 general
Artificial Intelligence: A Modern Approach
by Stuart Russell and Peter Norvig · 14 Jul 2019 · 2,466pp · 668,761 words
, the MNIST data set for handwritten digit recognition, ImageNet and COCO for image object recognition, SQUAD for natural language question answering, the WMT competition for machine translation, and the International SAT Competitions for Boolean satisfiability solvers. AI was founded in part as a rebellion against the limitations of existing fields like control
…
learning systems have exceeded human performance on some vision tasks (and lag behind in some other tasks). Similar gains have been reported in speech recognition, machine translation, medical diagnosis, and game playing. The use of a deep network to represent the evaluation function contributed to ALPHAGO’S victories over the leading human
…
Google Maps provide driving directions for hundreds of millions of users, quickly plotting an optimal route taking into account current and predicted future traffic conditions. Machine translation: Online machine translation systems now enable the reading of documents in over 100 languages, including the native languages of over 99% of humans, and render hundreds of
…
for HMMs first appeared in Russell and Norvig (2003). HMMs have found many applications in language processing (Charniak, 1993), speech recognition (Rabiner and Juang, 1993), machine translation (Och and Ney, 2003), computational biology (Krogh et al., 1994; Baldi et al., 1994), financial economics (Bhar and Hamori, 2004) and other fields. There have
…
model that makes you $10 on every trade, that’s great—but not if it costs you $20 in computation cost for each prediction. A machine translation program that runs on your phone and allows you to read signs in a foreign city is helpful—but not if it runs down the
…
computation paths from inputs to outputs have many steps. Deep learning is currently the most widely used approach for applications such as visual object recognition, machine translation, speech recognition, speech synthesis, and image synthesis; it also plays a significant role in reinforcement learning applications (see Chapter 23). Deep learning has its origins
…
algorithm for each experiment. This has encouraged an approach called end-to-end learning, in which a complex computational system for a task such as machine translation can be composed from several trainable subsystems; the entire system is then trained in an end-to-end fashion from input/output pairs. With this
…
translation tasks. Supervised translation consists of gathering many (x, y) pairs and training the model to map each x to the corresponding y. For example, machine translation systems are often trained on pairs of sentences that have been translated by professional human translators. For other kinds of translation, supervised training data may
…
with near-perfect accuracy. 22.8.2Natural language processing Deep learning has also had a huge impact on natural language processing (NLP) applications such as machine translation and speech recognition. Some advantages of deep learning for these applications include the possibility of end-to-end learning, the automatic generation of internal representations
…
encoders and decoders. End-to-end learning refers to the construction of entire systems as a single, learned function f. For example, an f for machine translation might take as input an English sentence SE and produce an equivalent Japanese sentence SJ = f (SE). Such an f can be learned from training
…
. (2016b) showed that end-to-end translation using deep learning reduced translation errors by 60% relative to a previous pipeline-based system. As of 2020, machine translation systems are approaching human performance for language pairs such as French and English for which very large paired data sets are available, and they are
…
for image processing and other tasks where the data have a grid topology. •Recurrent networks are effective for sequence-processing tasks including language modeling and machine translation. Bibliographical and Historical Notes The literature on neural networks is vast. Cowan and Sharp (1988b, 1988a) survey the early history, beginning with the work of
…
about 2/3 of listeners saying that the neural WaveNet system (van den Oord et al., 2016a) sounded more natural than the previous nonneural system. Machine translation transforms text in one language to another. Systems are usually trained using a bilingual corpus: a set of paired documents, where one member of the
…
pair is in, say, English, and the other is in, say, French. The documents do not need to be annotated in any way; the machine translation system learns to align sentences and phrases and then when presented with a novel sentence in one language, can generate a translation to the other
…
covers the use of recurrent neural networks to capture meaning and long-distance context as text is processed sequentially. Section 25.3 focuses primarily on machine translation, one of the major successes of deep learning applied to NLP. Sections 25.4 and 25.5 cover models that can be trained from large
…
sentences with many intervening words between the subject and verb. 25.3Sequence-to-Sequence Models One of the most widely studied tasks in NLP is machine translation (MT), where the goal is to translate a sentence from a source language to a target language—for example, from Spanish to English. We train
…
a basic sequence-to-sequence model, an example of which is shown in Figure 25.6. Sequence-to-sequence models are most commonly used for machine translation, but can also be used for a number of other tasks, like automatically generating a text caption from an image, or summarization: rewriting a long
…
the target RNN. Unlike most components of neural networks, attention probabilities are often interpretable by humans and intuitively meaningful. For example, in the case of machine translation, the attention probabilities often correspond to the word-to-word alignments that a human would generate. This is shown in Figure 25.7(b). Sequence
…
-to-sequence models are a natural for machine translation, but almost any natural language task can be encoded as a sequence-to-sequence problem. For example, a question-answering system can be trained on
…
called the transformer encoder. It is useful for text classification tasks. The full transformer architecture was originally designed as a sequence-to-sequence model for machine translation. Therefore, in addition to the encoder, it also includes a transformer decoder. The encoder and decoder are nearly identical, except that the decoder uses a
…
be used to train a question-answering system. Similarly, many Web sites publish side-by-side translations of texts, which can be used to train machine translation systems. Some text even comes with labels of a sort, such as review sites where users annotate their text reviews with a 5-star rating
…
. If this model is trained on a large corpus of text, it generates pretrained representations that perform well across a wide variety of NLP tasks (machine translation, question answering, summarization, grammaticality judgments, and others). 25.6State of the art Deep learning and transfer learning have markedly advanced the state of the art
…
can effectively model local and long-distance context by retaining relevant information in their hidden-state vectors. •Sequence-to-sequence models can be used for machine translation and text generation problems. •Transformer models use self-attention and can model long-distance context as well as local context. They can make effective use
…
) while maintaining high accuracy. The XLM system (Lample and Conneau, 2019) is a transformer model with training data from multiple languages. This is useful for machine translation, but also provides more robust representations for monolingual tasks. Two other important systems, GPT-2 (Radford et al., 2019) and T5 (Raffel et al., 2019
…
France the same time zone as the UK?”) and two reading comprehension tasks involving answering questions after reading either a paragraph or a news article. Machine translation is a major application of language models. In 1933, Petr Troyanskii received a patent for a “translating machine,” but there were no computers available to
…
computing resources to make the approach practical. In the 1970s that began to change, and the SYSTRAN system (Toma, 1977) was the first commercially successful machine translation system. SYSTRAN relied on lexical and grammatical rules hand-crafted by linguists as well as on training data. In the 1980s, the community embraced purely
…
but not fluent results (Och and Ney, 2004; Zollmann et al., 2008). Och and Ney (2002) show how discriminative training led to an advance in machine translation in the early 2000s. Sutskever et al. (2015) first showed that it is possible to learn an end-to-end sequence-to-sequence neural model
…
for machine translation. Bahdanau et al. (2015) demonstrated the advantage of a model that jointly learns to align sentences in the source and target language and to translate
…
between the languages. Vaswani et al. (2018) showed that neural machine translation systems can further be improved by replacing LSTMS with transformer architectures, which use the attention mechanism to capture context. These neural translation systems quickly overtook
…
workers face, and free them to concentrate on more interesting aspects. People with disabilities will benefit from AI-based assistance in seeing, hearing, and mobility. Machine translation already allows people from different cultures to communicate. Software-based AI solutions have near zero marginal cost of production, and so have the potential to
…
, IBM, Salesforce) have begun competing to offer machine learning APIs with pre-built models for specific tasks such as visual object recognition, speech recognition, and machine translation. These models can be used as is, or can serve as a baseline to be customized with your particular data for your particular application. We
…
adversarial networks (GANs) and transformer language models each opened up new areas of research. We have also seen steps towards “diversity of behaviour.” For example, machine translation systems in the 1990s were built one at a time for each language pair (such as French to English), but today a single system can
…
. and Schneider, J. (2001). Autonomous helicopter control using reinforcement learning policy search methods. In ICRA-01. Bahdanau, D., Cho, K., and Bengio, Y. (2015). Neural machine translation by jointly learning to align and translate. In ICLR-15. Bahubalendruni, M. R. and Biswal, B. B. (2016). A review on assembly sequence generation and
…
. Sixth Conference on Applied Natural Language Processing. Brants, T., Popat, A. C., Xu, P., Och, F. J., and Dean, J. (2007). Large language models in machine translation. In EMNLP-CoNLL-07. Bratko, I. (2009). Prolog Programming for Artificial Intelligence (4th edition). Addison-Wesley. Bratman, M. E. (1987). Intention, Plans, and Practical Reason
…
, G., Jones, L., Parmar, N., Schuster, M., Chen, Z., Wu, Y., and Hughes, M. (2018). The best of both worlds: Combining recent advances in neural machine translation. In ACL-18. Chen, S. F. and Goodman, J. (1996). An empirical study of smoothing techniques for language modeling. In ACL-96. Chen, T. and
…
planning. In ECML-06. Koditschek, D. (1987). Exact robot navigation by means of potential functions: Some topological considerations. In ICRA-87. Koehn, P. (2009). Statistical Machine Translation. Cambridge University Press. Koelsch, S. and Siebel, W. A. (2005). Towards a neural basis of music perception. Trends in Cognitive Sciences, 9, 578–584. Koenderink
…
systematic comparison of various statistical alignment models. Computational Linguistics, 29, 19–51. Och, F. J. and Ney, H. (2004). The alignment template approach to statistical machine translation. Computational Linguistics, 30, 417–449. Och, F. J. and Ney, H. (2002). Discriminative training and maximum entropy models for statistical
…
machine translation. In COLING–02. Ogawa, S., Lee, T.-M., Kay, A. R., and Tank, D. W. (1990). Brain magnetic resonance imaging with contrast dependent on blood
…
, D. M., Smith, A. F. M., and Makov, U. E. (1985). Statistical Analysis of Finite Mixture Distributions. Wiley. Toma, P. (1977). SYSTRAN as a multilingual machine translation system. In Proc. Third European Congress on Information Systems and Networks: Overcoming the Language Barrier. Tomasi, C. and Kanade, T. (1992). Shape and motion from
…
., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016b). Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv:1609.08144. Wu, Y. and He, K. (2018). Group normalization. arXiv:1803.08494. Xiong, W., Wu, L., Alleva, F., Droppo, J
…
Experimentation by Computer. MIT Press. Yngve, V. (1955). A model and an hypothesis for language structure. In Locke, W. N. and Booth, A. D. (Eds.), Machine Translation of Languages. MIT Press. Yob, G. (1975). Hunt the wumpus! Creative Computing, Sep/Oct. Yoshikawa, T. (1990). FoundationsofRobotics: Analysis and Control. MIT Press. You, Y
…
, 1117 Macherey, W., 47, 834, 901, 916, 1090, 1117 Machina, M., 550, 1105 machine evolution, 39 machine learning, 20, 22, 53, 220, 669, 669–987 machine translation (MT), 900, 915, 931 unsupervised, 831 Machover, M., 297, 1087 Macià, N., 737, 1097 MacKay, D. J. C., 476, 738, 1105, 1106 MacKenzie, D., 331
…
predictive control), 963, 971 MPE, see explanation, most probable MPI (mutual preferential independence), 533 MRS (metalevel reasoning system), 316 MST (minimum spanning tree), 127 MT (machine translation), 900, 915, 931 MUC (Message Understanding Conference), 905 Mudd, H. R, 1058 Mudigonda, M., 48, 1103 Mueller, E. T., 334, 358, 1107, 1113 Muggleton, S
…
language, 884 of probability, 410 synthesis (by theorem provers), 327 of algorithms, 327 SYNTHIA (simulated environment), 873 Syrjänen, T., 360, 1107 systematic search, 93 SYSTRAN (machine translation software), 931 Syverson, C., 1049, 1089 Szafron, D., 637, 638, 1087, 1088, 1116 Szathmáry, E., 161, 1113 Szegedy, C., 327, 330, 837, 838, 1085, 1086
…
kernel, 462 transition model, 70, 83, 123, 140, 193, 256, 479, 482, 510, 514, 553, 585, 939 sparse, 560 transitivity (of preferences), 520 translation, see machine translation transparency transpose, 1077 transposition (in a game), 201 transposition table, 201 Trappenberg, T., 839, 1115 traveling salesperson problem (TSP), 88, 125, 127 Traverso, P., 399
Future Politics: Living Together in a World Transformed by Tech
by Jamie Susskind · 3 Sep 2018 · 533pp
preferred definition would be wider than mine (including manual and emotional tasks as well). 3. Yonghui Wu et al. ‘Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation’, arXiv, 8 October 2016 <https://arxiv.org/abs/1609.08144> (accessed 6 December 2017); Yaniv Taigman et al.,‘DeepFace: Closing the
…
, 2010. Wu, Yonghui, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammed Norouzi, Wolfgang Macherey, Maxim Krikun, et al. ‘Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation’. arXiv, 8 Oct. 2016 <https://arxiv.org/abs/ 1609.08144> (accessed 6 Dec. 2017). Xiong, Wei, Jasha Droppo, Xupeng Huang, Frank Seide
The Globotics Upheaval: Globalisation, Robotics and the Future of Work
by Richard Baldwin · 10 Jan 2019 · 301pp · 89,076 words
common in web development, and a few back-office jobs, but little else. Things are different now in two ways. Machine Translation and the Talent Tsunami First, machine translation unleashed a talent tsunami. Since machine translation went mainstream in 2017, anyone with a laptop, internet connection, and skills can potentially telecommute to US and European offices
…
enough” English has greatly restricted the pool of potential telemigrants. Digital technology, however, is relaxing that restriction thanks to an amazing application of AI called “machine translation.” Instant translation used to be the stuff of science fiction. Today it is a reality and available for free on smartphones, tablets, and laptops. It
…
is a long way from perfect, but progress since 2017 has been absolutely amazing—as a French tourist in Iceland found out in 2017. MACHINE TRANSLATION AND THE TALENT TSUNAMI In August 2017, an Icelandic landowner caught a French tourist fishing illegally on his land and called the police. Once the
…
proceed without a human translator since Google Translate is now so accurate. In June 2017, the US Army paid Raytheon four million dollars for a machine translation package that lets soldiers converse with Iraqi Arabic and Pashto speakers as well as read foreign-language documents and digital media on their smartphones and
…
rough first draft. But no longer. Now it is rivaling average human translation for popular language pairs. According to Google, which uses humans to score machine translations on a scale from zero (complete nonsense) to six (perfect), the AI-trained algorithm “Google Translate” got a grade of 3.6 in 2015—far
…
2016, Google Translate hits numbers like 5.8 And the capabilities are advancing in leaps and bounds. As is true of almost everything globots do, machine translation is not as good as expert humans, but it is a whole lot cheaper and a whole lot more convenient. Expert human translators, in particular
…
, are quick to heap scorn on the talents of machine translation. The Atlantic Monthly, for instance, published an article in 2018 by Douglas Hofstadter doing just this.9 Hofsadter is a very sophisticated observer with very
…
high standards when it comes to machine translation. With a father who won the 1961 Nobel Prize in Physics, a PhD in physics to his name and now a post as a professor
…
something deeply lacking in the approach, which is conveyed by a single word: understanding.” But then he goes on to reveal a deep abhorrence of machine translation. Writing about the day when AI gets so good that human translators become mere quality checkers, he states that this would “cause a soul-shattering
…
blown off in a high wind or slip and fall into a void of pure nonsense,” he writes. While he is willing to concede that machine translation is functional, he denies it could ever replace real humans completely: “Google is often adequate . . . but only in the way of a particularly uninspired apprentice
…
humans, but in the meantime international business will be transformed when these uninspired apprentice translators massively lower, but don’t eliminate, language barriers. Instant, free machine translation is not something that is lurking in computer laboratories. Free apps like Google Translate and iTranslate Voice are now quite good across the major language
…
pairs. Other smart-phone apps include SayHi and WayGo. And machine translation is widely used. Google, for example, does a billion translations a day for online users. Try it out. Machine translation works on any smartphone. Just open up a foreign language website and apply Google Translate to
…
’s camera at a page of, say, French, and you see the English translation on your phone’s screen. Instant and free. YouTube has instant machine translation for many foreign-language YouTube videos. You just go to the settings “gear,” click on captions, and choose “auto-caption.” Instant, free spoken translation is
…
in 2018. At the end of 2017, Amazon introduced its contender—Amazon Translate—via Amazon Web Services. Unbuilding the Tower of Babel The fact that machine translation is entering everyday life is a big change. As anyone who has traveled or done business internationally knows, language is a huge barrier to just
…
the Tower of Babel, where “babel” means a confused noise made by a number of voices. Not to put too fine an edge on it, machine translation is unbuilding the Tower of Babel. This, in turn, is accelerating the pace at which American and European office workers are coming into direct competition
…
in other major languages, English dominates the market to date, so only a billion people are potential participants in the new online freelancing movement. With machine translation being so good, and getting better so fast, the billion who speak English will soon find themselves in much more direct competition with the other
…
six billion who don’t. Think about that. Then think about it again. Machine translation means that all this foreign talent soon will speak English or other rich-nation languages like French, German, Japanese, or Spanish—not perfectly, but well
…
the term used in China. Just imagine the increase in competition that will happen now that these “ant tribes” can speak good-enough English (via machine translation) and sell their brain power over the internet to the US, Europe, Japan, and other rich nations. But why is this only happening now? The
…
deep answer is Moore’s law and Gilder’s law have shifted into their eruptive growth phases when it comes to machine translation. WHY NOW? THE DEEP LEARNING TAKEOVER For a decade, hundreds of Google engineers made incremental progress on translation using the traditional, hands-on approach. In
…
making it seem almost as if foreign freelancers are sitting side-by-side with us even when they are in a different country. As with machine translation, this is no longer something only seen in Star Trek episodes, or the Hitchhiker’s Guide to the Galaxy. What I like to call “Advanced
…
Peter Norvig (2003). Artificial Intelligence: A Modern Approach (Englewood Cliffs, NJ: Prentice Hall, 2003). 8. Yonghui Wu et al., “Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation,” Technical Report, 2016. 9. Douglas Hofstadter, “The Shallowness of Google Translate,” The Atlantic Monthly, January 30, 2018. 10. Andy Martin, “Google
…
it easy to hire domestic freelancers, there is little to stop them from switching to lower cost foreign freelancers. As mentioned, the massive progress in machine translation, the rise of international freelancing platforms, and improved telecommunications is making telemigration a reality. As this catches on, the swapping foreign freelancers for domestic ones
…
real people who have to be in frequent in-person contact, since that is something telemigrants can’t do. Digital technology—especially advanced communication technologies, machine translation, and online international freelancing platforms—are making is easy for talented, low-cost foreigners sitting abroad to undertake many tasks in our offices. Which tasks
…
surely be important in the fast-moving, future world-of-work. Language skills, by contrast, will provide less of an advantage than they did before machine translation got so good. Consider an example of how globots changed the meaning of success in the law profession. Until recently, a law degree and a
Talk on the Wild Side
by Lane Greene · 15 Dec 2018 · 284pp · 84,169 words
as proof that the breakthroughs could come on both sides, and must have seemed as otherworldly as Sputnik itself. So how far has machine translation come? Anyone who generously gave machine translation not five but Turing’s 50 years, and looked at the options like “BabelFish” available online around 2004, will have noticed that
…
realised. Progress was so slow that it became a joke in the scientific community that true machine translation was five years away, and always would be. Today, though, language technologies are no longer hopeless. Not just machine translation but speech recognition, speech synthesis, and the ability to carry out basic spoken commands have gone
…
finite, and computers are getting more powerful all the time. The problem is not in the hardware. Yehoshua Bar-Hillel was an Israeli pioneer in machine translation, organising the first international conference on the subject, in 1952. But he came to be a sceptic of the idea that computers could ever produce
…
, IBM, the same company that had been behind the dramatic display in 1954, tried the new, data-driven approach, which came to be called statistical machine translation. IBM’s new system was called Candide. The name was appropriately chosen; like Voltaire’s fictional young man, Candide was naive, starting out knowing nothing
…
to get translated as “great man” in Hansard, and homme grand tends to get translated as “large man”. This is the heart of a statistical machine-translation system: making good guesses, based on lots of past data, about what chunks will translate as what. But the system needs a second component, too
…
. French and English syntax differ quite a bit, and the ideal output will be a good English text, so statistical machine-translation systems also need a “language model”: essentially a model of what good English looks like. In other words, the translation engine is translated on lots
…
set out with the modest ambition of being a kind of instant index to the entire internet. In 2006, Google launched Google Translate, its own machine-translation system, and by 2007, Google relied on entirely statistical translation. Google Translate, like Candide, would use large hordes of documents translated accurately by humans to
…
blogger’s experiment with BabelFish versus the new Google Translate, from 2007, round-tripped to German and back: Original: “Most state-of-the-art commercial machine translation systems in use today have been developed using a rules-based approach and require a lot of work by linguists to define vocabularies and grammars
…
. We use then statistic acquisition techniques, in order to establish translation a model, “explain Franz Och. Google: “Most of the state-of-the-art commercial machine translation in use today have been developed, with a rules-based approach and requires a lot of work by linguists to define vocabulary and grammar. Several
…
of California, Berkeley pages of assistant researcher John Brandon Lowe, at http://www.linguistics.berkeley.edu/~jblowe/Lx158/schedule/shrdlu.html 7. “Candide: A Statistical Machine Translation System”, by Stephen DellaPietra and Vincent DellaPietra, IBM’s principal investigators, at https://aclweb.org/anthology/H/H94/H94–1100.pdf 8. http://blogs.warwick
Surfaces and Essences
by Douglas Hofstadter and Emmanuel Sander · 10 Sep 2012 · 1,079pp · 321,718 words
the recycling bin. Given all this, how can we explain the fact that, in terms of serious thought, machines lag woefully behind us? Why is machine translation so often inept and awkward? Why are robots so primitive? Why is computer vision restricted to the simplest kinds of tasks? Why is it that
…
in English, but it has been coded in some strange symbols. I will now proceed to decode.’ ” Weaver’s humorous statement expresses the credo underlying machine translation, which is that translation is an act of “decoding” essentially analogous to using a substitution cipher, in which, in order to encode or decode a
…
by its alphabetic predecessor, in this case yielding Abraham Lincoln’s famous opening gambit, “Four score and seven years ago”. In the early days of machine translation, translation between languages was seen as this same process but just on a larger scale, in the sense that it operated not on letters but
…
Language A. We might test the efficacy of this strategy by seeing how the world’s most readily available (and perhaps also its most sophisticated) machine-translation “engine” performs on our little Lincolnian phrase “Four score and seven years ago”. (The term “engine” is a friendly tip of the hat to Charles
…
to come up with appropriate analogues in Language B for strings of characters written in Language A. Only a few years after the dream of machine translation was hatched, it was already starting to run into profound problems. These problems were articulated by a number of skeptics, of whom perhaps the most
…
recreate some of the high-sounding flavor of Abraham Lincoln’s immortal phrase, while sidestepping various superficially enticing traps along the way. Potential Progress in Machine Translation The preceding anecdote confirms the pervasive thesis of Warren Weaver’s book Alice in Many Tongues, which is that to translate well, the use of
…
and then to judge their appropriateness, one must carefully exploit one’s full inventory of mental resources, including one’s storehouse of life experiences. Could machine translation possibly do anything of the sort? Is it conceivable that one day, computer programs will be able to carry out translation at a high level
…
of artistry? A couple of decades ago, some machine-translation researchers, spurred by the low quality of what had then been achieved in their field, began to question the methods on which the field had
…
. What emerged with considerable vigor was the idea of statistical translation, which today has become a very important strategy used in tackling the challenge of machine translation. This approach is based on the use of statistically-based educated guesswork, where the data base in which all guesses are rooted consists of an
…
a data base is a marvelous treasurehouse of linguistic information, if only one can figure out how to exploit it. The basic idea of statistical machine translation is to choose among the many possible meanings of a “chunk” (that is, a word or several-word segment) in a piece of input text
…
order to do so, we’ll take a careful look at a short piece of French text in order to see how two extremely different machine-translation programs dealt with it — one using the old strategy, and one using the new strategy. The passage we will examine is taken from an obituary
…
the translation furnished by Google’s translation engine shortly after the obituary appeared. At that time, the Google engine was based on the original “Weaverian” machine-translation philosophy — namely, first via lookup in a very big on-line dictionary, followed by enhancement using grammatical “patching”. Original paragraph from Le Monde, September 2004
…
as they is merry of narguer the fate. It is obvious that the “decoding” technique — the technique that lay behind the original optimistic vision of machine translation — was hopelessly inadequate to the task, since the output that the translation engine yielded is pretty much nonsensical to an English speaker. It is ironic
…
apparently not in the engine’s on-line dictionary, so it was simply left in French.) This example gives a sense for the quality of machine translation in the fall of 2004. But now let us fast-forward to the spring of the year 2009. At that point, Google’s translation-engine
…
developers had radically switched strategies in favor of the new idea of statistical machine translation, so their new engine had little in common, other than its name, with its former incarnation. Given the inadequacy of the old method, which we
…
have any idea of the glee she took in laughing in the face of destiny. On reading this hand-done translation along with the two machine translations given earlier, one sees that humans and translation engines are not playing on the same turf — in fact, they are not even playing at the
…
languages no less than in monolingual speech production. But the analogies involved in carrying out human translation are not of the sort exploited by current machine-translation techniques, which make surface-level analogies between linguistic “chunks” based on statistical information that can be extracted, using intense calculation, from human-translated bilingual data
…
to make a large, coherent structure. Various and Sundry Challenges in Translating this Book The preceding section’s purpose was not to deride efforts at machine translation. Indeed, the blinding speed with which virtually anyone today can, entirely for free, get a glimpse of what is going on in a piece of
…
it to the New York subway, it would have been a wooden, almost mechanical kind of translation (although light-years more sophisticated than today’s machine translation). Could we not have elected, after having chosen an airport scene over a subway scene in the American version, to go back to the original
…
Cognition, 23, pp. 946–967. Khong, Yuen Foong (1992). Analogies at War. Princeton: Princeton University Press. Locke, W. N. and A. D. Booth (eds.) (1955). Machine Translation of Languages. Cambridge, Mass.: MIT Press. Macrae, C. Neil, Charles Stangor, and Miles Hewstone (1996). Stereotypes and Stereotyping. New York: The Guilford Press. Martin, Shirley
…
(1990). “Analog retrieval by constraint satisfaction”. Artificial Intelligence, 46, pp. 259–310. Weaver, Warren (1955). “Translation”. In W. N. Locke and A. D. Booth (eds.), Machine Translation of Languages. Cambridge, Mass.: MIT Press, pp. 15–23. ————— (1964). Alice in Many Tongues. Madison, Wisconsin: University of Wisconsin Press. Wharton, Charles M., Keith J
…
, list of, 136; stereotypes of, 135–136, 392, 486; strength of, as reflecting number of resemblances, 516; as strokes of genius, 16–17; superficial, in machine translation, 373, 375; taboo cases of, 104; as training wheels, 392; trivial and meaningless, ceaseless production of, 282, 284–286; unconscious, 259–281, 282, 285–286
…
unconscious analogies, 156 Bezout, Étienne, 413, 415, 420 bibles, category of 220, 229 bike-rental anomaly, explained by analogy, 328–330 bilingual data bases in machine translation, 369, 372–373 biplans: involving actions, 279; linguistic, 268–270 bird: as an example of an imprecise category, 55–56, 58, 59–60; as a
…
–35, 44 “lustre”, as possible French analogue to English word “score” in Gettysburg-address translation challenge, 372 —M— Macbeth effect, 289–290 Mach, Ernst, 487 machine translation, see translation Madonna, 223; syllogistic proof of mortality of, 193 magical angel stung by randomly buzzing interplanetary bumblebee, 493 magnet in motion, giving rise to
…
of, 442–443 squares, as questionable rectangles, 234–238, 255 staircases, negotiated by analogy, 507, 509, 516 Stargell, Willie, 325–326, 383 statistical approach to machine translation, 372–374 staying on the surface versus going into depth, 344 stealing, conceptual halo around, 106–107 stereotypes: of analogy-making, 135–136, 392, 521
…
, 386, 390, 403–407; see also categorical blinders, errors, fleeting analogies, latent analogies, naïve analogies understanding: as becoming used to something, 416; bypassing of, in machine translation, 368–370, 372–375; indispensability of, in human translation, 375–377 undo, on-line concept of, frame-blended with physical world, 406 undressing a banana
New Dark Age: Technology and the End of the Future
by James Bridle · 18 Jun 2018 · 301pp · 85,263 words
Natural language processing with Python
by Steven Bird, Ewan Klein and Edward Loper · 15 Dec 2009 · 504pp · 89,238 words
Found in Translation: How Language Shapes Our Lives and Transforms the World
by Nataly Kelly and Jost Zetzsche · 1 Oct 2012 · 274pp · 73,344 words
I'm Feeling Lucky: The Confessions of Google Employee Number 59
by Douglas Edwards · 11 Jul 2011 · 496pp · 154,363 words
The Creativity Code: How AI Is Learning to Write, Paint and Think
by Marcus Du Sautoy · 7 Mar 2019 · 337pp · 103,522 words
Robot Rules: Regulating Artificial Intelligence
by Jacob Turner · 29 Oct 2018 · 688pp · 147,571 words
AIQ: How People and Machines Are Smarter Together
by Nick Polson and James Scott · 14 May 2018 · 301pp · 85,126 words
The Cultural Logic of Computation
by David Golumbia · 31 Mar 2009 · 268pp · 109,447 words
The Equality Machine: Harnessing Digital Technology for a Brighter, More Inclusive Future
by Orly Lobel · 17 Oct 2022 · 370pp · 112,809 words
Text Analytics With Python: A Practical Real-World Approach to Gaining Actionable Insights From Your Data
by Dipanjan Sarkar · 1 Dec 2016
In Our Own Image: Savior or Destroyer? The History and Future of Artificial Intelligence
by George Zarkadakis · 7 Mar 2016 · 405pp · 117,219 words
Shape: The Hidden Geometry of Information, Biology, Strategy, Democracy, and Everything Else
by Jordan Ellenberg · 14 May 2021 · 665pp · 159,350 words
Architects of Intelligence
by Martin Ford · 16 Nov 2018 · 586pp · 186,548 words
Big Data: A Revolution That Will Transform How We Live, Work, and Think
by Viktor Mayer-Schonberger and Kenneth Cukier · 5 Mar 2013 · 304pp · 82,395 words
Masterminds of Programming: Conversations With the Creators of Major Programming Languages
by Federico Biancuzzi and Shane Warden · 21 Mar 2009 · 496pp · 174,084 words
Who Owns the Future?
by Jaron Lanier · 6 May 2013 · 510pp · 120,048 words
The Industries of the Future
by Alec Ross · 2 Feb 2016 · 364pp · 99,897 words
Talk to Me: How Voice Computing Will Transform the Way We Live, Work, and Think
by James Vlahos · 1 Mar 2019 · 392pp · 108,745 words
Kingdom of Characters: The Language Revolution That Made China Modern
by Jing Tsu · 18 Jan 2022 · 408pp · 105,715 words
The Singularity Is Nearer: When We Merge with AI
by Ray Kurzweil · 25 Jun 2024
Life After Google: The Fall of Big Data and the Rise of the Blockchain Economy
by George Gilder · 16 Jul 2018 · 332pp · 93,672 words
The Alignment Problem: Machine Learning and Human Values
by Brian Christian · 5 Oct 2020 · 625pp · 167,349 words
Deep Thinking: Where Machine Intelligence Ends and Human Creativity Begins
by Garry Kasparov · 1 May 2017 · 331pp · 104,366 words
Fluent Forever: How to Learn Any Language Fast and Never Forget It
by Gabriel Wyner · 4 Aug 2014 · 366pp · 87,916 words
AI Superpowers: China, Silicon Valley, and the New World Order
by Kai-Fu Lee · 14 Sep 2018 · 307pp · 88,180 words
Automate This: How Algorithms Came to Rule Our World
by Christopher Steiner · 29 Aug 2012 · 317pp · 84,400 words
Beautiful Data: The Stories Behind Elegant Data Solutions
by Toby Segaran and Jeff Hammerbacher · 1 Jul 2009
The Zero Marginal Cost Society: The Internet of Things, the Collaborative Commons, and the Eclipse of Capitalism
by Jeremy Rifkin · 31 Mar 2014 · 565pp · 151,129 words
The Age of AI: And Our Human Future
by Henry A Kissinger, Eric Schmidt and Daniel Huttenlocher · 2 Nov 2021 · 194pp · 57,434 words
Superminds: The Surprising Power of People and Computers Thinking Together
by Thomas W. Malone · 14 May 2018 · 344pp · 104,077 words
Artificial Intelligence: A Guide for Thinking Humans
by Melanie Mitchell · 14 Oct 2019 · 350pp · 98,077 words
Why Machines Learn: The Elegant Math Behind Modern AI
by Anil Ananthaswamy · 15 Jul 2024 · 416pp · 118,522 words
The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant From Two Centuries of Controversy
by Sharon Bertsch McGrayne · 16 May 2011 · 561pp · 120,899 words
Rule of the Robots: How Artificial Intelligence Will Transform Everything
by Martin Ford · 13 Sep 2021 · 288pp · 86,995 words
Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems
by Martin Kleppmann · 16 Mar 2017 · 1,237pp · 227,370 words
Ten Billion Tomorrows: How Science Fiction Technology Became Reality and Shapes the Future
by Brian Clegg · 8 Dec 2015 · 315pp · 92,151 words
The Big Nine: How the Tech Titans and Their Thinking Machines Could Warp Humanity
by Amy Webb · 5 Mar 2019 · 340pp · 97,723 words
Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems
by Martin Kleppmann · 17 Apr 2017
Overcomplicated: Technology at the Limits of Comprehension
by Samuel Arbesman · 18 Jul 2016 · 222pp · 53,317 words
The uplift war
by David Brin · 1 Jun 1987 · 789pp · 213,716 words
The Most Human Human: What Talking With Computers Teaches Us About What It Means to Be Alive
by Brian Christian · 1 Mar 2011 · 370pp · 94,968 words
Nerds on Wall Street: Math, Machines and Wired Markets
by David J. Leinweber · 31 Dec 2008 · 402pp · 110,972 words
Rise of the Robots: Technology and the Threat of a Jobless Future
by Martin Ford · 4 May 2015 · 484pp · 104,873 words
The Technology Trap: Capital, Labor, and Power in the Age of Automation
by Carl Benedikt Frey · 17 Jun 2019 · 626pp · 167,836 words
The Long History of the Future: Why Tomorrow's Technology Still Isn't Here
by Nicole Kobie · 3 Jul 2024 · 348pp · 119,358 words
A New History of the Future in 100 Objects: A Fiction
by Adrian Hon · 5 Oct 2020 · 340pp · 101,675 words
Accelerando
by Stross, Charles · 22 Jan 2005 · 489pp · 148,885 words
In the Plex: How Google Thinks, Works, and Shapes Our Lives
by Steven Levy · 12 Apr 2011 · 666pp · 181,495 words
Paper Knowledge: Toward a Media History of Documents
by Lisa Gitelman · 26 Mar 2014
The Inevitable: Understanding the 12 Technological Forces That Will Shape Our Future
by Kevin Kelly · 6 Jun 2016 · 371pp · 108,317 words
The Last Lingua Franca: English Until the Return of Babel
by Nicholas Ostler · 23 Nov 2010 · 484pp · 120,507 words
Is That a Fish in Your Ear?: Translation and the Meaning of Everything
by David Bellos · 10 Oct 2011 · 396pp · 107,814 words
Hands-On Machine Learning With Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems
by Aurélien Géron · 13 Mar 2017 · 1,331pp · 163,200 words
Our Final Invention: Artificial Intelligence and the End of the Human Era
by James Barrat · 30 Sep 2013 · 294pp · 81,292 words
Future Crimes: Everything Is Connected, Everyone Is Vulnerable and What We Can Do About It
by Marc Goodman · 24 Feb 2015 · 677pp · 206,548 words
Facebook: The Inside Story
by Steven Levy · 25 Feb 2020 · 706pp · 202,591 words
Thinking Machines: The Inside Story of Artificial Intelligence and Our Race to Build the Future
by Luke Dormehl · 10 Aug 2016 · 252pp · 74,167 words
Human Compatible: Artificial Intelligence and the Problem of Control
by Stuart Russell · 7 Oct 2019 · 416pp · 112,268 words
Doing Data Science: Straight Talk From the Frontline
by Cathy O'Neil and Rachel Schutt · 8 Oct 2013 · 523pp · 112,185 words
Reinventing Discovery: The New Era of Networked Science
by Michael Nielsen · 2 Oct 2011 · 400pp · 94,847 words
Eternity
by Greg Bear · 2 Jan 1988 · 523pp · 129,580 words
Ghost Work: How to Stop Silicon Valley From Building a New Global Underclass
by Mary L. Gray and Siddharth Suri · 6 May 2019 · 346pp · 97,330 words
These Strange New Minds: How AI Learned to Talk and What It Means
by Christopher Summerfield · 11 Mar 2025 · 412pp · 122,298 words
Stealth
by Peter Westwick · 22 Nov 2019 · 474pp · 87,687 words
Children of Ruin
by Adrian Tchaikovsky · 13 May 2019 · 471pp · 147,210 words
The Dream Machine: J.C.R. Licklider and the Revolution That Made Computing Personal
by M. Mitchell Waldrop · 14 Apr 2001
I, Warbot: The Dawn of Artificially Intelligent Conflict
by Kenneth Payne · 16 Jun 2021 · 339pp · 92,785 words
Structure and interpretation of computer programs
by Harold Abelson, Gerald Jay Sussman and Julie Sussman · 25 Jul 1996 · 893pp · 199,542 words
Final Jeopardy: Man vs. Machine and the Quest to Know Everything
by Stephen Baker · 17 Feb 2011 · 238pp · 77,730 words
What to Think About Machines That Think: Today's Leading Thinkers on the Age of Machine Intelligence
by John Brockman · 5 Oct 2015 · 481pp · 125,946 words
The Seventh Sense: Power, Fortune, and Survival in the Age of Networks
by Joshua Cooper Ramo · 16 May 2016 · 326pp · 103,170 words
Track Changes
by Matthew G. Kirschenbaum · 1 May 2016 · 519pp · 142,646 words
GCHQ
by Richard Aldrich · 10 Jun 2010 · 826pp · 231,966 words
The Invisible Web: Uncovering Information Sources Search Engines Can't See
by Gary Price, Chris Sherman and Danny Sullivan · 2 Jan 2003 · 481pp · 121,669 words
The Wealth of Humans: Work, Power, and Status in the Twenty-First Century
by Ryan Avent · 20 Sep 2016 · 323pp · 90,868 words
The Formula: How Algorithms Solve All Our Problems-And Create More
by Luke Dormehl · 4 Nov 2014 · 268pp · 75,850 words
Structure and Interpretation of Computer Programs, Second Edition
by Harold Abelson, Gerald Jay Sussman and Julie Sussman · 1 Jan 1984 · 1,387pp · 202,295 words
There's a War Going on but No One Can See It
by Huib Modderkolk · 1 Sep 2021 · 295pp · 84,843 words
The Man Who Invented the Computer
by Jane Smiley · 18 Oct 2010 · 253pp · 80,074 words
Race Against the Machine: How the Digital Revolution Is Accelerating Innovation, Driving Productivity, and Irreversibly Transforming Employment and the Economy
by Erik Brynjolfsson · 23 Jan 2012 · 72pp · 21,361 words
The Information: A History, a Theory, a Flood
by James Gleick · 1 Mar 2011 · 855pp · 178,507 words
A World Without Work: Technology, Automation, and How We Should Respond
by Daniel Susskind · 14 Jan 2020 · 419pp · 109,241 words
Cage of Souls
by Adrian Tchaikovsky · 4 Apr 2019 · 703pp · 196,052 words
Data-Ism: The Revolution Transforming Decision Making, Consumer Behavior, and Almost Everything Else
by Steve Lohr · 10 Mar 2015 · 239pp · 70,206 words
Genius Makers: The Mavericks Who Brought A. I. To Google, Facebook, and the World
by Cade Metz · 15 Mar 2021 · 414pp · 109,622 words
Supremacy: AI, ChatGPT, and the Race That Will Change the World
by Parmy Olson · 284pp · 96,087 words
Artificial Unintelligence: How Computers Misunderstand the World
by Meredith Broussard · 19 Apr 2018 · 245pp · 83,272 words
Natural Language Annotation for Machine Learning
by James Pustejovsky and Amber Stubbs · 14 Oct 2012 · 502pp · 107,510 words
More Money Than God: Hedge Funds and the Making of a New Elite
by Sebastian Mallaby · 9 Jun 2010 · 584pp · 187,436 words
Only Humans Need Apply: Winners and Losers in the Age of Smart Machines
by Thomas H. Davenport and Julia Kirby · 23 May 2016 · 347pp · 97,721 words
The Caryatids
by Bruce Sterling · 24 Feb 2009 · 387pp · 105,250 words
On Language: Chomsky's Classic Works Language and Responsibility and Reflections on Language in One Volume
by Noam Chomsky and Mitsou Ronat · 26 Jul 2011
Red Moon
by Kim Stanley Robinson · 22 Oct 2018 · 492pp · 141,544 words
WTF?: What's the Future and Why It's Up to Us
by Tim O'Reilly · 9 Oct 2017 · 561pp · 157,589 words
Machine Learning Design Patterns: Solutions to Common Challenges in Data Preparation, Model Building, and MLOps
by Valliappa Lakshmanan, Sara Robinson and Michael Munn · 31 Oct 2020
Possible Minds: Twenty-Five Ways of Looking at AI
by John Brockman · 19 Feb 2019 · 339pp · 94,769 words
We Are Data: Algorithms and the Making of Our Digital Selves
by John Cheney-Lippold · 1 May 2017 · 420pp · 100,811 words
The AI Economy: Work, Wealth and Welfare in the Robot Age
by Roger Bootle · 4 Sep 2019 · 374pp · 111,284 words
AI in Museums: Reflections, Perspectives and Applications
by Sonja Thiel and Johannes C. Bernhardt · 31 Dec 2023 · 321pp · 113,564 words
The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World
by Pedro Domingos · 21 Sep 2015 · 396pp · 117,149 words
The Future of the Professions: How Technology Will Transform the Work of Human Experts
by Richard Susskind and Daniel Susskind · 24 Aug 2015 · 742pp · 137,937 words
Superintelligence: Paths, Dangers, Strategies
by Nick Bostrom · 3 Jun 2014 · 574pp · 164,509 words
Free Speech: Ten Principles for a Connected World
by Timothy Garton Ash · 23 May 2016 · 743pp · 201,651 words
Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again
by Eric Topol · 1 Jan 2019 · 424pp · 114,905 words
Empire of AI: Dreams and Nightmares in Sam Altman's OpenAI
by Karen Hao · 19 May 2025 · 660pp · 179,531 words
Who Owns This Sentence?: A History of Copyrights and Wrongs
by David Bellos and Alexandre Montagu · 23 Jan 2024 · 305pp · 101,093 words
Literary Theory for Robots: How Computers Learned to Write
by Dennis Yi Tenen · 6 Feb 2024 · 169pp · 41,887 words
Information: A Very Short Introduction
by Luciano Floridi · 25 Feb 2010 · 137pp · 36,231 words
Narrative Economics: How Stories Go Viral and Drive Major Economic Events
by Robert J. Shiller · 14 Oct 2019 · 611pp · 130,419 words
Because Internet: Understanding the New Rules of Language
by Gretchen McCulloch · 22 Jul 2019 · 413pp · 106,479 words
Human + Machine: Reimagining Work in the Age of AI
by Paul R. Daugherty and H. James Wilson · 15 Jan 2018 · 523pp · 61,179 words
Pattern Recognition
by William Gibson · 2 Jan 2003 · 385pp · 99,985 words
Noam Chomsky: A Life of Dissent
by Robert F. Barsky · 2 Feb 1997
Applied Artificial Intelligence: A Handbook for Business Leaders
by Mariya Yao, Adelyn Zhou and Marlene Jia · 1 Jun 2018 · 161pp · 39,526 words
What We Owe the Future: A Million-Year View
by William MacAskill · 31 Aug 2022 · 451pp · 125,201 words
Code Dependent: Living in the Shadow of AI
by Madhumita Murgia · 20 Mar 2024 · 336pp · 91,806 words
Mastering Structured Data on the Semantic Web: From HTML5 Microdata to Linked Open Data
by Leslie Sikos · 10 Jul 2015
The Optimist: Sam Altman, OpenAI, and the Race to Invent the Future
by Keach Hagey · 19 May 2025 · 439pp · 125,379 words
The Mysterious Mr. Nakamoto: A Fifteen-Year Quest to Unmask the Secret Genius Behind Crypto
by Benjamin Wallace · 18 Mar 2025 · 431pp · 116,274 words
This Is for Everyone: The Captivating Memoir From the Inventor of the World Wide Web
by Tim Berners-Lee · 8 Sep 2025 · 347pp · 100,038 words
The Mind Is Flat: The Illusion of Mental Depth and the Improvised Mind
by Nick Chater · 28 Mar 2018 · 263pp · 81,527 words
This Will Make You Smarter: 150 New Scientific Concepts to Improve Your Thinking
by John Brockman · 14 Feb 2012 · 416pp · 106,582 words
Sparks: China's Underground Historians and Their Battle for the Future
by Ian Johnson · 26 Sep 2023 · 407pp · 119,073 words
Devil's Bargain: Steve Bannon, Donald Trump, and the Storming of the Presidency
by Joshua Green · 17 Jul 2017 · 296pp · 78,112 words
Calling Bullshit: The Art of Scepticism in a Data-Driven World
by Jevin D. West and Carl T. Bergstrom · 3 Aug 2020
Babel No More: The Search for the World's Most Extraordinary Language Learners
by Michael Erard · 10 Jan 2012 · 392pp · 104,760 words
Blue Ocean Strategy, Expanded Edition: How to Create Uncontested Market Space and Make the Competition Irrelevant
by W. Chan Kim and Renée A. Mauborgne · 20 Jan 2014 · 287pp · 80,180 words
Surviving AI: The Promise and Peril of Artificial Intelligence
by Calum Chace · 28 Jul 2015 · 144pp · 43,356 words
The Fractalist
by Benoit Mandelbrot · 30 Oct 2012
Collaborative Society
by Dariusz Jemielniak and Aleksandra Przegalinska · 18 Feb 2020 · 187pp · 50,083 words
Simple Rules: How to Thrive in a Complex World
by Donald Sull and Kathleen M. Eisenhardt · 20 Apr 2015 · 294pp · 82,438 words
The Economic Singularity: Artificial Intelligence and the Death of Capitalism
by Calum Chace · 17 Jul 2016 · 477pp · 75,408 words
Picnic Comma Lightning: In Search of a New Reality
by Laurence Scott · 11 Jul 2018 · 244pp · 81,334 words
You Are What You Speak: Grammar Grouches, Language Laws, and the Politics of Identity
by Robert Lane Greene · 8 Mar 2011 · 319pp · 95,854 words
The Business of Platforms: Strategy in the Age of Digital Competition, Innovation, and Power
by Michael A. Cusumano, Annabelle Gawer and David B. Yoffie · 6 May 2019 · 328pp · 84,682 words
The Future Is Asian
by Parag Khanna · 5 Feb 2019 · 496pp · 131,938 words
Smarter Than Us: The Rise of Machine Intelligence
by Stuart Armstrong · 1 Feb 2014 · 48pp · 12,437 words
The Internet Trap: How the Digital Economy Builds Monopolies and Undermines Democracy
by Matthew Hindman · 24 Sep 2018
Covid-19: The Pandemic That Never Should Have Happened and How to Stop the Next One
by Debora MacKenzie · 13 Jul 2020 · 266pp · 80,273 words
Why Things Bite Back: Technology and the Revenge of Unintended Consequences
by Edward Tenner · 1 Sep 1997
How Doctors Think
by Jerome Groopman · 15 Jan 2007 · 292pp · 94,324 words
The Left Case Against the EU
by Costas Lapavitsas · 17 Dec 2018 · 221pp · 46,396 words
Spike: The Virus vs The People - The Inside Story
by Jeremy Farrar and Anjana Ahuja · 15 Jan 2021 · 245pp · 71,886 words
Language and Mind
by Noam Chomsky · 1 Jan 1968
Understanding Sponsored Search: Core Elements of Keyword Advertising
by Jim Jansen · 25 Jul 2011 · 298pp · 43,745 words
Pandora's Brain
by Calum Chace · 4 Feb 2014 · 345pp · 104,404 words