Machine Translation
by Thierry Poibeau · 14 Sep 2017 · 174pp · 56,405 words
the Modern Corporation, James W. Cortada Intellectual Property Strategy, John Palfrey The Internet of Things, Samuel Greengard Machine Learning: The New AI, Ethem Alpaydin Machine Translation, Thierry Poibeau Memes in Digital Culture, Limor Shifman Metadata, Jeffrey Pomerantz The Mind–Body Problem, Jonathan Westphal MOOCs, Jonathan Haber Neuroplasticity, Moheb Costandi Open
…
Self-Tracking, Gina Neff and Dawn Nafus Sustainability, Kent E. Portney The Technological Singularity, Murray Shanahan Understanding Beliefs, Nils J. Nilsson Waves, Frederic Raichlen Machine Translation Thierry Poibeau The MIT Press Cambridge, Massachusetts London, England © 2017 Massachusetts Institute of Technology All rights reserved. No part of this book may be reproduced
…
Report and Its Consequences 7 Parallel Corpora and Sentence Alignment 8 Example-Based Machine Translation 9 Statistical Machine Translation and Word Alignment 10 Segment-Based Machine Translation 11 Challenges and Limitations of Statistical Machine Translation 12 Deep Learning Machine Translation 13 The Evaluation of Machine Translation Systems 14 The Machine Translation Industry: Between Professional and Mass-Market Applications 15 Conclusion: The Future of
…
Machine Translation Glossary Bibliography and Further Reading Index About Author List of Tables Table 1 Example of possible
…
from the analysis of knowledge and reasoning, which explains the interest shown by philosophers and specialists of artificial intelligence as well as cognitive sciences in [machine translation]. Machine translation involves different processes that make it at least as challenging as developing an automatic dialoguing system. The degree of “understanding” shown by the machine
…
inferred information, which is of course highly challenging and simply goes beyond the current state of the art. The Revolution of Statistical Machine Translation Systems The classification of machine translation systems provided in the previous section is challenged by new approaches that have appeared since the early 1990s. The availability of huge quantities
…
in such symbols could be translated by all who possessed the dictionary” (Descartes, letter to Mersenne on November 20, 1629). This passage greatly inspired machine translation pioneers, since Descartes’ proposal aimed to replace words with unambiguous codes (“symbols” corresponding to numerical codes that are independent from the languages considered; symbols
…
in automatic translation systems. Although these projects resulted in relatively advanced proposals with vocabularies and grammar systems, they have rarely been actively used for machine translation. Esperanto was used during the 1980s in the European Distributed Translation Language project and within the Fujitsu company in Japan, but these two projects
…
it must therefore be translated; in other words, decoded in the target language). Beginning in 1947, Weaver corresponded with the cyberneticist Norbert Wiener concerning machine translation. He proposed that translation could be considered a “decoding” problem: One naturally wonders if the problem of translation could conceivably be treated as a problem
…
more powerful than symbolic rules to resolve ambiguities). The implementation of the proposed techniques, however, required efforts that went beyond anything the pioneers of machine translation had ever imagined. In particular, the inherent ambiguity of natural languages showed that traditional encryption models were not sufficient to render the complexity of automatic
…
-quality translation in the short or medium term (FAHQT, or fully automated high-quality translation; also found as FAHQMT, or fully automated high-quality machine translation). Instead of automatic translation, Bar-Hillel recommended that researchers turn toward computer-assisted translation systems, which constitute a relatively different project, clearly less exciting scientifically
…
the other hand, with the increasing amount of translations available on the Internet, it is now possible to directly design statistical models for machine translation. This approach, known as statistical machine translation, is the most popular today. Unlike a translation memory, which can be relatively small, automatic processing presumes the availability of an
…
it would be more convenient to directly use fragments of translation that one can find in existing bilingual corpora. An Overview of Example-Based Machine Translation Example-based machine translation typically operates in three stages to translate a given sentence: The system tries to find fragments of the sentence to be translated in the
…
sparsity (it is very difficult to collect enough relevant examples at phrase level). Appeal and Limitations of Example-Based Machine Translation Example-based machine translation generated great interest during the 1980s. Rather than developing a machine translation system manually, which is long and very costly, the example-based approach allowed for optimal exploitation of large
…
information of a syntactic and semantic nature has been progressively integrated into models to compensate for the limitation of purely statistical approaches. Toward Segment-Based Machine Translation The IBM models have been subjected to numerous enhancements. The most significant improvement was to take into consideration the notion of segments (or sequences
…
have described in this section have, however, helped improve the IBM models and can still be considered currently as the state of the art in machine translation. Introduction of Linguistic Information into Statistical Models Statistical translation models, despite their increasing complexity to better fit language specificities, have not solved all the
…
techniques. Lastly, for rare languages with too few data to make it possible to develop statistical systems, rule-based systems remain the norm. Hybrid Machine Translation Systems Following the success of statistical translation systems, the majority of traditional systems (based on large lexicons and transfer rules) gradually tried to incorporate statistical
…
deep learning provides an interesting approach that seems especially fitted for the challenges involved in improving human language processing. An Overview of Deep Learning for Machine Translation Deep learning achieved its first success in image recognition. Rather than using a group of predefined characteristics, deep learning generally operates from a very
…
to let the system infer by itself the best representation from the data. A translation system based solely on deep learning (aka “deep learning machine translation” or “neural machine translation”) thus simply consists of an “encoder” (the part of the system that analyzes the training data) and a “decoder” (the part of the
…
toward the resolution of such problems, hence the great success of this technique among researchers in the domain. Current Challenges for Deep Learning Machine Translation Until recently, machine translation systems based on deep learning performed well on simple sentences but still lagged behind traditional statistical systems for more complex sentences. There were different
…
of the internal model calculated by the neural network, so as to better understand how the whole approach works. The deep learning approach to machine translation (or neural machine translation) has proven efficient, first, on short sentences in closely related languages, and more recently on long sentences as well as more diverse languages.
…
initiated research in this area. A 1994 article (White et al., 1994) reviewed the first attempts at evaluation from the beginnings of research on machine translation. The article specifically reported the various possible strategies and their limits, described below. Comprehension Evaluation To assess comprehension, professional human translators first translated English newspaper
…
articles into different languages. Machine translation systems then translated the text back into English, and human analysts answered “multiple choice questions about the content of the articles” to evaluate the automatic
…
Fluency After the previous attempts involving human experts, DARPA then resorted to two evaluation scores: adequacy and fluency. As White and colleagues described of this machine translation (MT) evaluation method: “In an adequacy evaluation, literate, monolingual English speakers make judgments determining the degree to which the information in a professional translation
…
automatically. It is, rather, the capacity of the system to provide relevant translational elements that should be evaluated. 14 The Machine Translation Industry: Between Professional and Mass-Market Applications Machine translation is a popular application because it answers a very direct and simple need. Everybody can clearly see the importance of a system
…
analysis and translation systems used by Samsung’s connected devices (cell phones, tablets, and other technological gadgets). Facebook bought out different companies specialized in machine translation (such as Jibbigo in 2013 for voice messages in particular). Apple and Google are also regularly buying startups in the communication and information technology domains
…
is actually twofold. First, improving the productivity of translators: this involves efficient systems and strategies to make the best of the output of machine translation tools. Second, improving machine translation systems directly: this means being able to dynamically reuse end-user feedback to make the system evolve and propose more accurate translations in
…
Warren Weaver and the launching of MT: Brief biographical note”) and Y. Bar-Hillel (“Yehoshua Bar-Hillel: A philosopher’s contribution to machine translation”), both in Early Years in Machine Translation (see the full reference at the beginning of this chapter). Chapter 6: The 1966 ALPAC Report and Its Consequences The ALPAC report and
…
previous website (http://www.statmt.org) is probably the best source of information for recent trends related to statistical machine translation, of which segment-based machine translation is part. Chapter 11: Challenges and Limitations of Statistical Machine Translation See http://www.statmt.org,as for chapter 10 above. Kenneth Church (2011). “A pendulum swung too
…
Journal of Translation Studies 13 (1–2): 29–70. Special issue: The teaching of computer-aided translation, ed. Chan Sin Wai. Philipp Koehn (2009). Statistical Machine Translation. Cambridge: Cambridge University Press. Jorg Tiedemann (2011). Bitext Alignment. San Rafael, CA: Morgan and Claypool Publishers. Dan Jurafsky and James H. Martin (2016). Speech
…
3: The Correspondence. Cambridge: Cambridge University Press. Umberto Eco (1997). The Search for the Perfect Language. Oxford: Wiley. John Hutchins (2004). “Two precursors of machine translation: Artsrouni and Trojanskij.” International Journal of Translation 16 (1): 11–31. Philip P. Wiener (ed., 1951). Leibniz Selections. New York: Simon and Schuster. Yehoshua Bar
…
Human Intelligence (A. Elithorn and R. Banerji, eds.). Elsevier Science Publishers, Amsterdam. Eiichiro Sumita and Hitoshi Iida (1991). “Experiments and prospects of example-based machine translation.” Proceedings of the Twenty-Ninth Conference of the Association for Computational Linguistics, 185–192. Berkeley, CA. Thomas R. Green (1979). “The necessity of syntax markers
…
: Two experiments with artificial languages.” Verbal Learning and Verbal Behavior 18: 481–496. Harold Somers (1999). “Example-based machine translation.” Machine translation 14 (2): 113–157. Nano Gough and Andy Way (2004). “Robust large-scale EBMT with marker-based segmentation.” Proceedings of the Tenth International Conference on
…
Theoretical and Methodological Issues in Machine Translation, 95–104. Baltimore, MD. Peter Brown, John Cocke, Stephen Della Pietra, Vincent Della Pietra, Frederick Jelinek, Robert Mercer, and Paul Roossin (1988). “A
…
, Yoshua Bengio and Aaron Courville (2016). Deep Learning. Cambridge, MA: MIT Press. Yonghui Wu, et al. (2016). “Google's neural machine translation system: Bridging the gap between human and machine translation.” Published online. arXiv:1609.08144. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu (2002). “BLEU: A method for automatic
…
51, 56, 61, 63, 100, 118, 164–168, 215–216, 250 Environment Canada, 87, 223 Error rate, 241. See also Evaluation Errors (in machine translation). See Typology of errors in machine translation Escher, M. C., 20 Esperanto, 28, 42, 44 Estonian, 97, 212, 213 Europarl corpus, 97, 210, 223 Europe, 36, 41, 86, 97
…
58, 60, 85, 179 Logos Corporation, 88 Machine learning, 175, 181, 183, 236. See also Deep learning Machine translation evaluation. See Evaluation Machine translation industry. See Machine translation market Machine translation market, 89, 221–246, 247–251 Machine translation quality. See Evaluation Machine translation systems Apertium, 172 Ariane-78 system, 85 Babelfish, 227, 228 Bing Translation, 33, 36, 194, 226–229,
…
Systran) TAUM Météo (see Météo) Watson, 241 Maintenance applications, 243 Maltese, 212, 213 Manual correction, 138. See also Post-edition Mass-market applications of machine translation. See Machine translation market Mass media, 239 Mathematical model of communication. See Model of communication Meaning, 8, 15, 17–21, 34, 52–55, 64–67, 70–71
…
, Igor, 69 Memorandum, Warren Weaver’s, 50, 52–59 Mercer, Robert, 93, 94, 166, 216, 258 Mersel, Jules, 76 Metal. See Machine translation systems Metaphysics, 179 Météo. See Machine translation systems Meteor. See Evaluation measure and test Michigan University, 81 Microsoft, 227–229, 240, 248–250 MIT, 60–62 Mobile application, 229, 232
…
264 System comparison (see Evaluation) maintenance, 243 quality (see Evaluation) Systran, 85–89, 171, 194, 223, 226, 227–229, 231–236 Systranet. See Machine translation systems TAUM Météo. See Machine translation systems Technical text, 4, 11, 13, 88, 223, 244 Technical translation, 92. See also Technical text Terminology, 92, 101, 119, 226, 228, 244
The Codebreakers: The Comprehensive History of Secret Communication From Ancient Times to the Internet
by David Kahn · 1 Feb 1963 · 1,799pp · 532,462 words
Maya: E. V. Yevreinov, Yu. G. Kosarev, and V. A. Ustinov, three 1961 articles from different Russian sources translated and published as Foreign Developments in Machine Translation and Information Processing, No. 40, by the United States, Department of Commerce, Office of Technical Services, Joint Publications Research Service, No. 10508; and criticism by
…
-Formation and Semantics,” published in Materialy po mate-maticheskoy lingvistika i mashinnomu perevodu, II (Leningrad University, 1963), which has been translated as Foreign Developments in Machine Translation and Information Processing, No. 161, United States, Department of Commerce, Office of Technical Services, Joint Publications Research Service, No. 26209. Dr. Andreyev has proposed an
The Globotics Upheaval: Globalisation, Robotics and the Future of Work
by Richard Baldwin · 10 Jan 2019 · 301pp · 89,076 words
common in web development, and a few back-office jobs, but little else. Things are different now in two ways. Machine Translation and the Talent Tsunami First, machine translation unleashed a talent tsunami. Since machine translation went mainstream in 2017, anyone with a laptop, internet connection, and skills can potentially telecommute to US and European offices
…
enough” English has greatly restricted the pool of potential telemigrants. Digital technology, however, is relaxing that restriction thanks to an amazing application of AI called “machine translation.” Instant translation used to be the stuff of science fiction. Today it is a reality and available for free on smartphones, tablets, and laptops. It
…
is a long way from perfect, but progress since 2017 has been absolutely amazing—as a French tourist in Iceland found out in 2017. MACHINE TRANSLATION AND THE TALENT TSUNAMI In August 2017, an Icelandic landowner caught a French tourist fishing illegally on his land and called the police. Once the
…
proceed without a human translator since Google Translate is now so accurate. In June 2017, the US Army paid Raytheon four million dollars for a machine translation package that lets soldiers converse with Iraqi Arabic and Pashto speakers as well as read foreign-language documents and digital media on their smartphones and
…
rough first draft. But no longer. Now it is rivaling average human translation for popular language pairs. According to Google, which uses humans to score machine translations on a scale from zero (complete nonsense) to six (perfect), the AI-trained algorithm “Google Translate” got a grade of 3.6 in 2015—far
…
2016, Google Translate hits numbers like 5.8 And the capabilities are advancing in leaps and bounds. As is true of almost everything globots do, machine translation is not as good as expert humans, but it is a whole lot cheaper and a whole lot more convenient. Expert human translators, in particular
…
, are quick to heap scorn on the talents of machine translation. The Atlantic Monthly, for instance, published an article in 2018 by Douglas Hofstadter doing just this.9 Hofsadter is a very sophisticated observer with very
…
high standards when it comes to machine translation. With a father who won the 1961 Nobel Prize in Physics, a PhD in physics to his name and now a post as a professor
…
something deeply lacking in the approach, which is conveyed by a single word: understanding.” But then he goes on to reveal a deep abhorrence of machine translation. Writing about the day when AI gets so good that human translators become mere quality checkers, he states that this would “cause a soul-shattering
…
blown off in a high wind or slip and fall into a void of pure nonsense,” he writes. While he is willing to concede that machine translation is functional, he denies it could ever replace real humans completely: “Google is often adequate . . . but only in the way of a particularly uninspired apprentice
…
humans, but in the meantime international business will be transformed when these uninspired apprentice translators massively lower, but don’t eliminate, language barriers. Instant, free machine translation is not something that is lurking in computer laboratories. Free apps like Google Translate and iTranslate Voice are now quite good across the major language
…
pairs. Other smart-phone apps include SayHi and WayGo. And machine translation is widely used. Google, for example, does a billion translations a day for online users. Try it out. Machine translation works on any smartphone. Just open up a foreign language website and apply Google Translate to
…
’s camera at a page of, say, French, and you see the English translation on your phone’s screen. Instant and free. YouTube has instant machine translation for many foreign-language YouTube videos. You just go to the settings “gear,” click on captions, and choose “auto-caption.” Instant, free spoken translation is
…
in 2018. At the end of 2017, Amazon introduced its contender—Amazon Translate—via Amazon Web Services. Unbuilding the Tower of Babel The fact that machine translation is entering everyday life is a big change. As anyone who has traveled or done business internationally knows, language is a huge barrier to just
…
the Tower of Babel, where “babel” means a confused noise made by a number of voices. Not to put too fine an edge on it, machine translation is unbuilding the Tower of Babel. This, in turn, is accelerating the pace at which American and European office workers are coming into direct competition
…
in other major languages, English dominates the market to date, so only a billion people are potential participants in the new online freelancing movement. With machine translation being so good, and getting better so fast, the billion who speak English will soon find themselves in much more direct competition with the other
…
six billion who don’t. Think about that. Then think about it again. Machine translation means that all this foreign talent soon will speak English or other rich-nation languages like French, German, Japanese, or Spanish—not perfectly, but well
…
the term used in China. Just imagine the increase in competition that will happen now that these “ant tribes” can speak good-enough English (via machine translation) and sell their brain power over the internet to the US, Europe, Japan, and other rich nations. But why is this only happening now? The
…
deep answer is Moore’s law and Gilder’s law have shifted into their eruptive growth phases when it comes to machine translation. WHY NOW? THE DEEP LEARNING TAKEOVER For a decade, hundreds of Google engineers made incremental progress on translation using the traditional, hands-on approach. In
…
making it seem almost as if foreign freelancers are sitting side-by-side with us even when they are in a different country. As with machine translation, this is no longer something only seen in Star Trek episodes, or the Hitchhiker’s Guide to the Galaxy. What I like to call “Advanced
…
Peter Norvig (2003). Artificial Intelligence: A Modern Approach (Englewood Cliffs, NJ: Prentice Hall, 2003). 8. Yonghui Wu et al., “Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation,” Technical Report, 2016. 9. Douglas Hofstadter, “The Shallowness of Google Translate,” The Atlantic Monthly, January 30, 2018. 10. Andy Martin, “Google
…
it easy to hire domestic freelancers, there is little to stop them from switching to lower cost foreign freelancers. As mentioned, the massive progress in machine translation, the rise of international freelancing platforms, and improved telecommunications is making telemigration a reality. As this catches on, the swapping foreign freelancers for domestic ones
…
real people who have to be in frequent in-person contact, since that is something telemigrants can’t do. Digital technology—especially advanced communication technologies, machine translation, and online international freelancing platforms—are making is easy for talented, low-cost foreigners sitting abroad to undertake many tasks in our offices. Which tasks
…
surely be important in the fast-moving, future world-of-work. Language skills, by contrast, will provide less of an advantage than they did before machine translation got so good. Consider an example of how globots changed the meaning of success in the law profession. Until recently, a law degree and a
Future Politics: Living Together in a World Transformed by Tech
by Jamie Susskind · 3 Sep 2018 · 533pp
preferred definition would be wider than mine (including manual and emotional tasks as well). 3. Yonghui Wu et al. ‘Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation’, arXiv, 8 October 2016 <https://arxiv.org/abs/1609.08144> (accessed 6 December 2017); Yaniv Taigman et al.,‘DeepFace: Closing the
…
, 2010. Wu, Yonghui, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammed Norouzi, Wolfgang Macherey, Maxim Krikun, et al. ‘Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation’. arXiv, 8 Oct. 2016 <https://arxiv.org/abs/ 1609.08144> (accessed 6 Dec. 2017). Xiong, Wei, Jasha Droppo, Xupeng Huang, Frank Seide
Natural Language Annotation for Machine Learning
by James Pustejovsky and Amber Stubbs · 14 Oct 2012 · 502pp · 107,510 words
coherent summary of their content. Such programs also aim to provide snap “elevator summaries” of longer documents, and possibly even turn them into slide presentations. Machine Translation The holy grail of NLP applications, this was the first major area of research and engineering in the field. Programs such as Google Translate are
…
a standard for the representation of texts in digital form. 2000s: As the World Wide Web grows, more data is available for statistical models for Machine Translation and other applications. The American National Corpus (ANC) project releases a 22-million-word subcorpus, and the Corpus of Contemporary American English (COCA) is released
…
Hidden Markov Models [HMMS]) that worked well enough to recognize a limited vocabulary of words in a very narrow domain. In the 1990s, work in Machine Translation began to see the influence of larger and larger datasets, and with this, the rise of statistical language modeling for translation. Eventually, both memory and
…
following sentences, is “clean dishes” a noun phrase or an imperative verb phrase? Clean dishes are in the cabinet. Clean dishes before going to work! Machine Translation Getting the POS tags and the subsequent parse right makes all the difference when translating the expressions in the preceding list item into another language
…
correct parts of speech in a sentence is a necessary step in building many natural language applications, such as parsers, Named Entity Recognizers, QAS, and Machine Translation systems. It is also an important step toward identifying larger structural units such as phrase structure. Note Use the NLTK tagger to assign POS tags
…
,VPZ), Dom(VP,NP2), Dom(S,VP), Prec(NP1,VP), Prec(VPZ,NP2)} Any sophisticated natural language application requires some level of syntactic analysis, including Machine Translation. If the resources for full parsing (such as that shown earlier) are not available, then some sort of shallow parsing can be used. This is
…
following points: Natural language annotation is an important step in the process of training computers to understand human speech for tasks such as Question Answering, Machine Translation, and summarization. All of the layers of linguistic research, from phonetics to semantics to discourse analysis, are used in different combinations for different ML tasks
…
three years as part of the Association for Computational Linguistics. It involves a variety of challenges including word sense disambiguation, temporal and spatial reasoning, and Machine Translation. Conference on Natural Language Learning (CoNLL) Shared Task This is a yearly NLP challenge held as part of the Special Interest Group on Natural Language
…
are important for a wide range of applications in Natural Language Processing (NLP), because fairly straightforward language models can be built using them, for speech, Machine Translation, indexing, Information Retrieval (IR), and, as we will see, classification. Imagine that we have a string of tokens, W, consisting of the elements w1, w2
…
statistical language models that predict sequence behaviors. Sequence behavior is involved in recognizing the next X in a sequence of Xs; for example, Speech Recognition, Machine Translation, and so forth. Language modeling predicts the next element in a sequence, given the previously encountered elements. Let’s see more precisely just how this
…
at all, but rather can take immediate advantage of the features of the text as individual tokens. kNN techniques have been applied to work in Machine Translation and a number of other NLP problems, including semantic relation extraction (Panchenko et al. 2012). Support Vector Machine (SVM) is a binary classifier that takes
…
The Language Grid is “an online multilingual service platform which enables easy registration and sharing of language services such as online dictionaries, bilingual corpora, and machine translators.” In addition to providing a central repository for translation resources, it is affiliated with a number of language research projects, such as the Wikipedia Translation
…
/ Estonian Reference Corpus Modality: Written Language: Estonian URL: http://www.cl.ut.ee/korpused/segakorpus/index.php?lang=en Europarl—A Parallel Corpus for Statistical Machine Translation Modality: Written Languages: French, Italian, Spanish, Portuguese, Romanian, English, Dutch, German, Danish, Swedish, Bulgarian, Czech, Polish, Slovak, Slovene, Finnish, Hungarian, Estonian, Latvian, Lithuanian, Greek URL
…
: Structural markup, sentence boundaries, part-of-speech annotations, noun chunks, verb chunks URL: http://www.anc.org/OANC OPUS (Open Parallel Corpus) Modality: Written Use: Machine Translation Languages: Various URL: http://opus.lingfil.uu.se/ PAN-PC-11 Plagiarism Detection Corpus Modality: Written Language: English URL: http://www.uni-weimar.de/medien
…
.cuni.cz/tred/ WebAnnotator Modality: Websites Use: Web annotation Language: Language-independent URL: http://www.limsi.fr/Individu/xtannier/en/WebAnnotator/ WordAligner Modality: Written Use: Machine Translation word alignment Language: Language-independent URL: http://www.bultreebank.bas.bg/aligner/index.php Automated Annotation Tools Multipurpose tools fnTBL Modality: Written Use: Part-of
…
taggers/syntactic parsers Alpino Modality: Written Use: Dependency parser Language: Dutch URL: http://www.let.rug.nl/vannoord/alp/Alpino/ Apertium-kir Modality: Written Use: Machine Translation Languages: Various URL: http://sourceforge.net/projects/apertium/ Automatic Syntactic Analysis for Polish Language (ASA-PL) Modality: Written Use: Syntactic analysis Language: Polish URL: http
…
: Written Use: Coreference/anaphora resolution Language: English URL: http://www.bart-coref.org/ GIZA++ Modality: Written Use: Machine Translation Languages: Various URL: http://code.google.com/p/giza-pp/ Google Translate Modality: Written Use: Machine Translation Languages: Various URL: http://www.translate.google.com HeidelTime Modality: Written Use: Temporal expression tagger Languages: Various URL
…
in Computing Systems. Kunchukuttan, Anoop, Shourya Roy, Pratik Patel, Kushal Ladha, Somya Gupta, Mitesh M. Khapra, and Pushpak Bhattacharyya. 2012. “Experiences in Resource Generation for Machine Translation through Crowdsourcing.” In Proceedings of the 8th International Conference on Language Resources and Evaluation (LREC’12), Istanbul, Turkey. Marujo, Luís, Anatole Gershman, Jaime Carbonell, Robert
Found in Translation: How Language Shapes Our Lives and Transforms the World
by Nataly Kelly and Jost Zetzsche · 1 Oct 2012 · 274pp · 73,344 words
realized that the daily diet of approximately four thousand original articles with potentially relevant content could be handled only with a mixture of computerized or machine translation and appropriate human oversight. So the developers chose several software programs to automatically translate information in the various language combinations. Once the articles are
…
translation tools to translate content from other Wikipedias is possible but does not always work very well. “Efforts to machine-translate Wikipedia articles and then bring in volunteers to build on top of those machine translations have not been particularly successful,” Jay explains. Wikipedia currently boasts more than twenty million articles across all languages
…
context. And just like translation memories, termbases can be shared among many translators working in real time in virtual teams. Finally, some translators also use machine translation, in which a software program or online tool automatically translates the text according to its own set of rules. Believe it or not, this can
…
the correct vocabulary, and tweak their understanding of grammar. Automated translation does have its place, especially in certain industries. For example, in the legal field, machine translation is often used to mine extensive amounts of data, such as case law, to flag items that might be relevant for attorneys working on a
…
given case. In the manufacturing sector, machine translation is sometimes used for the extensive support content and documentation that often has a high degree of predictable structure and repeated terms and phrases. In
…
game, go to www.translationparty.com, a site that keeps on translating between Japanese and English until an equilibrium is reached.) One famous example of machine translation gone awry is actually an urban legend. As the story goes, the sentence “The spirit is willing, but the flesh is weak” was plugged into
…
a machine translation system to be rendered into Russian. Allegedly, the computer produced “The vodka is strong, but the meat is rotten” in Russian. This tale has never
…
been substantiated, but it’s not completely inconceivable. The story probably serves a good purpose as a warning that generic machine translation cannot and should not be blindly trusted. Parlez-Vous C++? Anyone who’s taken a language course in school knows how hard it is to
…
, users typed in the words “Google Translate.”20 This means that half of all Google users who are interested in translation automatically turn to the machine translation tool that Google offers. Surprised by that number? That probably just means you’re a native (or competent) English speaker. You see, if you search
…
the best way to translate a given phrase or paragraph by doing what it does best—crunching lots of numbers. This approach, known as statistical machine translation, feeds computers with very large amounts of language data. With the help of ever-more-sophisticated algorithms, the computers process these data and then employ
…
a team at Carnegie Mellon University and other sources to release a version of Haitian Creole within days. (Microsoft used the same material for its machine translation engine and released the Haitian Creole version at around the same time.) Though it wouldn’t have passed the company’s quality threshold under other
…
little content on the web. The team employs everything that is deemed useful (with the exception of translations produced by Google’s own or other machine translation programs) to continuously train existing and new engines. And the results? It all depends on the language pair and the expectation. For language pairs like
…
written—better than they do today. He points to the fact that automated translation options like Google Translate are already available on mobile phones. However, machine translation will never be perfect. Many of the stories we’ve shared so far in this book make it clear that the tasks of translation and
…
human intelligence is our ability to command language. He even characterizes translation as “the most high-level type of work one can imagine.” Tools like machine translation, he says, will only boost humans’ ability to use, transform, and manipulate language. And, as with many things in Kurzweil’s career, it all comes
…
, 191 Lucas, George, 177 Luhtanen, Sari, 178–79 Luther, Martin, 119–22 luxury brands, 144–45 MacArthur Fellowship (Genius Grant), 29 MAC Cosmetics, 142–44 machine translation, 77, 203–4, 218–19, 226, 227–28, 229, 231 Maguire, Sarah, 105 Mahayana Buddhism, 115, 116 Mahfouz, Naguib, 99 Majd, Hooman, 47 Makepeace, Anne
…
Starbucks, 63, 65 Stargate (TV show), 223 Star Trek (movies and TV show), 223, 229 starving artists, 93–95 Star Wars (movies), 177, 223 statistical machine translation, 227–28 Steiner, George, 123 storytelling and religion in translation, 93–122 Street Fighter II (video game), 181 Stylesight network, 146–47 Sudan, 41–43
The Creativity Code: How AI Is Learning to Write, Paint and Think
by Marcus Du Sautoy · 7 Mar 2019 · 337pp · 103,522 words
be relevant when we come later to the idea of the Chinese room experiment devised by John Searle. This thought experiment explores the idea of machine translation and tries to illustrate why following rules doesn’t show intelligence or understanding. Nevertheless, follow the rules of the mathematical game and you get mathematical
Artificial Intelligence: A Guide for Thinking Humans
by Melanie Mitchell · 14 Oct 2019 · 350pp · 98,077 words
human language.” (In AI-speak, “natural” means “human.”) Natural-language processing (abbreviated NLP) includes topics such as speech recognition, web search, automated question answering, and machine translation. Similar to what we’ve seen in previous chapters, deep learning has been the driving force behind most of the recent advances in NLP. I
…
in AI, such “decoding” turned out to be harder than people originally expected. Like other AI research in the early days, the original approaches to machine translation relied on complicated sets of human-specified rules. With the goal of translating from a source language (for example, English) to a target language (for
…
instance, Russian), a machine-translation system would be given syntax rules for both languages as well as rules for mappings between syntactical structures. In addition, human programmers would create dictionaries
…
for the machine-translation system with word-to-word (and simple phrase-to-phrase) equivalences. Like many other efforts in symbolic AI, while these approaches worked well in some
…
Nations transcripts, which are translated into the six official languages of the UN, and from other large sets of original and translated documents. The statistical machine-translation systems of the 1990s to the 2000s typically computed large tables of probabilities linking phrases in the source and target languages. When given a new
…
worked together as a sentence, but the main driver of the translation was the probabilities of phrases learned from the training data. Even though statistical machine-translation systems had very little knowledge of syntax in either language, on the whole these methods produced better translations than the earlier rule-based approaches. Google
…
Translate—probably the most widely used automated-translation program—employed these kinds of statistical machine-translation methods from the time of its launch in 2006 until 2016, at which time Google researchers had developed what they claimed was a superior translation
…
method based on deep learning, called neural machine translation. Soon after, neural machine translation was adopted for all state-of-the-art machine-translation programs. Encoder, Meet Decoder Figure 38 gives a sketch of what’s under the hood when you use Google
…
neural network. While figure 38 shows the encoder and decoder networks abstractly as white rectangles, such networks are actually made up of LSTM units. Automated machine translation in the deep-learning age is a triumph of big data and fast computation. To create a pair of encoder-decoder networks to translate from
…
believe is that neural networks are learning the underlying semantic meaning of the language.”14 The CEO of the specialty translation company DeepL bragged, “Our [machine-translation] neural networks have developed an astounding sense of understanding.”15 In general, such declarations are in part fueled by the race among tech companies to
…
company and you want to translate a large volume of documents or provide translation for customers on your websites, you can find many fee-based machine-translation services available, all powered by the same encoder-decoder architecture. To what extent should we believe the claims that machines are actually learning “semantic meaning
…
” or that machine translation is swiftly closing in on human levels of accuracy? To answer this, let’s look more closely at the actual results these claims are based
…
to design an automatic method for computing the system’s accuracy. The claims of “human parity” and “bridging the gap between machines and humans” in machine translation are based on two methods of evaluating translation results. The first is an automated method—a computer program—that compares a machine’s translation with
…
out a score. The second method employs bilingual humans to manually evaluate translations. For the first method, the program used in virtually all evaluations of machine translation is called bilingual evaluation understudy, or BLEU.16 To measure the quality of a translation, BLEU essentially counts the number of matches—between words and
…
phrases of varying lengths—in a machine-translated sentence and one or more human-created “reference” (that is, “correct”) translations. While the ratings produced by BLEU often correlate with human judgments of translation
…
quality, BLEU tends to overrate bad translations. Several machine-translation researchers have told me that BLEU is a flawed way to evaluate translations, used only because no one has yet found an automatic method that
…
works better in general. Given the drawbacks of BLEU, the “gold standard” for evaluating a machine-translation system is for bilingual humans to manually rate the translations produced by the system. These same human evaluators can also rate corresponding translations created by
…
professional human translators in order to compare with the machine-translation ratings. But there are also drawbacks to this gold-standard approach: hiring humans costs money, of course, and unlike computers humans get tired after rating
…
you can hire an army of bilingual human raters who have a lot of time on their hands, your evaluation process will be limited. The machine-translation groups at both Google and Microsoft carried out this kind of gold-standard (albeit limited) evaluation by hiring small groups of bilingual human evaluators to
…
set of sentences in a source language, along with translations of those sentences into the target language. The translations were created both by the neural machine-translation system and by professional human translators. Google’s evaluation consisted of about five hundred sentences from news stories and from Wikipedia articles in several different
…
each evaluator’s ratings over all sentences, and then averaging over the evaluators, the Google researchers found that the average rating given to their neural machine-translation system was close to (though below) the ratings given to the human-translated sentences. This was the case for all of the language pairs in
…
. Microsoft used a similar averaging method to evaluate translations of news stories from Chinese to English. The ratings of the translations by Microsoft’s neural machine-translation system were very close to (and sometimes even exceeded) the ratings of the human translations. In all cases, the human evaluators rated the translations produced
…
one another in important ways that can be missed if the sentences are translated in isolation. I haven’t seen any formal studies of evaluating machine translation for longer passages, but my general experience is that the translation quality of, say, Google Translate declines significantly when it is given whole paragraphs instead
…
from news stories and Wikipedia pages, which are typically written with care to avoid ambiguous or idiomatic language; such language can cause serious problems for machine-translation systems. Lost in Translation Remember my “Restaurant” story from the beginning of the previous chapter? I didn’t design that story to test translation systems
…
, but the story actually does a good job of illustrating the challenges presented to machine-translation systems by colloquial, idiomatic, and potentially ambiguous language. I used Google Translate to translate the “Restaurant” story from English into three target languages: French, Italian
…
improved, and some of the specific translation errors seen here may be fixed by the time you are reading this. However, I’m skeptical that machine translation will actually reach the level of human translators—except perhaps in narrow circumstances—for a long time to come. The main obstacle is this: like
…
speech-recognition systems, machine-translation systems perform their task without actually understanding the text they are processing.21 In translation as well as in speech recognition, the question remains: To
…
for his meal than about “proposed legislation.” Hofstadter’s words were echoed in a recent article by the AI researchers Ernest Davis and Gary Marcus: “Machine translation … often involves problems of ambiguity that can only be resolved by achieving an actual understanding of the text—and bringing real-world knowledge to bear
…
is still an open question and is the subject of intense debate in the AI community. For now, I’ll simply say that while neural machine translation can be impressively effective and useful in many applications, the translations, without post-editing by knowledgeable humans, are still fundamentally unreliable. If you use
…
Symposium on Intelligent Data Analysis (2018), 328–39. 12: Translation as Encoding and Decoding 1. Q. V. Le and M. Schuster, “A Neural Network for Machine Translation, at Production Scale,” AI Blog, Google, Sept. 27, 2016, ai.googleblog.com/2016/09/a-neural-network-for-machine.html. 2. W. Weaver, “Translation,” in
…
Machine Translation of Languages, ed. W. N. Locke and A. D. Booth (New York: Technology Press and John Wiley & Sons, 1955), 15–23. 3. This is the
…
for some less common languages. 4. For more details, see Y. Wu et al., “Google’s Neural Machine Translation System: Bridging the Gap Between Human and Machine Translation,” arXiv:1609.08144 (2016). 5. In Google’s neural machine-translation system, the word vectors are learned as part of the training of the entire network. 6. More
…
network are probabilities for each possible word in the network’s vocabulary (here, French). More details are given in Wu et al., “Google’s Neural Machine Translation System.” 7. At the time of this writing, Google Translate and other translation systems work by translating one sentence at a time. An example of
…
on going beyond sentence-by-sentence translation is described in L. M. Werlen and A. Popescu-Belis, “Using Coreference Links to Improve Spanish-to-English Machine Translation,” in Proceedings of the 2nd Workshop on Coreference Resolution Beyond OntoNotes (2017), 30–40. 8. S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural
…
Computation 9, no. 8 (1997): 1735–80. 9. Wu et al., “Google’s Neural Machine Translation System.” 10. Ibid. 11. T. Simonite, “Google’s New Service Translates Languages Almost as Well as Humans Can,” Technology Review, Sept. 27, 2016, www.technologyreview
…
Historic Milestone, Using AI to Match Human Performance in Translating News from Chinese to English,” AI Blog, Microsoft, March 14, 2018, blogs.microsoft.com/ai/machine-translation-news-test-set-human-parity. 13. “IBM Watson Is Now Fluent in Nine Languages (and Counting),” Wired, Oct. 6, 2016, www.wired.co.uk/article
…
of; inspiration from neuroscience; lack of reliability; as narrow AI; need for big data; see also convolutional neural networks; encoder-decoder system; encoder networks; neural machine translation; recurrent neural networks DeepMind; acquisition by Google; see also AlphaGo; Breakout deep neural networks, see deep learning deep Q-learning; adversarial examples for; on Breakout
…
Go (board game); see also AlphaGo Gödel, Escher, Bach (book) GOFAI Good, I. J. Goodfellow, Ian Google DeepMind, see DeepMind Google Translate; see also neural machine translation Gottschall, Jonathan GPS, see General Problem Solver GPUs, see graphical processing units gradient descent graphical processing units H HAL Hassabis, Demis Hawking, Stephen Hearst, Eliot
…
adversarial learning; bias in, see bias; interpretable, see explainable AI; overfitting in, see overfitting; transfer learning in, see transfer learning machine morality, see moral AI machine translation; comparison between humans and machines; evaluating; neural; statistical; see also Google Translate Manning, Christopher Marcus, Gary Markoff, John Marshall, James McCarthy, John McClelland, James Mechanical
…
: adversarial attacks on; challenges for; definition of; rule-based approaches to; statistical approaches to; see also machine translation; question answering; reading comprehension; sentiment classification; speech recognition; word vectors neocognitron network neural engineering neural machine translation; see also Google Translate; machine translation neural networks: activations in; classification in; convolutional, see convolutional neural networks; deep, see deep learning
…
, B. F. Smith, Brad speech recognition; adversarial examples for; word-error rate in Stanford Question Answering Dataset (SQuAD); human accuracy on Star Trek computer statistical machine translation strong AI; see also general or human-level AI subsymbolic AI; contrast with symbolic methods; integration with symbolic methods suitcase words Summer Vision Project (MIT
…
difference learning; see also reinforcement learning test set theory of mind thought vectors training, see supervised learning training set transfer learning; for Breakout translation, see machine translation trolley problem Turing, Alan Turing test; Kurzweil and Kapor wager on; Kurzweil’s predictions for U understanding: in analogy; ascribing to computers; in automated image
…
captioning; for creativity; in Cyc; in deep learning; in humans; in IBM Watson; in machine translation; for morality; for natural-language processing; in question-answering systems; for self-driving cars; in speech-recognition systems; in Star Trek computer; for vision; 263
The Industries of the Future
by Alec Ross · 2 Feb 2016 · 364pp · 99,897 words
me in response. It’s basically good enough to ask where the bathroom is and then hope somebody points in the right direction. Today’s machine translation is leaps and bounds faster and more effective than my old dictionary method, but it still falls short in accuracy, functionality, and delivery. In essence
…
amount of data that informs translation grows exponentially, the machines will grow exponentially more accurate and be able to parse the smallest detail. Whenever the machine translations get it wrong, users can flag the error—and that data too will be incorporated into future attempts. We just need more data, more computing
…
the passage of time and will fill in the communication gaps in areas including pronunciation and interpreting a spoken response. The most interesting innovations in machine translation will come with the human interface. In ten years, a small earpiece will whisper what is being said to you in your native language near
…
the personal device of 2025 is. Today’s translation tools also tend to move only between two languages. Try to engage in any sort of machine translation exercise involving three languages, and it is an incoherent mess. In the future, the number of languages being spoken will not matter. You could host
…
at the table speaking eight different languages, and the voice in your ear will always be whispering the one language you want to hear. Universal machine translation will accelerate globalization on a massive scale. While the current stage of globalization was propelled in no small part by the adoption of English as
…
longer will there be this need, opening the door for nonelites and a massive number of non-English speakers to the world of global business. Machine translation will also take markets that are viewed as being difficult to navigate because of language barriers and make them more accessible. I think of Indonesia
…
will take economically isolated parts of the world and help fold them into the global economy. As with any new technology, the rise of universal machine translation will also have its downsides—and two in particular come to mind. The first is the near-obliteration of a profession. The only professional translators
…
in ten years are going to be the people who work on the translation software. Most machine translation programs (such as Google’s) continue to rely heavily on human translations, but once the data sets of translations are large enough, the translators won
…
explanatory suppleness of even a mediocre novel.” It is also the case that while analyzing ever-larger data sets will produce outcomes like near-perfect machine translation, it will also produce a larger number of spurious correlations. The larger and more expansive the data sets, the more correlations there are, both spurious
…
, 185 precision agriculture, 161–66, 181, 191–93 Rwanda and, 238 Soviet Union and, 68 Tanzania and, 235 technology and, 3, 5, 160–62 universal machine translation and, 160 Airbnb, 91–97 AIST, 17 Alexander, Keith, 129 Alibaba, 82, 228 AltaVista, 119 Amazon, 4, 31, 48, 90, 93, 98, 157 Andela, 234
…
/Harvard Cancer Center, 72 data: agriculture and, 161–66 business and, 172–74 concerns about 179–82 finance and, 166–72 future of, 182–85 machine translation and, 158–61 overview, 152–57 permanence, 175 privacy and, 174–79 Davis, Ronald W., 48, 60, 74 distributed denial-of-service (DDoS) attacks, 125
…
, 249 innovation and, 195, 204, 215, 233, 249 jobs and, 23, 38–39 medicine and, 72 right side of, 11–12 robots and, 42 universal machine translation and, 159 women and, 227 wrong side of, 1–7 Glodek, William, 146 gold, 112, 114, 118 Goldberg, Ken, 27, 33, 35 Goldman Sachs, 113
Hands-On Machine Learning With Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems
by Aurélien Géron · 13 Mar 2017 · 1,331pp · 163,200 words
speedup for sparser models trained across 500 GPUs. As you can see, sparse models really do scale better. Here are a few concrete examples: Neural Machine Translation: 6x speedup on 8 GPUs Inception/ImageNet: 32x speedup on 50 GPUs RankBrain: 300x speedup on 500 GPUs These numbers represent the state of the
…
. Along the way, as always, we will show how to implement RNNs using TensorFlow. Finally, we will take a look at the architecture of a machine translation system. Recurrent Neurons Up to now we have mostly looked at feedforward neural networks, where the activations flow only in one direction, from the input
…
recent years, in particular for applications in natural language processing (NLP). Natural Language Processing Most of the state-of-the-art NLP applications, such as machine translation, automatic summarization, parsing, sentiment analysis, and more, are now based (at least in part) on RNNs. In this last section, we will take a quick
…
look at what a machine translation model looks like. This topic is very well covered by TensorFlow’s awesome Word2Vec and Seq2Seq tutorials, so you should definitely check them out. Word
…
, and so on. You now have almost all the tools you need to implement a machine translation system. Let’s look at this now. An Encoder–Decoder Network for Machine Translation Let’s take a look at a simple machine translation model10 that will translate English sentences to French (see Figure 14-15). Figure 14-15
…
. A simple machine translation model The English sentences are fed to the encoder, and the decoder outputs the
…
peek into the input sequence. Attention augmented RNNs are beyond the scope of this book, but if you are interested there are helpful papers about machine translation,13 machine reading,14 and image captions15 using attention. Finally, the tutorial’s implementation makes use of the tf.nn.legacy_seq2seq module, which provides
…
al. (2015). 6 “Recurrent Nets that Time and Count,” F. Gers and J. Schmidhuber (2000). 7 “Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation,” K. Cho et al. (2014). 8 A 2015 paper by Klaus Greff et al., “LSTM: A Search Space Odyssey,” seems to show that all LSTM
…
et al. (2014). 11 The bucket sizes used in the tutorial are different. 12 “On Using Very Large Target Vocabulary for Neural Machine Translation,” S. Jean et al. (2015). 13 “Neural Machine Translation by Jointly Learning to Align and Translate,” D. Bahdanau et al. (2014). 14 “Long Short-Term Memory-Networks for Machine Reading
…
. Chapter 14: Recurrent Neural Networks Here are a few RNN applications: For a sequence-to-sequence RNN: predicting the weather (or any other time series), machine translation (using an encoder–decoder architecture), video captioning, speech to text, music generation (or other sequence generation), identifying the chords of a song. For a sequence
…
Assumptions asynchronous updates, Asynchronous updates-Asynchronous updates asynchrous communication, Asynchronous Communication Using TensorFlow Queues-PaddingFifoQueue atrous_conv2d(), ResNet attention mechanism, An Encoder–Decoder Network for Machine Translation attributes, Supervised learning, Take a Quick Look at the Data Structure-Take a Quick Look at the Data Structure(see also data structure) combinations of
…
unrolling through time, Dynamic Unrolling Through Time dynamic_rnn(), Dynamic Unrolling Through Time, Distributing a Deep RNN Across Multiple GPUs, An Encoder–Decoder Network for Machine Translation E early stopping, Early Stopping-Early Stopping, Gradient Boosting, Number of Neurons per Hidden Layer, Early Stopping Elastic Net, Elastic Net embedded device blocks, Sharding
…
-v4, ResNet incremental learning, Online learning, Incremental PCA inequality constraints, SVM Dual Problem inference, Model-based learning, Exercises, Memory Requirements, An Encoder–Decoder Network for Machine Translation info(), Take a Quick Look at the Data Structure information gain, Gini Impurity or Entropy? information theory, Gini Impurity or Entropy? init node, Saving and
…
, Instance-Based Versus Model-Based Learning-Model-based learning supervised/unsupervised learning, Supervised/Unsupervised Learning-Reinforcement Learning workflow example, Model-based learning-Model-based learning machine translation (see natural language processing (NLP)) make(), Introduction to OpenAI Gym Manhattan norm, Select a Performance Measure manifold assumption/hypothesis, Manifold Learning Manifold Learning, Manifold Learning
…
Neural Networks, Natural Language Processing-An Encoder–Decoder Network for Machine Translationencoder-decoder network for machine translation, An Encoder–Decoder Network for Machine Translation-An Encoder–Decoder Network for Machine Translation TensorFlow tutorials, Natural Language Processing, An Encoder–Decoder Network for Machine Translation word embeddings, Word Embeddings-Word Embeddings Nesterov Accelerated Gradient (NAG), Nesterov Accelerated Gradient-Nesterov Accelerated
…
and Output Sequences-Input and Output Sequences LSTM cell, LSTM Cell-GRU Cell natural language processing (NLP), Natural Language Processing-An Encoder–Decoder Network for Machine Translation in TensorFlow, Basic RNNs in TensorFlow-Handling Variable-Length Output Sequencesdynamic unrolling through time, Dynamic Unrolling Through Time static unrolling through time, Static Unrolling Through
…
run(), Creating Your First Graph and Running It in a Session, In-Graph Versus Between-Graph Replication S Sampled Softmax, An Encoder–Decoder Network for Machine Translation sampling bias, Nonrepresentative Training Data-Poor-Quality Data, Create a Test Set sampling noise, Nonrepresentative Training Data save(), Saving and Restoring Models Saver node, Saving
…
Networks separable_conv2d(), ResNet sequences, Recurrent Neural Networks sequence_length, Handling Variable Length Input Sequences-Handling Variable-Length Output Sequences, An Encoder–Decoder Network for Machine Translation Shannon's information theory, Gini Impurity or Entropy? shortcut connections, ResNet show(), Take a Quick Look at the Data Structure show_graph(), Visualizing the Graph
…
through time, Static Unrolling Through Time-Static Unrolling Through Time static_rnn(), Static Unrolling Through Time-Static Unrolling Through Time, An Encoder–Decoder Network for Machine Translation stationary point, SVM Dual Problem-SVM Dual Problem statistical mode, Bagging and Pasting statistical significance, Regularization Hyperparameters stemming, Exercises step functions, The Perceptron step(), Introduction
…
, Take a Quick Look at the Data Structure target attributes, Take a Quick Look at the Data Structure target_weights, An Encoder–Decoder Network for Machine Translation tasks, Multiple Devices Across Multiple Servers Temporal Difference (TD) Learning, Temporal Difference Learning and Q-Learning-Temporal Difference Learning and Q-Learning tensor processing units
…
Momentum optimization in, Momentum optimization name scopes, Name Scopes neural network policies, Neural Network Policies NLP tutorials, Natural Language Processing, An Encoder–Decoder Network for Machine Translation node value lifecycle, Lifecycle of a Node Value operations (ops), Linear Regression with TensorFlow optimizer, Using an Optimizer overview, Up and Running with TensorFlow-Up
…
a Deep RNN Across Multiple GPUs tf.contrib.rnn.static_rnn(), Basic RNNs in TensorFlow-Handling Variable Length Input Sequences, An Encoder–Decoder Network for Machine Translation-Exercises, Chapter 14: Recurrent Neural Networks-Chapter 14: Recurrent Neural Networks tf.contrib.slim module, Up and Running with TensorFlow, Exercises tf.contrib.slim.nets
…
Through Time, Training a Sequence Classifier, Training to Predict Time Series, Training to Predict Time Series, Deep RNNs-Applying Dropout, An Encoder–Decoder Network for Machine Translation-Exercises, Chapter 14: Recurrent Neural Networks-Chapter 14: Recurrent Neural Networks tf.nn.elu(), Nonsaturating Activation Functions, TensorFlow Implementation-Tying Weights, Variational Autoencoders, Neural Network
…
Dropout The Difficulty of Training over Many Time Steps LSTM Cell Peephole Connections GRU Cell Natural Language Processing Word Embeddings An Encoder–Decoder Network for Machine Translation Exercises 15. Autoencoders Efficient Data Representations Performing PCA with an Undercomplete Linear Autoencoder Stacked Autoencoders TensorFlow Implementation Tying Weights Training One Autoencoder at a Time
The Technology Trap: Capital, Labor, and Power in the Age of Automation
by Carl Benedikt Frey · 17 Jun 2019 · 626pp · 167,836 words
What to Think About Machines That Think: Today's Leading Thinkers on the Age of Machine Intelligence
by John Brockman · 5 Oct 2015 · 481pp · 125,946 words
Ghost Work: How to Stop Silicon Valley From Building a New Global Underclass
by Mary L. Gray and Siddharth Suri · 6 May 2019 · 346pp · 97,330 words
Thinking Machines: The Inside Story of Artificial Intelligence and Our Race to Build the Future
by Luke Dormehl · 10 Aug 2016 · 252pp · 74,167 words
In the Plex: How Google Thinks, Works, and Shapes Our Lives
by Steven Levy · 12 Apr 2011 · 666pp · 181,495 words
Kingdom of Characters: The Language Revolution That Made China Modern
by Jing Tsu · 18 Jan 2022 · 408pp · 105,715 words
The Age of AI: And Our Human Future
by Henry A Kissinger, Eric Schmidt and Daniel Huttenlocher · 2 Nov 2021 · 194pp · 57,434 words
AI in Museums: Reflections, Perspectives and Applications
by Sonja Thiel and Johannes C. Bernhardt · 31 Dec 2023 · 321pp · 113,564 words
The Inevitable: Understanding the 12 Technological Forces That Will Shape Our Future
by Kevin Kelly · 6 Jun 2016 · 371pp · 108,317 words
In Our Own Image: Savior or Destroyer? The History and Future of Artificial Intelligence
by George Zarkadakis · 7 Mar 2016 · 405pp · 117,219 words
Accelerando
by Stross, Charles · 22 Jan 2005 · 489pp · 148,885 words
The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World
by Pedro Domingos · 21 Sep 2015 · 396pp · 117,149 words
Rule of the Robots: How Artificial Intelligence Will Transform Everything
by Martin Ford · 13 Sep 2021 · 288pp · 86,995 words
These Strange New Minds: How AI Learned to Talk and What It Means
by Christopher Summerfield · 11 Mar 2025 · 412pp · 122,298 words
Paper Knowledge: Toward a Media History of Documents
by Lisa Gitelman · 26 Mar 2014
The Man Who Invented the Computer
by Jane Smiley · 18 Oct 2010 · 253pp · 80,074 words
Surfaces and Essences
by Douglas Hofstadter and Emmanuel Sander · 10 Sep 2012 · 1,079pp · 321,718 words
Track Changes
by Matthew G. Kirschenbaum · 1 May 2016 · 519pp · 142,646 words
The Dream Machine: J.C.R. Licklider and the Revolution That Made Computing Personal
by M. Mitchell Waldrop · 14 Apr 2001
The Most Human Human: What Talking With Computers Teaches Us About What It Means to Be Alive
by Brian Christian · 1 Mar 2011 · 370pp · 94,968 words
Final Jeopardy: Man vs. Machine and the Quest to Know Everything
by Stephen Baker · 17 Feb 2011 · 238pp · 77,730 words
AI Superpowers: China, Silicon Valley, and the New World Order
by Kai-Fu Lee · 14 Sep 2018 · 307pp · 88,180 words
WTF?: What's the Future and Why It's Up to Us
by Tim O'Reilly · 9 Oct 2017 · 561pp · 157,589 words
The AI Economy: Work, Wealth and Welfare in the Robot Age
by Roger Bootle · 4 Sep 2019 · 374pp · 111,284 words
The Equality Machine: Harnessing Digital Technology for a Brighter, More Inclusive Future
by Orly Lobel · 17 Oct 2022 · 370pp · 112,809 words
The Seventh Sense: Power, Fortune, and Survival in the Age of Networks
by Joshua Cooper Ramo · 16 May 2016 · 326pp · 103,170 words
The Information: A History, a Theory, a Flood
by James Gleick · 1 Mar 2011 · 855pp · 178,507 words
The Caryatids
by Bruce Sterling · 24 Feb 2009 · 387pp · 105,250 words
The Future of the Professions: How Technology Will Transform the Work of Human Experts
by Richard Susskind and Daniel Susskind · 24 Aug 2015 · 742pp · 137,937 words
The Last Lingua Franca: English Until the Return of Babel
by Nicholas Ostler · 23 Nov 2010 · 484pp · 120,507 words
The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant From Two Centuries of Controversy
by Sharon Bertsch McGrayne · 16 May 2011 · 561pp · 120,899 words
Is That a Fish in Your Ear?: Translation and the Meaning of Everything
by David Bellos · 10 Oct 2011 · 396pp · 107,814 words
Talk on the Wild Side
by Lane Greene · 15 Dec 2018 · 284pp · 84,169 words
New Dark Age: Technology and the End of the Future
by James Bridle · 18 Jun 2018 · 301pp · 85,263 words
Text Analytics With Python: A Practical Real-World Approach to Gaining Actionable Insights From Your Data
by Dipanjan Sarkar · 1 Dec 2016
Architects of Intelligence
by Martin Ford · 16 Nov 2018 · 586pp · 186,548 words
Genius Makers: The Mavericks Who Brought A. I. To Google, Facebook, and the World
by Cade Metz · 15 Mar 2021 · 414pp · 109,622 words
Only Humans Need Apply: Winners and Losers in the Age of Smart Machines
by Thomas H. Davenport and Julia Kirby · 23 May 2016 · 347pp · 97,721 words
GCHQ
by Richard Aldrich · 10 Jun 2010 · 826pp · 231,966 words
Race Against the Machine: How the Digital Revolution Is Accelerating Innovation, Driving Productivity, and Irreversibly Transforming Employment and the Economy
by Erik Brynjolfsson · 23 Jan 2012 · 72pp · 21,361 words
We Are Data: Algorithms and the Making of Our Digital Selves
by John Cheney-Lippold · 1 May 2017 · 420pp · 100,811 words
Automate This: How Algorithms Came to Rule Our World
by Christopher Steiner · 29 Aug 2012 · 317pp · 84,400 words
Big Data: A Revolution That Will Transform How We Live, Work, and Think
by Viktor Mayer-Schonberger and Kenneth Cukier · 5 Mar 2013 · 304pp · 82,395 words
The Invisible Web: Uncovering Information Sources Search Engines Can't See
by Gary Price, Chris Sherman and Danny Sullivan · 2 Jan 2003 · 481pp · 121,669 words
Fluent Forever: How to Learn Any Language Fast and Never Forget It
by Gabriel Wyner · 4 Aug 2014 · 366pp · 87,916 words
Structure and Interpretation of Computer Programs, Second Edition
by Harold Abelson, Gerald Jay Sussman and Julie Sussman · 1 Jan 1984 · 1,387pp · 202,295 words
Overcomplicated: Technology at the Limits of Comprehension
by Samuel Arbesman · 18 Jul 2016 · 222pp · 53,317 words
Facebook: The Inside Story
by Steven Levy · 25 Feb 2020 · 706pp · 202,591 words
There's a War Going on but No One Can See It
by Huib Modderkolk · 1 Sep 2021 · 295pp · 84,843 words
Machine Learning Design Patterns: Solutions to Common Challenges in Data Preparation, Model Building, and MLOps
by Valliappa Lakshmanan, Sara Robinson and Michael Munn · 31 Oct 2020
Machine: A White Space Novel
by Elizabeth Bear · 5 Oct 2020 · 537pp · 146,610 words
I, Warbot: The Dawn of Artificially Intelligent Conflict
by Kenneth Payne · 16 Jun 2021 · 339pp · 92,785 words
Supremacy: AI, ChatGPT, and the Race That Will Change the World
by Parmy Olson · 284pp · 96,087 words
Free Speech: Ten Principles for a Connected World
by Timothy Garton Ash · 23 May 2016 · 743pp · 201,651 words
Narrative Economics: How Stories Go Viral and Drive Major Economic Events
by Robert J. Shiller · 14 Oct 2019 · 611pp · 130,419 words
Who Owns the Future?
by Jaron Lanier · 6 May 2013 · 510pp · 120,048 words
Nerds on Wall Street: Math, Machines and Wired Markets
by David J. Leinweber · 31 Dec 2008 · 402pp · 110,972 words
Babel No More: The Search for the World's Most Extraordinary Language Learners
by Michael Erard · 10 Jan 2012 · 392pp · 104,760 words
The Big Nine: How the Tech Titans and Their Thinking Machines Could Warp Humanity
by Amy Webb · 5 Mar 2019 · 340pp · 97,723 words
Cage of Souls
by Adrian Tchaikovsky · 4 Apr 2019 · 703pp · 196,052 words
The Alignment Problem: Machine Learning and Human Values
by Brian Christian · 5 Oct 2020 · 625pp · 167,349 words
Artificial Unintelligence: How Computers Misunderstand the World
by Meredith Broussard · 19 Apr 2018 · 245pp · 83,272 words
Sparks: China's Underground Historians and Their Battle for the Future
by Ian Johnson · 26 Sep 2023 · 407pp · 119,073 words
Literary Theory for Robots: How Computers Learned to Write
by Dennis Yi Tenen · 6 Feb 2024 · 169pp · 41,887 words
Who Owns This Sentence?: A History of Copyrights and Wrongs
by David Bellos and Alexandre Montagu · 23 Jan 2024 · 305pp · 101,093 words
Empire of AI: Dreams and Nightmares in Sam Altman's OpenAI
by Karen Hao · 19 May 2025 · 660pp · 179,531 words
Artificial Intelligence: A Modern Approach
by Stuart Russell and Peter Norvig · 14 Jul 2019 · 2,466pp · 668,761 words
Our Final Invention: Artificial Intelligence and the End of the Human Era
by James Barrat · 30 Sep 2013 · 294pp · 81,292 words
The Fractalist
by Benoit Mandelbrot · 30 Oct 2012
Superintelligence: Paths, Dangers, Strategies
by Nick Bostrom · 3 Jun 2014 · 574pp · 164,509 words
Pattern Recognition
by William Gibson · 2 Jan 2003 · 385pp · 99,985 words
Red Moon
by Kim Stanley Robinson · 22 Oct 2018 · 492pp · 141,544 words
Information: A Very Short Introduction
by Luciano Floridi · 25 Feb 2010 · 137pp · 36,231 words
Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again
by Eric Topol · 1 Jan 2019 · 424pp · 114,905 words
Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems
by Martin Kleppmann · 16 Mar 2017 · 1,237pp · 227,370 words
The Myth of Artificial Intelligence: Why Computers Can't Think the Way We Do
by Erik J. Larson · 5 Apr 2021
Life After Google: The Fall of Big Data and the Rise of the Blockchain Economy
by George Gilder · 16 Jul 2018 · 332pp · 93,672 words
Rise of the Robots: Technology and the Threat of a Jobless Future
by Martin Ford · 4 May 2015 · 484pp · 104,873 words
Future Crimes: Everything Is Connected, Everyone Is Vulnerable and What We Can Do About It
by Marc Goodman · 24 Feb 2015 · 677pp · 206,548 words
I'm Feeling Lucky: The Confessions of Google Employee Number 59
by Douglas Edwards · 11 Jul 2011 · 496pp · 154,363 words
Deep Thinking: Where Machine Intelligence Ends and Human Creativity Begins
by Garry Kasparov · 1 May 2017 · 331pp · 104,366 words
The Cultural Logic of Computation
by David Golumbia · 31 Mar 2009 · 268pp · 109,447 words
A New History of the Future in 100 Objects: A Fiction
by Adrian Hon · 5 Oct 2020 · 340pp · 101,675 words
Stealth
by Peter Westwick · 22 Nov 2019 · 474pp · 87,687 words
Reinventing Discovery: The New Era of Networked Science
by Michael Nielsen · 2 Oct 2011 · 400pp · 94,847 words
Eternity
by Greg Bear · 2 Jan 1988 · 523pp · 129,580 words
The Formula: How Algorithms Solve All Our Problems-And Create More
by Luke Dormehl · 4 Nov 2014 · 268pp · 75,850 words
Natural language processing with Python
by Steven Bird, Ewan Klein and Edward Loper · 15 Dec 2009 · 504pp · 89,238 words
Masterminds of Programming: Conversations With the Creators of Major Programming Languages
by Federico Biancuzzi and Shane Warden · 21 Mar 2009 · 496pp · 174,084 words
The Wealth of Humans: Work, Power, and Status in the Twenty-First Century
by Ryan Avent · 20 Sep 2016 · 323pp · 90,868 words
Ten Billion Tomorrows: How Science Fiction Technology Became Reality and Shapes the Future
by Brian Clegg · 8 Dec 2015 · 315pp · 92,151 words
Robot Rules: Regulating Artificial Intelligence
by Jacob Turner · 29 Oct 2018 · 688pp · 147,571 words
Talk to Me: How Voice Computing Will Transform the Way We Live, Work, and Think
by James Vlahos · 1 Mar 2019 · 392pp · 108,745 words
AIQ: How People and Machines Are Smarter Together
by Nick Polson and James Scott · 14 May 2018 · 301pp · 85,126 words
Children of Ruin
by Adrian Tchaikovsky · 13 May 2019 · 471pp · 147,210 words
The Means of Prediction: How AI Really Works (And Who Benefits)
by Maximilian Kasy · 15 Jan 2025 · 209pp · 63,332 words
The Singularity Is Nearer: When We Merge with AI
by Ray Kurzweil · 25 Jun 2024
Shape: The Hidden Geometry of Information, Biology, Strategy, Democracy, and Everything Else
by Jordan Ellenberg · 14 May 2021 · 665pp · 159,350 words
The Mysterious Mr. Nakamoto: A Fifteen-Year Quest to Unmask the Secret Genius Behind Crypto
by Benjamin Wallace · 18 Mar 2025 · 431pp · 116,274 words
Data-Ism: The Revolution Transforming Decision Making, Consumer Behavior, and Almost Everything Else
by Steve Lohr · 10 Mar 2015 · 239pp · 70,206 words
Structure and interpretation of computer programs
by Harold Abelson, Gerald Jay Sussman and Julie Sussman · 25 Jul 1996 · 893pp · 199,542 words
Beautiful Data: The Stories Behind Elegant Data Solutions
by Toby Segaran and Jeff Hammerbacher · 1 Jul 2009
The Zero Marginal Cost Society: The Internet of Things, the Collaborative Commons, and the Eclipse of Capitalism
by Jeremy Rifkin · 31 Mar 2014 · 565pp · 151,129 words
On Language: Chomsky's Classic Works Language and Responsibility and Reflections on Language in One Volume
by Noam Chomsky and Mitsou Ronat · 26 Jul 2011
Simple Rules: How to Thrive in a Complex World
by Donald Sull and Kathleen M. Eisenhardt · 20 Apr 2015 · 294pp · 82,438 words
Mastering Structured Data on the Semantic Web: From HTML5 Microdata to Linked Open Data
by Leslie Sikos · 10 Jul 2015
Superminds: The Surprising Power of People and Computers Thinking Together
by Thomas W. Malone · 14 May 2018 · 344pp · 104,077 words
Human Compatible: Artificial Intelligence and the Problem of Control
by Stuart Russell · 7 Oct 2019 · 416pp · 112,268 words
The Optimist: Sam Altman, OpenAI, and the Race to Invent the Future
by Keach Hagey · 19 May 2025 · 439pp · 125,379 words
The Infinity Machine: Demis Hassabis, DeepMind, and the Quest for Superintelligence
by Sebastian Mallaby; · 30 Mar 2026 · 607pp · 161,998 words
Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems
by Martin Kleppmann · 17 Apr 2017
Doing Data Science: Straight Talk From the Frontline
by Cathy O'Neil and Rachel Schutt · 8 Oct 2013 · 523pp · 112,185 words
Code Dependent: Living in the Shadow of AI
by Madhumita Murgia · 20 Mar 2024 · 336pp · 91,806 words
Why Machines Learn: The Elegant Math Behind Modern AI
by Anil Ananthaswamy · 15 Jul 2024 · 416pp · 118,522 words
Calling Bullshit: The Art of Scepticism in a Data-Driven World
by Jevin D. West and Carl T. Bergstrom · 3 Aug 2020
Blue Ocean Strategy, Expanded Edition: How to Create Uncontested Market Space and Make the Competition Irrelevant
by W. Chan Kim and Renée A. Mauborgne · 20 Jan 2014 · 287pp · 80,180 words
More Money Than God: Hedge Funds and the Making of a New Elite
by Sebastian Mallaby · 9 Jun 2010 · 584pp · 187,436 words
Devil's Bargain: Steve Bannon, Donald Trump, and the Storming of the Presidency
by Joshua Green · 17 Jul 2017 · 296pp · 78,112 words
The uplift war
by David Brin · 1 Jun 1987 · 789pp · 213,716 words
Smarter Than Us: The Rise of Machine Intelligence
by Stuart Armstrong · 1 Feb 2014 · 48pp · 12,437 words
Human + Machine: Reimagining Work in the Age of AI
by Paul R. Daugherty and H. James Wilson · 15 Jan 2018 · 523pp · 61,179 words
The Long History of the Future: Why Tomorrow's Technology Still Isn't Here
by Nicole Kobie · 3 Jul 2024 · 348pp · 119,358 words
The Mind Is Flat: The Illusion of Mental Depth and the Improvised Mind
by Nick Chater · 28 Mar 2018 · 263pp · 81,527 words
A World Without Work: Technology, Automation, and How We Should Respond
by Daniel Susskind · 14 Jan 2020 · 419pp · 109,241 words
This Is for Everyone: The Captivating Memoir From the Inventor of the World Wide Web
by Tim Berners-Lee · 8 Sep 2025 · 347pp · 100,038 words
What We Owe the Future: A Million-Year View
by William MacAskill · 31 Aug 2022 · 451pp · 125,201 words
Because Internet: Understanding the New Rules of Language
by Gretchen McCulloch · 22 Jul 2019 · 413pp · 106,479 words
The Economic Singularity: Artificial Intelligence and the Death of Capitalism
by Calum Chace · 17 Jul 2016 · 477pp · 75,408 words
Possible Minds: Twenty-Five Ways of Looking at AI
by John Brockman · 19 Feb 2019 · 339pp · 94,769 words
The Future Is Asian
by Parag Khanna · 5 Feb 2019 · 496pp · 131,938 words
The Business of Platforms: Strategy in the Age of Digital Competition, Innovation, and Power
by Michael A. Cusumano, Annabelle Gawer and David B. Yoffie · 6 May 2019 · 328pp · 84,682 words
The Reverse Centaur's Guide to Life After AI: How to Think About Artificial Intelligence—Before It's Too Late
by Cory Doctorow · 22 Jun 2026 · 175pp · 51,295 words
You Are What You Speak: Grammar Grouches, Language Laws, and the Politics of Identity
by Robert Lane Greene · 8 Mar 2011 · 319pp · 95,854 words
This Will Make You Smarter: 150 New Scientific Concepts to Improve Your Thinking
by John Brockman · 14 Feb 2012 · 416pp · 106,582 words
Surviving AI: The Promise and Peril of Artificial Intelligence
by Calum Chace · 28 Jul 2015 · 144pp · 43,356 words
The Internet Trap: How the Digital Economy Builds Monopolies and Undermines Democracy
by Matthew Hindman · 24 Sep 2018
Noam Chomsky: A Life of Dissent
by Robert F. Barsky · 2 Feb 1997
Applied Artificial Intelligence: A Handbook for Business Leaders
by Mariya Yao, Adelyn Zhou and Marlene Jia · 1 Jun 2018 · 161pp · 39,526 words
Picnic Comma Lightning: In Search of a New Reality
by Laurence Scott · 11 Jul 2018 · 244pp · 81,334 words
How Doctors Think
by Jerome Groopman · 15 Jan 2007 · 292pp · 94,324 words
Covid-19: The Pandemic That Never Should Have Happened and How to Stop the Next One
by Debora MacKenzie · 13 Jul 2020 · 266pp · 80,273 words
Collaborative Society
by Dariusz Jemielniak and Aleksandra Przegalinska · 18 Feb 2020 · 187pp · 50,083 words
Understanding Sponsored Search: Core Elements of Keyword Advertising
by Jim Jansen · 25 Jul 2011 · 298pp · 43,745 words
The Left Case Against the EU
by Costas Lapavitsas · 17 Dec 2018 · 221pp · 46,396 words
Why Things Bite Back: Technology and the Revenge of Unintended Consequences
by Edward Tenner · 1 Sep 1997
Language and Mind
by Noam Chomsky · 1 Jan 1968
Spike: The Virus vs The People - The Inside Story
by Jeremy Farrar and Anjana Ahuja · 15 Jan 2021 · 245pp · 71,886 words
Pandora's Brain
by Calum Chace · 4 Feb 2014 · 345pp · 104,404 words