Book Image

Natural Language Processing with TensorFlow

By : Motaz Saad, Thushan Ganegedara
Book Image

Natural Language Processing with TensorFlow

By: Motaz Saad, Thushan Ganegedara

Overview of this book

Natural language processing (NLP) supplies the majority of data available to deep learning applications, while TensorFlow is the most important deep learning framework currently available. Natural Language Processing with TensorFlow brings TensorFlow and NLP together to give you invaluable tools to work with the immense volume of unstructured data in today’s data streams, and apply these tools to specific NLP tasks. Thushan Ganegedara starts by giving you a grounding in NLP and TensorFlow basics. You'll then learn how to use Word2vec, including advanced extensions, to create word embeddings that turn sequences of words into vectors accessible to deep learning algorithms. Chapters on classical deep learning algorithms, like convolutional neural networks (CNN) and recurrent neural networks (RNN), demonstrate important NLP tasks as sentence classification and language generation. You will learn how to apply high-performance RNN models, like long short-term memory (LSTM) cells, to NLP tasks. You will also explore neural machine translation and implement a neural machine translator. After reading this book, you will gain an understanding of NLP and you'll have the skills to apply TensorFlow in deep learning NLP applications, and how to perform specific NLP tasks.

Related Content you might be interested in

Current Title:

Small book image
Natural Language Processing with TensorFlow
Motaz Saad | Thushan Ganegedara
May, 2018 | 472 pages
Small book image
Deep Learning Essentials
Wei Di | Anurag Bhardwaj | Jianing Wei
Jan, 2018 | 284 pages
Small book image
Hands-On Natural Language Processing with PyTorch 1.x
Thomas Dop
Jul, 2020 | 276 pages
Small book image
Recurrent Neural Networks with Python Quick Start Guide
Simeon Kostadinov
Nov, 2018 | 122 pages
Table of Contents (16 chapters)
Natural Language Processing with TensorFlow
Natural Language Processing with TensorFlow
Contributors
Contributors
Preface
Preface
Free Chapter
1
Introduction to Natural Language Processing
Introduction to Natural Language Processing
What is Natural Language Processing?
Tasks of Natural Language Processing
The traditional approach to Natural Language Processing
The deep learning approach to Natural Language Processing
The roadmap – beyond this chapter
Introduction to the technical tools
Summary
2
Understanding TensorFlow
Understanding TensorFlow
What is TensorFlow?
Inputs, variables, outputs, and operations
Reusing variables with scoping
Implementing our first neural network
Summary
3
Word2vec – Learning Word Embeddings
Word2vec – Learning Word Embeddings
What is a word representation or meaning?
Classical approaches to learning word representation
Word2vec – a neural network-based approach to learning word representation
The skip-gram algorithm
The Continuous Bag-of-Words algorithm
Summary
4
Advanced Word2vec
Advanced Word2vec
The original skip-gram algorithm
Comparing skip-gram with CBOW
Extensions to the word embeddings algorithms
More recent algorithms extending skip-gram and CBOW
GloVe – Global Vectors representation
Document classification with Word2vec
Summary
5
Sentence Classification with Convolutional Neural Networks
Sentence Classification with Convolutional Neural Networks
Introducing Convolution Neural Networks
Understanding Convolution Neural Networks
Exercise – image classification on MNIST with CNN
Using CNNs for sentence classification
Summary
6
Recurrent Neural Networks
Recurrent Neural Networks
Understanding Recurrent Neural Networks
Backpropagation Through Time
Applications of RNNs
Generating text with RNNs
Evaluating text results output from the RNN
Perplexity – measuring the quality of the text result
Recurrent Neural Networks with Context Features – RNNs with longer memory
Summary
7
Long Short-Term Memory Networks
Long Short-Term Memory Networks
Understanding Long Short-Term Memory Networks
How LSTMs solve the vanishing gradient problem
Other variants of LSTMs
Summary
8
Applications of LSTM – Generating Text
Applications of LSTM – Generating Text
Our data
Implementing an LSTM
Comparing LSTMs to LSTMs with peephole connections and GRUs
Improving LSTMs – beam search
Improving LSTMs – generating text with words instead of n-grams
Using the TensorFlow RNN API
Summary
9
Applications of LSTM – Image Caption Generation
Applications of LSTM – Image Caption Generation
Getting to know the data
The machine learning pipeline for image caption generation
Extracting image features with CNNs
Implementation – loading weights and inferencing with VGG-
Learning word embeddings
Preparing captions for feeding into LSTMs
Generating data for LSTMs
Defining the LSTM
Evaluating the results quantitatively
Captions generated for test images
Using TensorFlow RNN API with pretrained GloVe word vectors
Summary
10
Sequence-to-Sequence Learning – Neural Machine Translation
Sequence-to-Sequence Learning – Neural Machine Translation
Machine translation
A brief historical tour of machine translation
Understanding Neural Machine Translation
Preparing data for the NMT system
Training the NMT
Inference with NMT
The BLEU score – evaluating the machine translation systems
Implementing an NMT from scratch – a German to English translator
Training an NMT jointly with word embeddings
Improving NMTs
Attention
Other applications of Seq2Seq models – chatbots
Summary
11
Current Trends and the Future of Natural Language Processing
Current Trends and the Future of Natural Language Processing
Current trends in NLP
Penetration into other research fields
Towards Artificial General Intelligence
NLP for social media
New tasks emerging
Newer machine learning models
Summary
References
Mathematical Foundations and Advanced TensorFlow
Mathematical Foundations and Advanced TensorFlow
Basic data structures
Special types of matrices
Tensor/matrix operations
Probability
Introduction to Keras
Introduction to the TensorFlow seq2seq library
Visualizing word embeddings with TensorBoard
Summary
Index
Index

Contributors

About the author

Thushan Ganegedara is currently a third year Ph.D. student at the University of Sydney, Australia. He is specializing in machine learning and has a liking for deep learning. He lives dangerously and runs algorithms on untested data. He also works as the chief data scientist for AssessThreat, an Australian start-up. He got his BSc. (Hons) from the University of Moratuwa, Sri Lanka. He frequently writes technical articles and tutorials about machine learning. Additionally, he also strives for a healthy lifestyle by including swimming in his daily schedule.

I would like to thank my parents, my siblings, and my wife for the faith they had in me and the support they have given, also all my teachers and my Ph.D advisor for the guidance he provided me with.

About the reviewers

Motaz Saad holds a Ph.D. in computer science from the University of Lorraine. He loves data and he likes to play with it. He has over 10 years, professional experience in NLP, computational linguistics, data science, and machine learning. He currently works as an assistant professor at the faculty of information technology, IUG.

Dr Joseph O'Connor is a data scientist with a deep passion for deep learning. His company, Deep Learn Analytics, a UK-based data science consultancy, works with businesses to develop machine learning applications and infrastructure from concept to deployment. He was awarded a Ph.D. from University College London for his work analyzing data on the MINOS high-energy physics experiment. Since then, he has developed ML products for a number of companies in the private sector, specializing in NLP and time series forecasting. You can find him at http://deeplearnanalytics.com/.

Packt is searching for authors like you

If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.

Previous Section
Next Chapter