Book Image

Natural Language Processing with TensorFlow

By : Motaz Saad, Thushan Ganegedara
Book Image

Natural Language Processing with TensorFlow

By: Motaz Saad, Thushan Ganegedara

Overview of this book

Natural language processing (NLP) supplies the majority of data available to deep learning applications, while TensorFlow is the most important deep learning framework currently available. Natural Language Processing with TensorFlow brings TensorFlow and NLP together to give you invaluable tools to work with the immense volume of unstructured data in today’s data streams, and apply these tools to specific NLP tasks. Thushan Ganegedara starts by giving you a grounding in NLP and TensorFlow basics. You'll then learn how to use Word2vec, including advanced extensions, to create word embeddings that turn sequences of words into vectors accessible to deep learning algorithms. Chapters on classical deep learning algorithms, like convolutional neural networks (CNN) and recurrent neural networks (RNN), demonstrate important NLP tasks as sentence classification and language generation. You will learn how to apply high-performance RNN models, like long short-term memory (LSTM) cells, to NLP tasks. You will also explore neural machine translation and implement a neural machine translator. After reading this book, you will gain an understanding of NLP and you'll have the skills to apply TensorFlow in deep learning NLP applications, and how to perform specific NLP tasks.

Related Content you might be interested in

Current Title:

Small book image
Natural Language Processing with TensorFlow
Motaz Saad | Thushan Ganegedara
May, 2018 | 472 pages
Small book image
Deep Learning Essentials
Wei Di | Anurag Bhardwaj | Jianing Wei
Jan, 2018 | 284 pages
Small book image
Hands-On Natural Language Processing with PyTorch 1.x
Thomas Dop
Jul, 2020 | 276 pages
Small book image
Recurrent Neural Networks with Python Quick Start Guide
Simeon Kostadinov
Nov, 2018 | 122 pages
Table of Contents (16 chapters)
Natural Language Processing with TensorFlow
Natural Language Processing with TensorFlow
Contributors
Contributors
Preface
Preface
Free Chapter
1
Introduction to Natural Language Processing
Introduction to Natural Language Processing
What is Natural Language Processing?
Tasks of Natural Language Processing
The traditional approach to Natural Language Processing
The deep learning approach to Natural Language Processing
The roadmap – beyond this chapter
Introduction to the technical tools
Summary
2
Understanding TensorFlow
Understanding TensorFlow
What is TensorFlow?
Inputs, variables, outputs, and operations
Reusing variables with scoping
Implementing our first neural network
Summary
3
Word2vec – Learning Word Embeddings
Word2vec – Learning Word Embeddings
What is a word representation or meaning?
Classical approaches to learning word representation
Word2vec – a neural network-based approach to learning word representation
The skip-gram algorithm
The Continuous Bag-of-Words algorithm
Summary
4
Advanced Word2vec
Advanced Word2vec
The original skip-gram algorithm
Comparing skip-gram with CBOW
Extensions to the word embeddings algorithms
More recent algorithms extending skip-gram and CBOW
GloVe – Global Vectors representation
Document classification with Word2vec
Summary
5
Sentence Classification with Convolutional Neural Networks
Sentence Classification with Convolutional Neural Networks
Introducing Convolution Neural Networks
Understanding Convolution Neural Networks
Exercise – image classification on MNIST with CNN
Using CNNs for sentence classification
Summary
6
Recurrent Neural Networks
Recurrent Neural Networks
Understanding Recurrent Neural Networks
Backpropagation Through Time
Applications of RNNs
Generating text with RNNs
Evaluating text results output from the RNN
Perplexity – measuring the quality of the text result
Recurrent Neural Networks with Context Features – RNNs with longer memory
Summary
7
Long Short-Term Memory Networks
Long Short-Term Memory Networks
Understanding Long Short-Term Memory Networks
How LSTMs solve the vanishing gradient problem
Other variants of LSTMs
Summary
8
Applications of LSTM – Generating Text
Applications of LSTM – Generating Text
Our data
Implementing an LSTM
Comparing LSTMs to LSTMs with peephole connections and GRUs
Improving LSTMs – beam search
Improving LSTMs – generating text with words instead of n-grams
Using the TensorFlow RNN API
Summary
9
Applications of LSTM – Image Caption Generation
Applications of LSTM – Image Caption Generation
Getting to know the data
The machine learning pipeline for image caption generation
Extracting image features with CNNs
Implementation – loading weights and inferencing with VGG-
Learning word embeddings
Preparing captions for feeding into LSTMs
Generating data for LSTMs
Defining the LSTM
Evaluating the results quantitatively
Captions generated for test images
Using TensorFlow RNN API with pretrained GloVe word vectors
Summary
10
Sequence-to-Sequence Learning – Neural Machine Translation
Sequence-to-Sequence Learning – Neural Machine Translation
Machine translation
A brief historical tour of machine translation
Understanding Neural Machine Translation
Preparing data for the NMT system
Training the NMT
Inference with NMT
The BLEU score – evaluating the machine translation systems
Implementing an NMT from scratch – a German to English translator
Training an NMT jointly with word embeddings
Improving NMTs
Attention
Other applications of Seq2Seq models – chatbots
Summary
11
Current Trends and the Future of Natural Language Processing
Current Trends and the Future of Natural Language Processing
Current trends in NLP
Penetration into other research fields
Towards Artificial General Intelligence
NLP for social media
New tasks emerging
Newer machine learning models
Summary
References
Mathematical Foundations and Advanced TensorFlow
Mathematical Foundations and Advanced TensorFlow
Basic data structures
Special types of matrices
Tensor/matrix operations
Probability
Introduction to Keras
Introduction to the TensorFlow seq2seq library
Visualizing word embeddings with TensorBoard
Summary
Index
Index

Natural Language Processing with TensorFlow

Copyright © 2018 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Acquisition Editor: Frank Pohlmann

Project Editor: Radhika Atitkar

Content Development Editor: Chris Nelson

Technical Editor: Bhagyashree Rai

Copy Editor: Tom Jacob

Proofreader: Safis Editing

Indexer: Rekha Nair

Graphics: Tom Scaria

Production Coordinator: Nilesh Mohite

First published: May 2018

Production reference: 2310518

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-78847-831-1

www.packtpub.com

mapt.io

Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.

Why subscribe?

  • Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals

  • Learn better with Skill Plans built especially for you

  • Get a free eBook or video every month

  • Mapt is fully searchable

  • Copy and paste, print, and bookmark content

PacktPub.com

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.

Previous Section
Next Chapter