Book Image

Java Deep Learning Essentials

By : Yusuke Sugomori
Book Image

Java Deep Learning Essentials

By: Yusuke Sugomori

Overview of this book

AI and Deep Learning are transforming the way we understand software, making computers more intelligent than we could even imagine just a decade ago. Deep Learning algorithms are being used across a broad range of industries – as the fundamental driver of AI, being able to tackle Deep Learning is going to a vital and valuable skill not only within the tech world but also for the wider global economy that depends upon knowledge and insight for growth and success. It’s something that’s moving beyond the realm of data science – if you’re a Java developer, this book gives you a great opportunity to expand your skillset. Starting with an introduction to basic machine learning algorithms, to give you a solid foundation, Deep Learning with Java takes you further into this vital world of stunning predictive insights and remarkable machine intelligence. Once you’ve got to grips with the fundamental mathematical principles, you’ll start exploring neural networks and identify how to tackle challenges in large networks using advanced algorithms. You will learn how to use the DL4J library and apply Deep Learning to a range of real-world use cases. Featuring further guidance and insights to help you solve challenging problems in image processing, speech recognition, language modeling, this book will make you rethink what you can do with Java, showing you how to use it for truly cutting-edge predictive insights. As a bonus, you’ll also be able to get to grips with Theano and Caffe, two of the most important tools in Deep Learning today. By the end of the book, you’ll be ready to tackle Deep Learning with Java. Wherever you’ve come from – whether you’re a data scientist or Java developer – you will become a part of the Deep Learning revolution!
Table of Contents (15 chapters)
Java Deep Learning Essentials
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
7
Other Important Deep Learning Libraries
Index

Index

A

  • ADADELTA / Learning rate optimization
  • ADAGRAD
    • URL / Learning rate optimization
  • AI
    • defining / Definition of AI
    • history / AI booms in the past
    • and deep learning / AI and deep learning, Expected next actions
  • AI transition
    • defining / Transition of AI
  • AlphaGo
    • about / Breaking news about deep learning
  • automatic colorization
    • reference link / Expected next actions
  • automatic differentiation / Theano

B

  • backpropagated error
    • about / Multi-layer perceptrons (multi-layer neural networks)
  • backpropagation formula
    • about / Multi-layer perceptrons (multi-layer neural networks)
  • Backpropagation through Time (BPTT) / Recurrent neural networks
  • BBC
    • URL / AI and deep learning
  • benchmark tests
    • URL / Summary
  • Bernoulli RBM
    • about / Restricted Boltzmann machines
  • bigram
    • about / Feed-forward neural networks for NLP
  • Boltzmann Machines (BMs)
    • about / Restricted Boltzmann machines
  • Bot Store
    • URL / Expected next actions
  • breadth-first search (BFS)
    • about / AI booms in the past
  • breakdown-oriented approach, deep learning
    • about / The approaches to maximizing deep learning possibilities and abilities, Breakdown-oriented approach

C

  • Caffe
    • about / Caffe
    • URL / Caffe
  • Chainer
    • URL / Summary
  • clustering
    • about / Supervised and unsupervised learning
  • computational differentiation / Theano
  • constant error carousel (CEC) / Long short term memory networks
  • Contrastive Divergence (CD)
    • about / Restricted Boltzmann machines
  • Convolutional neural networks (CNN)
    • about / Convolutional neural networks
    • convolution layers / Convolution
    • kernels / Convolution
    • feature maps / Convolution
    • local receptive field / Convolution
    • translation invariance / Convolution
    • pooling layers / Pooling
    • equations / Equations and implementations
    • implementations / Equations and implementations
  • convolution layers / Convolution
  • Cyc
    • URL / AI booms in the past

D

  • decision boundary
    • about / The need for training in machine learning
  • Decode
    • about / Denoising Autoencoders
  • deep belief nets (DBN)
    • about / AI and deep learning
  • Deep Belief Nets (DBNs)
    • defining / Deep Belief Nets (DBNs)
  • Deep Dream
    • URL / AI and deep learning
  • deep learning
    • and AI / AI and deep learning, Expected next actions
    • defining / Deep learning's evolution – what was the breakthrough?
    • references / Deep learning's evolution – what was the breakthrough?
    • with pre-training / Deep learning with pre-training
    • algorithms / Deep learning algorithms
    • Restricted Boltzmann Machines (RBM) / Restricted Boltzmann machines
    • Deep Belief Nets (DBNs) / Deep Belief Nets (DBNs)
    • Denoising Autoencoders (DA) / Denoising Autoencoders
    • Stacked Denoising Autoencoders (SDA) / Stacked Denoising Autoencoders (SDA)
    • active, on fields / Fields where deep learning is active
    • image recognition field / Image recognition
    • natural language processing (NLP) / Natural language processing
    • issues / The difficulties of deep learning
    • possibilities, maximizing / The approaches to maximizing deep learning possibilities and abilities
    • abilities, maximizing / The approaches to maximizing deep learning possibilities and abilities
    • field-oriented approach / Field-oriented approach
    • breakdown-oriented approach / Breakdown-oriented approach
    • output-oriented approach / Output-oriented approach
    • about / Breaking news about deep learning
    • news sources / Useful news sources for deep learning
  • deep learning algorithm
    • URL / Image recognition
  • deep learning algorithms
    • without pre-training / Deep learning algorithms without pre-training
  • deep learning group
    • URL / Useful news sources for deep learning
  • Deep Learning News
    • URL / Useful news sources for deep learning
    • about / Useful news sources for deep learning
  • DeepMind
    • URL / Breaking news about deep learning
  • Denoising Autoencoders (DA)
    • defining / Denoising Autoencoders
  • depth-first search (DFS)
    • about / AI booms in the past
  • DL4J
    • implementing, with ND4J / Introducing DL4J and ND4J
    • URL / Introducing DL4J and ND4J
    • implementations / Implementations with DL4J
    • set up / Setup
    • building / Build
    • DBNIrisExample.java / DBNIrisExample.java
    • CSVExample.java / CSVExample.java
    • CNNMnistExample.java / LenetMnistExample.java / CNNMnistExample.java/LenetMnistExample.java
    • learning rate, optimization / Learning rate optimization
  • dropout
    • about / AI and deep learning
  • dropout algorithm
    • about / Dropout
    • rectifier / Dropout
    • Rectified Linear Unit (ReLU) / Dropout
    • softplus function / Dropout

E

  • Encode
    • about / Denoising Autoencoders

F

  • feature maps / Convolution
  • feed-forward neural networks / Feed-forward neural networks for NLP
  • field-oriented approach, deep learning
    • about / The approaches to maximizing deep learning possibilities and abilities
    • medicine / Medicine
    • automobiles / Automobiles
    • advert technologies / Advert technologies
    • profession or practice / Profession or practice
    • sports / Sports
  • fine-tuning
    • about / Deep learning's evolution – what was the breakthrough?
  • forget gate / Long short term memory networks
  • frame problem
    • about / AI booms in the past

G

  • GitHub
    • URL / Implementations with DL4J
  • GitXiv
    • URL / Useful news sources for deep learning
    • about / Useful news sources for deep learning

H

  • Hacker News
    • about / Useful news sources for deep learning
    • URL / Useful news sources for deep learning
  • Hidden Markov Model (HMM) / Hidden Markov Model (HMM)
  • hidden units
    • about / Restricted Boltzmann machines
  • Hopfield network
    • about / Restricted Boltzmann machines

I

  • Imagenet Large Scale Visual Recognition Challenge (ILSVRC)
    • about / AI and deep learning
  • image recognition
    • about / Image recognition
  • Inceptionism
    • about / AI and deep learning
    • URL / AI and deep learning
  • input gate / Long short term memory networks
  • input method editor (IME)
    • about / Machine learning evolves

K

  • K-fold cross-validation
    • about / Machine learning application flow
  • kernels / Convolution
  • knowledge base
    • about / AI booms in the past
  • Knowledge Representation (KR)
    • about / AI booms in the past

L

  • layer-wise training
    • about / Deep learning's evolution – what was the breakthrough?
  • library/framework
    • versus scratch implementations / Implementing from scratch versus a library/framework
  • logistic regression
    • defining / Logistic regression, Logistic regression
  • long short term memory (LSTM) network / Long short term memory networks
  • LSTM block / Long short term memory networks
  • LSTM memory block / Long short term memory networks

M

  • machine
    • and human, comparing / Things dividing a machine and human
  • machine learning
    • defining / Machine learning evolves
    • drawbacks / What even machine learning cannot do
    • need for training / The need for training in machine learning
    • application flow / Machine learning application flow
  • Markov model / Feed-forward neural networks for NLP
  • Markov process
    • about / Hidden Markov Model (HMM)
  • maximizing the margin
    • about / Support Vector Machine (SVM)
  • maximum likelihood estimation (MLE) / Feed-forward neural networks for NLP
  • mini-batch
    • about / Logistic regression
  • mini-batch stochastic gradient descent (MSGD)
    • about / Logistic regression
  • MLP implementation
    • URL / Theano
  • MNIST classifications
    • URL / TensorFlow
  • MNIST database
    • about / AI and deep learning
  • momentum coefficient / Learning rate optimization
  • multi-class logistic regression
    • defining / Multi-class logistic regression
  • multi-layer neural networks (MLP)
    • about / Multi-layer perceptrons (multi-layer neural networks)
  • multi-layer perceptrons
    • defining / Multi-layer perceptrons (multi-layer neural networks)

N

  • N-gram
    • about / Feed-forward neural networks for NLP
  • natural language processing (NLP)
    • about / Hidden Markov Model (HMM), Natural language processing
    • feed-forward neural networks / Feed-forward neural networks for NLP
    • deep learning for / Deep learning for NLP
    • recurrent neural networks / Recurrent neural networks
    • long short term memory networks / Long short term memory networks
  • ND4J
    • URL / Introducing DL4J and ND4J, Implementations with ND4J
    • implementations / Implementations with ND4J
  • Nervana
    • URL / Summary
  • Nesterov's Accelerated Gradient Descent / Learning rate optimization
  • neural-storyteller
    • URL / Expected next actions
  • Neural Network Language Model (NLMM)
    • URL / Feed-forward neural networks for NLP
  • neural networks
    • defining / Neural networks
    • theories and algorithms / Theories and algorithms of neural networks
    • perceptron algorithm / Perceptrons (single-layer neural networks)
    • logistic regression / Logistic regression
    • multi-class logistic regression / Multi-class logistic regression
    • multi-layer perceptrons / Multi-layer perceptrons (multi-layer neural networks)
    • problems / Neural networks fall, Neural networks' revenge
  • No Free Lunch Theorem (NFLT) / Image recognition

O

  • output-oriented approach, deep learning / The approaches to maximizing deep learning possibilities and abilities, Output-oriented approach
  • output gate / Long short term memory networks
  • overfitting problem
    • about / Machine learning application flow

P

  • paper, deep belief nets (DBN)
    • URL / AI and deep learning
  • peephole connections / Long short term memory networks
  • perceptron
    • about / Theories and algorithms of neural networks
  • perceptron algorithm / Perceptrons (single-layer neural networks)
  • pooling layers / Pooling
  • pre-training
    • about / Deep learning's evolution – what was the breakthrough?
  • pretraining
    • about / AI and deep learning
  • probabilistic statistical model
    • about / Machine learning evolves
  • protocol file
    • about / Caffe
  • Pylearn2
    • URL / Summary

R

  • Rectified Linear Unit (ReLU) / Dropout
  • recurrent neural network (RNN) / Recurrent neural networks
  • recurrent neural network language model (RNNLM)
    • URL / Recurrent neural networks
  • reinforcement learning
    • defining / Reinforcement learning
  • Restricted Boltzmann Machines (RBM) / Restricted Boltzmann machines
  • RMSProp / Learning rate optimization
  • RMSProp + momentum / Learning rate optimization

S

  • scratch implementations
    • versus library/framework / Implementing from scratch versus a library/framework
  • signe
    • about / Things dividing a machine and human
  • signifiant
    • about / Things dividing a machine and human
  • signifié
    • about / Things dividing a machine and human
  • Skymind
    • URL / Introducing DL4J and ND4J
  • softplus function / Dropout
  • Stacked Denoising Autoencoders (SDA)
    • defining / Stacked Denoising Autoencoders (SDA)
  • stochastic gradient descent (SGD)
    • about / Logistic regression
  • strong AI
    • about / Definition of AI
  • supervised learning
    • about / Supervised and unsupervised learning
    • Support Vector Machine (SVM) / Support Vector Machine (SVM)
  • Support Vector Machine (SVM) / Support Vector Machine (SVM)
  • support vectors
    • about / Support Vector Machine (SVM)
  • symbol content
    • about / Things dividing a machine and human
  • symbol grounding problem
    • about / AI booms in the past
  • symbol representation
    • about / Things dividing a machine and human

T

  • Tay
    • URL / Expected next actions
  • Technical Singularity
    • about / AI and deep learning
  • TensorFlow
    • URL / Introducing DL4J and ND4J, TensorFlow
    • about / TensorFlow
  • Theano
    • about / Theano
    • URL / Theano
    • deep learning algorithms, URL / Theano
  • trigram
    • about / Feed-forward neural networks for NLP
  • truncated BPTT / Recurrent neural networks

U

  • unigram
    • about / Feed-forward neural networks for NLP
  • unsupervised learning
    • about / Supervised and unsupervised learning
    • Hidden Markov Model (HMM) / Hidden Markov Model (HMM)

V

  • vanishing gradient problem
    • about / Neural networks fall
  • visible layer
    • about / Restricted Boltzmann machines
  • visible units
    • about / Restricted Boltzmann machines

W

  • weak AI
    • about / Definition of AI