Keras is a model-level library that provides high-level blocks for the development of deep learning models. Keras developers have focused their efforts on creating high-level models by neglecting low-level operations such as tensor products, convolutions, and so on. These operations have been entrusted to specialized and well-optimized tensor manipulation libraries that already exist, thus acting as a backend engine for Keras. Several backend engines can be connected perfectly to Keras. Actually, Keras has three backend implementations available—TensorFlow, Theano, and Microsoft Cognitive Toolkit (CNTK).

TensorFlow is an open source software library for numerical calculation based on graph modeling (data flow graphs). A graph is defined as an abstract pipeline of mathematical operations operating on tensors, and are also known as multidimensional arrays. Each graph consists of nodes and arcs, wherein the nodes are operations on the data, and the arcs represent the tensors that pass through the various operations.

TensorFlow is the most commonly used library in the field of machine learning and neural networks. It has numerous APIs, including the lowest level, that is, TensorFlow Core, allows complete control over programming. These APIs are those typically used in the field of machine learning, since they make it possible to check in detail all the elements of the model being implemented. The highest level APIs are available and built from TensorFlow Core. In some cases, they can make some operations such as repetitive and predefined tasks faster and simpler, but generally preclude the possibility of going into detail, and in the implementation of a neural network it is often necessary to have a more precise control over operations. However, they can still be useful for the development of standard machine learning models.

TensorFlow provides interfaces for different languages, including Python and C or C ++, with full support, and Go or Java in Beta. It also supports parallel computing on GPUs or CPUs, and distributed computation allows execution even on mobile devices.

A TensorFlow program is typically structured in the following two distinct phases:

• Construction: In this phase, the various operations of the graph that will be performed on the input tensors are defined.
• Execution: In this phase, the operations defined in the previous phase are evaluated so as to retrieve the numerical output. The execution of operations is managed through the session object of TensorFlow.

The fundamental unit in TensorFlow is the tensor. A tensor consists of a set of primitive type values ​​modeled as a multidimensional array. In TensorFlow, almost all the functions contained in the API take tensors as inputs and always output tensors. Each tensor contained in the graph has a unique name that can be specified by the user, or otherwise automatically assigned.

For more information on tensors and any other resources related to TensorFlow, you can refer to the official site of the framework, as shown in the following screenshot:

The default behavior in TensorFlow is to allocate all the components (tensors and operations) in the GPU memory (if it has been installed with computational support on a GPU). However, you can manually specify where to allocate each tensor and operation. It is recommended to minimize switches between the CPU and GPU as these slow down execution.

TensorFlow also allows you to define scopes for variables through which a namespace mechanism is managed; they facilitate the definition of complex models. Scopes are also very important for sharing variables between multiple graphs.

TensorFlow provides the following two options for reading input data:

• Manual: Observations are manually read and organized in batches, and then passed to the model. It is a simple mechanism to use, but it can become very slow because the data must be continually copied from the Python environment to the TensorFlow environment.
• Integrated: All operations to read data and organize them in batches of observations are implemented within the graph. It is a less intuitive mechanism to use, but presents a large increase in performance, since all data always remains in the TensorFlow environment.

All basic operations for neural networks are implemented in TensorFlow as graph nodes. The framework automatically manages everything needed to implement the forward and backward pass, including the automatic calculation of derivatives.

Among the main operations available for the construction of neural networks models, we find the following:

• Convolutions
• Sum of the bias
• Fully connected levels
• Activation functions
• Pooling
• Prediction functions

Finally, there is a suite of tools for graphic display that is fully integrated into TensorFlow called TensorBoard. It allows visualizing the computational graph of the model and many other statistics useful for the analysis of the training process.

Theano is an open source library of numerical computation for the Python programming language developed by a group of machine learning experts at the University of Montreal. In Theano, calculations are expressed using a syntax that is similar to NumPy's, and is compiled to perform efficiently on both CPU and GPU architectures.

It is named after a Greek mathematician, Theano of Crotone. Theano is a compiler for mathematical expressions written in Python. It allows defining, optimizing, and evaluating mathematical expressions, in particular those with multidimensional arrays. Using Theano, it is possible to achieve comparable speeds with C or C ++ applications for problems with large amounts of data. Theano combines aspects of a computer algebra system with aspects of an optimizing compiler. These characteristics are useful when complex mathematical expressions are evaluated repeatedly, and the evaluation speed is crucial. In the following screenshot, we can see the official home page of Theano:

It combines the convenience of the NumPy syntax with the speed of an optimized machine language. The user composes mathematical expressions in a high-level description that mimics the syntax and semantics of NumPy, even though it is statically typed. These expressions allow Theano to provide symbolic differentiation. Before performing the calculation, Theano optimizes the choice of expressions, translates them into C ++ (or CUDA for GPU), and automatically compiles them into dynamically loaded Python modules. The machine learning algorithms implemented with Theano are particularly fast.

The CNTK is an open source toolkit for commercial-grade applications that is distributed in deep learning. CNTK implements neural networks as a series of computational steps via a directed graph. CNTK is a command-line program that can do simple and deep neural network analyses. CNTK was originally developed as an internal Microsoft tool.

Using CNTK, the most common machine learning models are easily achievable and can be combined. CNTK is available as a library in Python, C#, or C++ environments, or it can be used as a standalone machine learning tool through its own model description language. In addition, CNTK model evaluation functionality can be used from Java programs. CNTK supports 64-bit Linux or 64-bit Windows operating systems. To install CNTK, you can either choose precompiled binary packages, or compile the toolkit from the source provided in GitHub. In the following screenshot, we can see the official home page of CNTK:

In the Tutorials section, there is a collection of code samples, recipes, and tutorials on the various ways you can use the toolkit with scenarios for images, text, and speech data.