Book Image

50 Algorithms Every Programmer Should Know - Second Edition

By : Imran Ahmad
4 (5)
Book Image

50 Algorithms Every Programmer Should Know - Second Edition

4 (5)
By: Imran Ahmad

Overview of this book

The ability to use algorithms to solve real-world problems is a must-have skill for any developer or programmer. This book will help you not only to develop the skills to select and use an algorithm to tackle problems in the real world but also to understand how it works. You'll start with an introduction to algorithms and discover various algorithm design techniques, before exploring how to implement different types of algorithms, with the help of practical examples. As you advance, you'll learn about linear programming, page ranking, and graphs, and will then work with machine learning algorithms to understand the math and logic behind them. Case studies will show you how to apply these algorithms optimally before you focus on deep learning algorithms and learn about different types of deep learning models along with their practical use. You will also learn about modern sequential models and their variants, algorithms, methodologies, and architectures that are used to implement Large Language Models (LLMs) such as ChatGPT. Finally, you'll become well versed in techniques that enable parallel processing, giving you the ability to use these algorithms for compute-intensive tasks. By the end of this programming book, you'll have become adept at solving real-world computational problems by using a wide range of algorithms.
Table of Contents (22 chapters)
Free Chapter
1
Section 1: Fundamentals and Core Algorithms
7
Section 2: Machine Learning Algorithms
14
Section 3: Advanced Topics
20
Other Books You May Enjoy
21
Index

Rectified linear unit (ReLU)

The output for the first two activation functions presented in this chapter was binary. That means that they will take a set of input variables and convert them into binary outputs. ReLU is an activation function that takes a set of input variables as input and converts them into a single continuous output. In neuralneural networks, ReLU is the most popular activation function and is usually used in the hidden layers, where we do not want to convert continuous variables into category variables.The following diagram summarizes the ReLU activation function:

Figure 8.12: Rectified linear unit

Note that when x≤ 0, that means y = 0. This means that any signal from the input that is zero or less than zero is translated into a zero output:

Shape Description automatically generated with medium confidence
 for 
 for
Shape Description automatically generated with medium confidence

As soon as x becomes more than zero, it is x.The ReLU function is one of the most used activation functions in neural neural networks. It can...