Book Image

50 Algorithms Every Programmer Should Know - Second Edition

By : Imran Ahmad
4 (5)
Book Image

50 Algorithms Every Programmer Should Know - Second Edition

4 (5)
By: Imran Ahmad

Overview of this book

The ability to use algorithms to solve real-world problems is a must-have skill for any developer or programmer. This book will help you not only to develop the skills to select and use an algorithm to tackle problems in the real world but also to understand how it works. You'll start with an introduction to algorithms and discover various algorithm design techniques, before exploring how to implement different types of algorithms, with the help of practical examples. As you advance, you'll learn about linear programming, page ranking, and graphs, and will then work with machine learning algorithms to understand the math and logic behind them. Case studies will show you how to apply these algorithms optimally before you focus on deep learning algorithms and learn about different types of deep learning models along with their practical use. You will also learn about modern sequential models and their variants, algorithms, methodologies, and architectures that are used to implement Large Language Models (LLMs) such as ChatGPT. Finally, you'll become well versed in techniques that enable parallel processing, giving you the ability to use these algorithms for compute-intensive tasks. By the end of this programming book, you'll have become adept at solving real-world computational problems by using a wide range of algorithms.

Related Content you might be interested in

Current Title:

Small book image
50 Algorithms Every Programmer Should Know - Second Edition
Imran Ahmad
Sep, 2023 | 538 pages
Small book image
AI for Absolute Beginners: A Clear Guide to Tomorrow
Oliver Theobald
Mar, 2024 | 188 pages
Small book image
MATLAB for Machine Learning
Giuseppe Ciaburro
Jan, 2024 | 374 pages
Small book image
Hands-On Data Structures and Algorithms with Python
Benjamin Baka | Dr Basant Agarwal
Oct, 2018 | 398 pages
Table of Contents (22 chapters)
Preface
Preface
Who this book is for
What this book covers
Get in touch
Free Chapter
1
Section 1: Fundamentals and Core Algorithms
Section 1: Fundamentals and Core Algorithms
2
Overview of Algorithms
Overview of Algorithms
What is an algorithm?
Python packages
Algorithm design techniques
Performance analysis
Selecting an algorithm
Validating an algorithm
Summary
3
Data Structures Used in Algorithms
Data Structures Used in Algorithms
Exploring Python built-in data types
Exploring abstract data types
Summary
4
Sorting and Searching Algorithms
Sorting and Searching Algorithms
Introducing sorting algorithms
Introduction to searching algorithms
Practical applications
Summary
5
Designing Algorithms
Designing Algorithms
Introducing the basic concepts of designing an algorithm
Understanding algorithmic strategies
A practical application – solving the TSP
Presenting the PageRank algorithm
Understanding linear programming
Summary
6
Graph Algorithms
Graph Algorithms
Understanding graphs: a brief introduction
Graph theory and network analysis
Representations of graphs
Graph mechanics and types
Introducing network analysis theory
Understanding graph traversals
Case study: fraud detection using SNA
Summary
7
Section 2: Machine Learning Algorithms
Section 2: Machine Learning Algorithms
8
Unsupervised Machine Learning Algorithms
Unsupervised Machine Learning Algorithms
Introducing unsupervised learning
Understanding clustering algorithms
Steps of hierarchical clustering
Coding a hierarchical clustering algorithm
Understanding DBSCAN
Creating clusters using DBSCAN in Python
Evaluating the clusters
Dimensionality reduction
Association rules mining
Summary
9
Traditional Supervised Learning Algorithms
Traditional Supervised Learning Algorithms
Understanding supervised machine learning
Formulating supervised machine learning problems
Understanding classification algorithms
Decision tree classification algorithm
Understanding the ensemble methods
Logistic regression
The SVM algorithm
Bayes’ theorem
For classification algorithms, the winner is...
Linear regression
For regression algorithms, the winner is...
Practical example – how to predict the weather
Summary
10
Neural Network Algorithms
Neural Network Algorithms
The evolution of neural networks
Understanding neural networks
Training a neural network
Understanding the anatomy of a neural network
Defining gradient descent
Activation functions
Tools and frameworks
Choosing a sequential or functional model
Understanding the types of neural networks
Using transfer learning
Case study – using deep learning for fraud detection
Summary
11
Algorithms for Natural Language Processing
Algorithms for Natural Language Processing
Introducing NLP
Understanding NLP terminology
Cleaning data using Python
Understanding the Term Document Matrix
Introduction to word embedding
Implementing word embedding with Word2Vec
Case study: Restaurant review sentiment analysis
Applications of NLP
Summary
12
Understanding Sequential Models
Understanding Sequential Models
Understanding sequential data
Data representation for sequential models
Introducing RNNs
GRU
Introducing LSTM
Summary
13
Advanced Sequential Modeling Algorithms
Advanced Sequential Modeling Algorithms
The evolution of advanced sequential modeling techniques
Exploring autoencoders
Understanding the Seq2Seq model
Understanding the attention mechanism
Delving into self-attention
Transformers: the evolution in neural networks after self-attention
LLMs
Bottom of Form
Summary
14
Section 3: Advanced Topics
Section 3: Advanced Topics
15
Recommendation Engines
Recommendation Engines
Introducing recommendation systems
Types of recommendation engines
Understanding the limitations of recommendation systems
Areas of practical applications
Practical example – creating a recommendation engine
Summary
16
Algorithmic Strategies for Data Handling
Algorithmic Strategies for Data Handling
Introduction to data algorithms
Presenting the CAP theorem
Decoding data compression algorithms
Practical example: Data management in AWS: A focus on CAP theorem and compression algorithms
Summary
17
Cryptography
Cryptography
Introduction to cryptography
Understanding the types of cryptographic techniques
Example: security concerns when deploying a machine learning model
Summary
18
Large-Scale Algorithms
Large-Scale Algorithms
Introduction to large-scale algorithms
Characterizing performant infrastructure for large-scale algorithms
Strategizing multi-resource processing
Understanding theoretical limitations of parallel computing
How Apache Spark empowers large-scale algorithm processing
Using large-scale algorithms in cloud computing
Summary
19
Practical Considerations
Practical Considerations
Challenges facing algorithmic solutions
Failure of Tay, the Twitter AI bot
The explainability of an algorithm
Understanding ethics and algorithms
Reducing bias in models
When to use algorithms
Summary
20
Other Books You May Enjoy
Other Books You May Enjoy
21
Index
Index

Transformers: the evolution in neural networks after self-attention

Our exploration into self-attention revealed its powerful capability to reinterpret sequence data, providing each word with a contextual understanding based on its relationships with other words. This principle set the stage for an evolutionary leap in neural network designs: the transformer architecture.

Introduced by the Google Brain team in their 2017 paper, Attention is All You Need (https://arxiv.org/abs/1706.03762), the transformer architecture is built upon the very essence of self-attention. Before its advent, RNNs were the go-to. Picture RNNs as diligent librarians reading an English sentence to translate it into German, word by word, ensuring the context is relayed from one word to the next. They’re reliable for short texts but can stumble when sentences get too long, misplacing the essence of earlier words.

transformer-self-attn

Figure 11.7: Encoder-decoder architecture of the original transformer

Transformers...

Previous Section
End of Section 7
Next Section