Book Image

Binary Analysis Cookbook

By : Michael Born
Book Image

Binary Analysis Cookbook

By: Michael Born

Overview of this book

Binary analysis is the process of examining a binary program to determine information security actions. It is a complex, constantly evolving, and challenging topic that crosses over into several domains of information technology and security. This binary analysis book is designed to help you get started with the basics, before gradually advancing to challenging topics. Using a recipe-based approach, this book guides you through building a lab of virtual machines and installing tools to analyze binaries effectively. You'll begin by learning about the IA32 and ELF32 as well as IA64 and ELF64 specifications. The book will then guide you in developing a methodology and exploring a variety of tools for Linux binary analysis. As you advance, you'll learn how to analyze malicious 32-bit and 64-bit binaries and identify vulnerabilities. You'll even examine obfuscation and anti-analysis techniques, analyze polymorphed malicious binaries, and get a high-level overview of dynamic taint analysis and binary instrumentation concepts. By the end of the book, you'll have gained comprehensive insights into binary analysis concepts and have developed the foundational skills to confidently delve into the realm of binary analysis.
Table of Contents (12 chapters)

Encoding and decoding

Another technique that we can use to try and evade signature detection is to use encoding to mask the bytes in a binary. This technique, however, does require a decoding stub in order to unmask the bytes so that the binary executes as originally intended.

From an analysis perspective, we need to understand what encoding and decoding looks like, especially from a static or dynamic analysis perspective. When it comes to signature detection, as we'll see, encoding and decoding may work in certain situations. It's becoming less and less frequent that it does, but every now and then, I'll find an encoding scheme that works, albeit increasingly rarely. Still, this is a good skill to have during analysis.

In this recipe, we'll look at an encoded version of our reverse shell and identify the decoder stub during our analysis. We won't cover...