Book Image

Mastering Reverse Engineering

By : Reginald Wong
Book Image

Mastering Reverse Engineering

By: Reginald Wong

Overview of this book

If you want to analyze software in order to exploit its weaknesses and strengthen its defenses, then you should explore reverse engineering. Reverse Engineering is a hackerfriendly tool used to expose security flaws and questionable privacy practices.In this book, you will learn how to analyse software even without having access to its source code or design documents. You will start off by learning the low-level language used to communicate with the computer and then move on to covering reverse engineering techniques. Next, you will explore analysis techniques using real-world tools such as IDA Pro and x86dbg. As you progress through the chapters, you will walk through use cases encountered in reverse engineering, such as encryption and compression, used to obfuscate code, and how to to identify and overcome anti-debugging and anti-analysis tricks. Lastly, you will learn how to analyse other types of files that contain code. By the end of this book, you will have the confidence to perform reverse engineering.
Table of Contents (20 chapters)
Title Page
Copyright and Credits
Packt Upsell
Contributors
Preface
Index

Anti-debugging tricks


Anti-debugging tricks are meantto ensure that the codes are not working under the influence of a debugger. Say we have a program with an anti-debugging code in it. The behavior of the program is just as if it were running without an anti-debugging code. The story becomes different, however, when the program is being debugged. While debugging, we encounter code that goes straight to exiting the program or jumps into code that doesn't make sense. This process is illustrated in the following diagram:

Developing anti-debugging code requires understanding the traits of the program and the system, both when normally running and when being debugged. For example, the Process Environment Block (PEB) contains a flag that is set when a program is being run under a debugger. Another popular trick is to use a Structured Exception Handler (SEH) to continue code that forces an error exception while debugging. To better understand how these work, let's discuss these tricks in a little...