Book Image

Mastering Reverse Engineering

By : Reginald Wong
Book Image

Mastering Reverse Engineering

By: Reginald Wong

Overview of this book

If you want to analyze software in order to exploit its weaknesses and strengthen its defenses, then you should explore reverse engineering. Reverse Engineering is a hackerfriendly tool used to expose security flaws and questionable privacy practices.In this book, you will learn how to analyse software even without having access to its source code or design documents. You will start off by learning the low-level language used to communicate with the computer and then move on to covering reverse engineering techniques. Next, you will explore analysis techniques using real-world tools such as IDA Pro and x86dbg. As you progress through the chapters, you will walk through use cases encountered in reverse engineering, such as encryption and compression, used to obfuscate code, and how to to identify and overcome anti-debugging and anti-analysis tricks. Lastly, you will learn how to analyse other types of files that contain code. By the end of this book, you will have the confidence to perform reverse engineering.
Table of Contents (20 chapters)
Title Page
Copyright and Credits
Packt Upsell
Contributors
Preface
Index

Preface

Reverse engineering is a tool used for analyzing software to exploit its weaknesses and strengthen its defenses. Hackers use reverse engineering as a tool to expose security flaws and questionable privacy practices. This book helps you to master the art of using reverse engineering.

Who this book is for

If you are a security engineer, analyst, or system programmer and want to use reverse engineering to improve your software and hardware, this is the book for you. You will also find this book useful if you are a developer who wants to explore and learn reverse engineering. 

What this book covers

Chapter 1, Preparing to Reverse,shows how to obtain the samples used throughout the book and explains the journey we are about to embark on.

Chapter 2, Identification and Extraction of Hidden Components, covers basics of the operating system and malware installation behavior. We will learn where malware usually drops files and makes registry entries.

Chapter 3, The Low-Level Language, briefly covers the Assembly language and why we must understand it in order to reverse engineer.

Chapter 4, Static and Dynamic Reversing,explains how static and dynamic analysis are implemented. We will also have a brief discussion regarding reversing of a file using a few tools.

Chapter 5, Tools of the Trade, compares and contrasts tools of the trade and explains their weaknesses and when a tool won't work as intended, allowing you to change your tools and know where to turn to get the job done without blaming a tool for lacking a capability.

Chapter 6, RE in Linux Platforms, explains how to perform a static and dynamic Windows analysis in a Linux environment.

Chapter 7, RE for Windows Platforms, explains how to perform static and dynamic Windows analysis directly in a Windows environment.

 

Chapter 8, Sandboxing: Virtualization as a Component for RE, shows how to use emulation to inform reverse engineering and overcome obstacles when running on hardware other than the target binary supports.

Chapter 9, Binary Obfuscation Techniques, explains how to reverse engineer simple obfuscation techniques.

Chapter 10, Packing and Encryption, covers using debuggers to pause execution and dump the contents of memory for analysis using our disassembly tools.

Chapter 11, Anti-analysis tricks, shows how to identify and handle anti-reversing and anti-debugging tricks.

Chapter 12, Practical Reverse Engineering of a Windows Executable, covers practical use of the tools we are familiar with at this point.

Chapter 13, Reversing Various File Types, covers analyzing various file types using up-to-date tools.

To get the most out of this book

  • Having some programming/shell scripting knowledge is an added bonus.
  • Knowledge about information security and x86 assembly language is an advantage.
  • Operating system used: Windows and Linux ( version will depend on the requirements of VirtualBox)
  • Processor with at least four cores, 4 GB of RAM, and 250 GB of disk space.
  • You may need to download virtual machines from Microsoft in advance, as these may take some time to download. See the developers' page at https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/.

Download the example code files

You can download the example code files for this book from your account at www.packt.com. If you purchased this book elsewhere, you can visit www.packt.com/support and register to have the files emailed directly to you.

 

You can download the code files by following these steps:

  1. Log in or register at www.packt.com.
  2. Select the SUPPORT tab.
  3. Click on Code Downloads & Errata.
  4. Enter the name of the book in the Search box and follow the onscreen instructions.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

  • WinRAR/7-Zip for Windows
  • Zipeg/iZip/UnRarX for Mac
  • 7-Zip/PeaZip for Linux

The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Mastering-Reverse-Engineering. In case there's an update to the code, it will be updated on the existing GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://www.packtpub.com/sites/default/files/downloads/9781788838849_ColorImages.pdf

Conventions used

There are a number of text conventions used throughout this book.

CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "The handle in hkResult is used by RegEnumValueA to begin enumerating each registry value under the registry key."

A block of code is set as follows:

 while (true) {
    for (char i = 1; i <= 255; i++) {
      if (GetAsyncKeyState(i) & 1) {
        sprintf_s(lpBuffer, "\\x%02x", i);
        LogFile(lpBuffer, (char*)"log.txt");
      }
    }

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

87 to base-2
87 divided by 2 is 43 remainder 1.
43 divided by 2 is 21 remainder 1.
21 divided by 2 is 10 remainder 1.
10 divided by 2 is 5 remainder 0.
5 divided by 2 is 2 remainder 1.

Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: "In VirtualBox, click on File|Import Appliance."

Note

Warnings or important notes appear like this.

Note

Tips and tricks appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, mention the book title in the subject of your message and email us at [email protected].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packt.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.

Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Reviews

Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!

For more information about Packt, please visit packt.com.