Book Image

Binary Analysis Cookbook

By : Michael Born
Book Image

Binary Analysis Cookbook

By: Michael Born

Overview of this book

Binary analysis is the process of examining a binary program to determine information security actions. It is a complex, constantly evolving, and challenging topic that crosses over into several domains of information technology and security. This binary analysis book is designed to help you get started with the basics, before gradually advancing to challenging topics. Using a recipe-based approach, this book guides you through building a lab of virtual machines and installing tools to analyze binaries effectively. You'll begin by learning about the IA32 and ELF32 as well as IA64 and ELF64 specifications. The book will then guide you in developing a methodology and exploring a variety of tools for Linux binary analysis. As you advance, you'll learn how to analyze malicious 32-bit and 64-bit binaries and identify vulnerabilities. You'll even examine obfuscation and anti-analysis techniques, analyze polymorphed malicious binaries, and get a high-level overview of dynamic taint analysis and binary instrumentation concepts. By the end of the book, you'll have gained comprehensive insights into binary analysis concepts and have developed the foundational skills to confidently delve into the realm of binary analysis.

Related Content you might be interested in

Current Title:

Small book image
Binary Analysis Cookbook
Michael Born
Sep, 2019 | 396 pages
Small book image
Offensive Shellcode from Scratch.
Rishalin Pillay
Apr, 2022 | 208 pages
Small book image
Penetration Testing with Shellcode
Hamza Megahed
Feb, 2018 | 346 pages
Small book image
Mastering Assembly Programming
Alexey Lyashko
Sep, 2017 | 290 pages
Table of Contents (12 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Sections
Get in touch
Free Chapter
1
Setting Up the Lab
Setting Up the Lab
Installing VirtualBox on Windows
Installing VirtualBox on Mac
Installing VirtualBox on Ubuntu
Installing a 32-bit Ubuntu 16.04 LTS Desktop virtual machine
Installing a 64-bit Ubuntu 16.04 LTS Desktop virtual machine
Installing the dependencies and the tools
Installing the code examples
Installing the EDB Debugger
Taking a snapshot of the virtual machines
2
32-bit Assembly on Linux and the ELF Specification
32-bit Assembly on Linux and the ELF Specification
Technical requirements
Differences between Intel and AT&T syntax
Introduction to the IA-32 registers
Introducing common IA-32 instructions
Making IA-32 system calls on Linux
Introducing the ELF 32-bit specification
3
64-bit Assembly on Linux and the ELF Specification
64-bit Assembly on Linux and the ELF Specification
Technical requirements
Introducing the IA64 registers
Introducing common IA64 instructions
Making IA64 system calls on Linux
Introducing the ELF 64-bit specification
4
Creating a Binary Analysis Methodology
Creating a Binary Analysis Methodology
Technical requirements
Performing binary discovery
Information gathering
Static analysis
Dynamic analysis
Iterating each step
Automating methodology tasks
Adapting the methodology steps
5
Linux Tools for Binary Analysis
Linux Tools for Binary Analysis
Technical requirements
Using file
Using strings
Using readelf
Using nm
Using objcopy
Using objdump
Using ltrace and strace
Using data duplicator (dd)
Using the GNU Debugger (GDB)
Using Evan's Debugger (EDB)
6
Analyzing a Simple Bind Shell
Analyzing a Simple Bind Shell
Technical requirements
Performing discovery
Gathering information
Performing static analysis
Using ltrace and strace
Using GDB for dynamic analysis
Finishing dynamic analysis
7
Analyzing a Simple Reverse Shell
Analyzing a Simple Reverse Shell
Technical requirements
Automating the initial phases
Static analysis with objdump
Editing the binary
Using GDB TUI mode
Continuing with dynamic analysis
Analyzing the execve system call
8
Identifying Vulnerabilities
Identifying Vulnerabilities
Technical requirements
Automating the initial phases
Extended static analysis
Identifying hard coded credentials with ltrace
Identifying hard coded credentials with a debugger
Validating a stack-based buffer overflow
9
Understanding Anti-Analysis Techniques
Understanding Anti-Analysis Techniques
Technical requirements
Understanding signature detection
Changing a binary's signature
Confusing static analysis tools
Encoding and decoding
10
A Simple Reverse Shell With Polymorphism
A Simple Reverse Shell With Polymorphism
Technical requirements
Automating the initial phases
Performing static analysis
Using EDB for dynamic analysis
Analyzing deobfuscation loops
Wrapping up dynamic analysis
11
Another Book You May Enjoy
Another Book You May Enjoy
Leave a review - let other readers know what you think

Installing a 64-bit Ubuntu 16.04 LTS Desktop virtual machine

Machines that support 64-bit operations are the norm nowadays, so it makes sense that we cover 64-bit binary analysis more extensively in this book. In order to do so, though, we need a viable virtual machine to work through the examples that will be presented in later chapters.

The following recipe will guide you through creating and configuring Ubuntu 16.04 LTS Desktop 64-bit as a virtual machine in VirtualBox. This virtual machine will get used extensively when we work through all of the 64-bit recipes that will be presented in later chapters.

Getting ready

Using a browser, download the 64-bit Ubuntu 16.04 LTS Desktop ISO file from the following location: http://releases.ubuntu.com/xenial/.

Why 32-bit as well as 64-bit? The answer is simple. When I was diving into the subject of learning Intel assembly on Linux a few years ago, I immediately experienced the benefits of learning 32-bit first, before taking on 64-bit. Besides, once we start covering analysis in 64-bit, you may need to recall some of those 32-bit registers. Assembly is particular about the processor and operating system you're running. Because of the differences in 32-bit assembly and 64-bit assembly on Linux, we'll need both operating system architectures and a processor that supports both.

How to do it...

The following instructions will guide you through creating and configuring Ubuntu 16.04 LTS Desktop 64-bit as a virtual machine in the newly installed VirtualBox:

  1. Open VirtualBox if it's not already open.
  2. Once the application launches, click on the New icon to begin configuring a new virtual machine.
  1. A new window called Name and operating system will appear, asking you to provide a name, virtual machine folder location, type, and version. Name the virtual machine BAC64, choose a Machine Folder location according to your storage needs, choose Linux from the Type: drop-down menu, and choose Ubuntu (64-bit) from the Version: drop-down menu. Once complete, click on Continue.
  2. In the Memory size window, set the memory size (RAM) options appropriate for your hardware, and click Continue. I used 4,096 MB since this will be a 64-bit virtual machine. You are welcome to increase this amount if your own host can support it, but I wouldn't configure this setting to any lower than 4,096 MB.
  3. In the Hard disk, keep the Create a virtual hard disk now option selected and click Create.
  4. A new window will appear titled Hard disk file type. Since, at some point in the future, we may need to switch to another virtualization platform, such as VMware Workstation, we will select VMDK (Virtual Machine Disk) and click Create.
  5. For the Storage on physical hard disk window, we will select the Dynamically allocated option and click Continue.
  6. In the File location and size window, choose the size of the virtual hard drive according to your storage restrictions and then click Create. I typically use 40 GB for my virtual machines in my lab and usually never fill that space. Since we selected the Dynamically allocated option in the previous step, this setting will allow us up to the amount we configure but will not use it all at once.
  7. Now, we will return to the Oracle VM VirtualBox Manager window, where we will see our newly created virtual machine. Make sure BAC64 is highlighted along the left-hand side, and then click Settings.
  1. The general settings window will be displayed. From here, click on the Storage icon (marked 1. in the following screenshot). Underneath the words Controller: IDE along the left-hand side, there will be a CD icon with the words Empty (marked 2. in the following screenshot). Click on that and a new subsection of the current window will appear along the right-hand side called Attributes. Next to the Optical Drive: drop-down, click the blue CD icon (marked 3.):
  1. In the pop-up menu that appears, select the Choose Virtual Optical Disk File option.
  2. A file selection window will appear. Navigate to the Ubuntu 16.04 Desktop 64-bit ISO file we downloaded previously, select it, and click Open.
  3. In the Storage settings window, click OK to accept the configuration.
  4. Back inside the Oracle VM VirtualBox Manager window, highlight the BAC64 virtual machine we just created along the left-hand side of the window and click the Start icon. This will start the virtual machine and will boot into the Ubuntu ISO.
  1. Follow the installation prompts within the virtual machine to install Ubuntu Desktop 16.04 LTS 64-bit. The default options are sufficient enough for this book. When you're prompted to set the hostname for the installation, name it bac64. This will help us to discern which virtual machine we need to use for the examples later in this book. At the end of the installation process, Ubuntu will ask you to hit Enter to reboot. Do so. Once rebooted, you'll have a working virtual machine.

How it works...

After acquiring the correct Ubuntu 16.04 LTS Desktop 64-bit ISO file, we told VirtualBox we wanted to create and configure a new virtual machine. VirtualBox presented various configuration options, to which we responded with the correct settings to install a 64-bit version of Ubuntu Linux as the operating system for the virtual machine. VirtualBox took those settings and guided us through the rest of the configuration options for naming the virtual machine, what size to configure the virtual hard drive at, how much virtual RAM we wanted VirtualBox to provision for this virtual machine, where to store the files associated with this virtual machine, and finally, to configure which ISO file to use for installing Ubuntu 16.04 LTS Desktop 64-bit. After all of that, we launched the virtual machine in order to actually work through the installation process for Ubuntu itself. Now, we have a working 64-bit Ubuntu virtual machine and are ready to install the tools and dependencies, along with the code examples for this book.

There's more...

If you plan on altering the resolution of this virtual machine, and you want to enable copy/paste between this virtual machine and your host operating system, feel free to install the Guest Additions. In the virtual machine menu bar, select Devices | Insert Guest Additions CD Image... and follow the installation prompts.

See also

There are many more operating systems you can install as virtual machines in VirtualBox. Windows, other Linux distributions, and virtual appliances are all available and are only limited by your research needs. I happen to like to run Windows Desktop as a virtual machine for research purposes, along with Kali Linux when I perform penetration assessments. Having both as virtual machines allows me to quickly revert back to previously saved snapshots, which we will cover later in this chapter, in order to start from a clean slate for the next penetration assessment I need to perform. I recommend doing this so that you always have a clean virtual machine to revert back to in the event something goes wrong while you're analyzing binaries or upgrading the operating system.

Previous Section
End of Section 6
Next Section