Book Image

Mastering Python Forensics

Book Image

Mastering Python Forensics

Overview of this book

Digital forensic analysis is the process of examining and extracting data digitally and examining it. Python has the combination of power, expressiveness, and ease of use that makes it an essential complementary tool to the traditional, off-the-shelf digital forensic tools. This book will teach you how to perform forensic analysis and investigations by exploring the capabilities of various Python libraries. The book starts by explaining the building blocks of the Python programming language, especially ctypes in-depth, along with how to automate typical tasks in file system analysis, common correlation tasks to discover anomalies, as well as templates for investigations. Next, we’ll show you cryptographic algorithms that can be used during forensic investigations to check for known files or to compare suspicious files with online services such as VirusTotal or Mobile-Sandbox. Moving on, you’ll learn how to sniff on the network, generate and analyze network flows, and perform log correlation with the help of Python scripts and tools. You’ll get to know about the concepts of virtualization and how virtualization influences IT forensics, and you’ll discover how to perform forensic analysis of a jailbroken/rooted mobile device that is based on iOS or Android. Finally, the book teaches you how to analyze volatile memory and search for known malware samples based on YARA rules.
Table of Contents (14 chapters)

Setting up the Lab


As a base for our scripts and investigations, we need a comprehensive and powerful lab environment that is able to handle a large number of different file types and structures as well as connections to mobile devices. To achieve this goal, we will use the latest Ubuntu LTS version 14.04.2 and install it in a virtual machine (VM). Within the following sections, we will explain the setup of the VM and introduce Python virtualenv, which we will use to establish our working environment.

Ubuntu

To work in a similar lab environment, we suggest you to download a copy of the latest Ubuntu LTS Desktop Distribution from http://www.ubuntu.com/download/desktop/, preferably the 32-bit version. The distribution provides a simple-to-use UI and already has the Python 2.7.6 environment installed and preconfigured. Throughout the book, we will use Python 2.7.x and not the newer 3.x versions. Several examples and case studies in this book will rely on the tools or libraries that are already a part of the Ubuntu distribution. When a chapter or section of the book requires a third-party package or library, we will provide the additional information on how to install it in the virtualenv (the setup of this environment will be explained in the next section) or on Ubuntu in general.

For better performance of the system, we recommend that the virtual machine that is used for the lab has at least 4 GB of volatile memory and about 40 GB of storage.

Figure 1: The Atom editor

To write your first Python script, you can use a simple editor such as vi or a powerful but cluttered IDE such as eclipse. As a really powerful alternative, we would suggest you to use atom, a very clean but highly customizable editor that can be freely downloaded from https://atom.io/.

Python virtual environment (virtualenv)

According to the official Python documentation, Virtual Environment is a tool to keep the dependencies required by different projects in separate places by creating virtual Python environments for them. It solves the "Project X depends on version 1.x, but Project Y needs 4.x" dilemma and keeps your global site-packages directory clean and manageable.

This is also what we will use in the following chapters to keep a common environment for all the readers of the book and not run into any compatibility issues. First of all, we have to install the virtualenv package. This is done by the following command:

user@lab:~$ pip install virtualenv

We will now create a folder in the users' home directory for our virtual Python environment. This directory will contain the executable Python files and a copy of the pip library, which can be used to install other packages in the environment. The name of the virtual environment (in our case, it is called labenv) can be of your choice. Our virtual lab environment can be created by executing the following command:

user@lab:~$ virtualenv labenv
New python executable in labenv/bin/python
Installing setuptools, pip...done.

To start working with the new lab environment, it first needs to be activated. This can be done through:

user@lab:~$ source labenv/bin/activate
(labenv)user@lab:~$

Now, you can see that the command prompt starts with the name of the virtual environment that we activated. From now on, any package that you install using pip will be placed in the labenv folder, isolated from the global Python installation in the underlying Ubuntu.

Throughout the book, we will use this virtual python environment and install new packages and libraries in it from time to time. So, every time you try to recap a shown example remember or challenge to change into the labenv environment before running your scripts.

If you are done working in the virtual environment for the moment and you want to return to your "normal" Python environment, you can deactivate the virtual environment by executing the following command:

(labenv)user@lab:~$ deactivate
user@lab:~$

This puts you back in the system's default Python interpreter with all its installed libraries and dependencies.

If you are using more than one virtual or physical machine for the investigations, the virtual environments can help you to keep your libraries and packages synced with all these workplaces. In order to ensure that your environments are consistent, it's a good idea to "freeze" the current state of environment packages. To do this, just run:

(labenv)user@lab:~$ pip freeze > requirenments.txt

This will create a requirements.txt file, which contains a simple list of all the packages in the current environment and their respective versions. If you want to now install the same packages using the same version on a different machine, just copy the requirements.txt file to the desired machine, create the labenv environment as described earlier and execute the following command:

(labenv)user@lab:~$ pip install -r requirements.txt

Now, you will have consistent Python environments on all the machines and don't need to worry about different library versions or other dependencies.

After we have created the Ubuntu virtual machine with our dedicated lab environment, we are nearly ready to start our first forensic analysis. But before that, we need more knowledge of the helpful Python libraries and backgrounds. Therefore, we will start with an introduction to the Python ctypes in the following section.