Book Image

Mastering Malware Analysis

By : Alexey Kleymenov, Amr Thabet
Book Image

Mastering Malware Analysis

By: Alexey Kleymenov, Amr Thabet

Overview of this book

With the ever-growing proliferation of technology, the risk of encountering malicious code or malware has also increased. Malware analysis has become one of the most trending topics in businesses in recent years due to multiple prominent ransomware attacks. Mastering Malware Analysis explains the universal patterns behind different malicious software types and how to analyze them using a variety of approaches. You will learn how to examine malware code and determine the damage it can possibly cause to your systems to ensure that it won't propagate any further. Moving forward, you will cover all aspects of malware analysis for the Windows platform in detail. Next, you will get to grips with obfuscation and anti-disassembly, anti-debugging, as well as anti-virtual machine techniques. This book will help you deal with modern cross-platform malware. Throughout the course of this book, you will explore real-world examples of static and dynamic malware analysis, unpacking and decrypting, and rootkit detection. Finally, this book will help you strengthen your defenses and prevent malware breaches for IoT devices and mobile platforms. By the end of this book, you will have learned to effectively analyze, investigate, and build innovative solutions to handle any malware incidents.
Table of Contents (18 chapters)
Free Chapter
1
Section 1: Fundamental Theory
3
Section 2: Diving Deep into Windows Malware
5
Unpacking, Decryption, and Deobfuscation
9
Section 3: Examining Cross-Platform Malware
13
Section 4: Looking into IoT and Other Platforms

What this book covers

Chapter 1, A Crash Course in CISC/RISC and Programming Basics, offers an insight into all widely used assembly languages, providing foundational knowledge to facilitate further reverse engineering efforts. While many security professionals spend most of their time reversing threats for the IA-32 (x86) platform on Windows as the prevalent source of threats nowadays, other platforms are increasingly gaining in popularity because of a changing landscape of the systems we use: from desktop to mobile, from IA-32 to x64. The main purpose of this part is to show the reader that there is pretty much the same logic behind any assembly language, and moving from one to another is not a problem, as long as you get the general idea of how they work.

Chapter 2, Basic Static and Dynamic Analysis for x86/x64, dives deeper into Windows executable files' inner structure, covering the PE header, PE loading, process and thread creation, and communication between the operating system and this newly created process. This chapter also covers the basic static and dynamic analysis of a malicious sample, and teaches you how to debug and alter its execution path and behavior.

Chapter 3, Unpacking, Decryption, and Deobfuscation, sharpens readers' skills to handle packed, encrypted malware for Windows, and all of the techniques that malware authors use to protect their samples against not experienced reverse engineers. This chapter covers malware packed with various types of packers, as well as detection and unpacking using various simple and advanced techniques. Also, it covers encryption algorithms, from simple XOR algorithms to advanced ones, such as 3DES and AES encryption, for protecting important information such as strings and APIs (especially related to C&C communications), as well as extra modules.

Chapter 4, Inspecting Process Injection and API Hooking, covers advanced techniques implemented in multiple APT, state-sponsored, and widespread cybercrime attacks, from basic process injection to process hollowing and API hooking. In addition, it explains the motivations behind using these techniques, how they work, and how to analyze and work around them.

Chapter 5, Bypassing Anti-Reverse Engineering Techniques, offers a guide on various anti-reverse engineering techniques that malware authors use to protect their samples and this way slow down the reverse engineering process. This chapter reveals a lot of these techniques, from detecting the debugger and other analysis tools to breakpoint detection, virtual machine (VM) detection, and even attacking the anti-malware tools and products. It also covers the VM and sandbox detection techniques that malware authors use to avoid the spam detection and automatic malware detection techniques implemented in various enterprises.

Chapter 6, Understanding Kernel-Mode Rootkits, digs deeper into the Windows kernel and its internal structures and mechanisms. We will be covering different techniques used by malware authors to hide their malware presence from users and antivirus products. We will be looking at different advanced kernel-mode hooking techniques, process injection from kernel mode, and how to perform static and dynamic analysis in kernel mode.

Chapter 7, Handling Exploits and Shellcode, gives the reader an idea of how exploits work in general, discussing the logic behind position-independent code. In addition, we will provide practical tips and tricks on how to analyze the most common file types associated with exploits that are actively used in modern attacks today.

Chapter 8, Reversing Bytecode Languages: .NET, Java, and More, introduces the reader to cross-platform-compiled programs that don't need to be ported to different systems. Here, we will take a look at how malware authors try to leverage these advantages for malign purposes. In addition, the reader will be provided with an arsenal of tools and techniques whose aim is to make the analysis quick and efficient.

Chapter 9, Scripts and Macros: Reversing, Deobfuscation, and Debugging, discusses scripts and macro-based threats. Web incorporated script languages a long time ago, and nowadays, other script languages are also becoming increasingly popular in various projects, from proofs of concepts and prototypes to production-level systems. This chapter will provide an overview of various techniques that script malware authors incorporate in order to complicate the analysis and prolong the infection, and how this can be dealt with.

Chapter 10, Dissecting Linux and IoT Malware, is a hands-on guide to analyzing Linux threats that have become increasingly popular with the growing popularity of IoT devices commonly powered by Linux. Once it became clear that these systems are often less immune to infections due to multiple historical factors, and that it is possible to monetize these weaknesses, the current IoT malware trend emerged. This chapter is dedicated to reverse engineering various pieces of Linux malware, from the now-classic Mirai and its recent modifications to more sophisticated cases.

Chapter 11, Introduction to macOS and iOS Threats, is dedicated to reverse engineering techniques applicable to Apple platforms. Once considered as virtually immune to any infection, nowadays, we see more and more attempts to compromise the security of the users of these platforms. While still relatively immature, the significance of this trend shouldn't be underestimated, especially with the rise of APT attacks.

Chapter 12, Analyzing Android Malware Samples, teaches the reader how to deal with Android malware, walking through the most common patterns and providing detailed guidelines on how to analyze them. As our lives become more and more dynamic, the world is gradually shifting from desktop to mobile systems. As a result, more and more of our valuable data, from personal information to financial access codes, is stored on phones and tablets and eventually attracts malicious actors, thereby creating a demand for reverse engineers experienced with this platform.