Book Image

Learning Linux Binary Analysis

By : Ryan "elfmaster" O'Neill
Book Image

Learning Linux Binary Analysis

By: Ryan "elfmaster" O'Neill

Overview of this book

Learning Linux Binary Analysis is packed with knowledge and code that will teach you the inner workings of the ELF format, and the methods used by hackers and security analysts for virus analysis, binary patching, software protection and more. This book will start by taking you through UNIX/Linux object utilities, and will move on to teaching you all about the ELF specimen. You will learn about process tracing, and will explore the different types of Linux and UNIX viruses, and how you can make use of ELF Virus Technology to deal with them. The latter half of the book discusses the usage of Kprobe instrumentation for kernel hacking, code patching, and debugging. You will discover how to detect and disinfect kernel-mode rootkits, and move on to analyze static code. Finally, you will be walked through complex userspace memory infection analysis. This book will lead you into territory that is uncharted even by some experts; right into the world of the computer hacker.
Table of Contents (17 chapters)
Learning Linux Binary Analysis
Credits
About the Author
Acknowledgments
About the Reviewers
www.PacktPub.com
Preface
Index

Obfuscation methods


A binary can be obfuscated or encrypted in many creative ways. Most binary protectors simply protect the entire binary with one or more layers of protection. At runtime, the binary is decrypted and can be dumped from the memory to acquire a copy of the unpacked binary. In more advanced protectors, such as Maya, every single function is encrypted individually, and allows only a single function to be decrypted at any given time.

Once a binary is encrypted, it must, of course, store the encryption keys somewhere. In the case of Maya (discussed earlier), a custom heap implementation that itself uses encryption to store encryption keys was designed. At some point, it would seem that a key has to be exposed (such as the key used to decrypt another key), but special techniques such as white-box cryptography can be used to make these final keys extremely obfuscated. If assistance from the kernel is used in a protector, then it is possible to store the key outside of the binary...