Book Image

Learning Linux Binary Analysis

By : Ryan "elfmaster" O'Neill
Book Image

Learning Linux Binary Analysis

By: Ryan "elfmaster" O'Neill

Overview of this book

Learning Linux Binary Analysis is packed with knowledge and code that will teach you the inner workings of the ELF format, and the methods used by hackers and security analysts for virus analysis, binary patching, software protection and more. This book will start by taking you through UNIX/Linux object utilities, and will move on to teaching you all about the ELF specimen. You will learn about process tracing, and will explore the different types of Linux and UNIX viruses, and how you can make use of ELF Virus Technology to deal with them. The latter half of the book discusses the usage of Kprobe instrumentation for kernel hacking, code patching, and debugging. You will discover how to detect and disinfect kernel-mode rootkits, and move on to analyze static code. Finally, you will be walked through complex userspace memory infection analysis. This book will lead you into territory that is uncharted even by some experts; right into the world of the computer hacker.
Table of Contents (17 chapters)
Learning Linux Binary Analysis
Credits
About the Author
Acknowledgments
About the Reviewers
www.PacktPub.com
Preface
Index

Stub mechanics and the userland exec


First, it is necessary to understand that a software protector is actually made up of two programs:

  • Protection phase code: The program that applies the protection to the target binary

  • Runtime engine or stub: The program that is merged with the target binary that is responsible for deobfuscation and anti-debugging at runtime

The protector program can vary greatly depending on the types of protection that are being applied to the target binary. Whatever type of protection is being applied to the target binary must be understood by the runtime code. The runtime code (or stub) must know how to decrypt or deobfuscate the binary that it is merged with. In most cases of software protection, there is a relatively simple runtime engine merged with the protected binary; its sole purpose is to decrypt the binary and pass control to the decrypted binary in memory.

This type of runtime engine is not so much an engine—really—and we call it a stub. The stub is generally...