Book Image

Learning Linux Binary Analysis

By : Ryan "elfmaster" O'Neill
Book Image

Learning Linux Binary Analysis

By: Ryan "elfmaster" O'Neill

Overview of this book

Learning Linux Binary Analysis is packed with knowledge and code that will teach you the inner workings of the ELF format, and the methods used by hackers and security analysts for virus analysis, binary patching, software protection and more. This book will start by taking you through UNIX/Linux object utilities, and will move on to teaching you all about the ELF specimen. You will learn about process tracing, and will explore the different types of Linux and UNIX viruses, and how you can make use of ELF Virus Technology to deal with them. The latter half of the book discusses the usage of Kprobe instrumentation for kernel hacking, code patching, and debugging. You will discover how to detect and disinfect kernel-mode rootkits, and move on to analyze static code. Finally, you will be walked through complex userspace memory infection analysis. This book will lead you into territory that is uncharted even by some experts; right into the world of the computer hacker.
Table of Contents (17 chapters)
Learning Linux Binary Analysis
Credits
About the Author
Acknowledgments
About the Reviewers
www.PacktPub.com
Preface
Index

Protecting control flow integrity


A protected binary should aim to protect the program during runtime (the process itself) just as much as—if not more than—the binary at rest on the disk. Runtime attacks can generally be classified into two types:

  • Attacks based on ptrace

  • Vulnerability-based attacks

Attacks based on ptrace

The first variety, ptrace based attacks, also falls under the category of debugging a process. As already discussed, a binary protector wants to make ptrace based debugging very difficult for a reverse engineer. Aside from debugging, however, there are many other attacks that could potentially help break a protected binary, and it is important to know and understand what some of these are in order to give further clarification as to why a binary protector wants to protect a running process from ptrace.

If a protector has gone so far that it is able to detect breakpoint instructions (and therefore make debugging more difficult) but is not able to protect itself from being traced...