Book Image

Big Data Forensics: Learning Hadoop Investigations

Book Image

Big Data Forensics: Learning Hadoop Investigations

Overview of this book

Related Content you might be interested in

Current Title:

Small book image
Big Data Forensics: Learning Hadoop Investigations
Aug, 2015 | 264 pages
No titles found
Table of Contents (15 chapters)
Big Data Forensics – Learning Hadoop Investigations
Big Data Forensics – Learning Hadoop Investigations
Credits
Credits
About the Author
About the Author
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Starting Out with Forensic Investigations and Big Data
Starting Out with Forensic Investigations and Big Data
An overview of computer forensics
What is Big Data?
Big Data forensics
Summary
2
Understanding Hadoop Internals and Architecture
Understanding Hadoop Internals and Architecture
The Hadoop architecture
Hadoop data analysis tools
Managing files in Hadoop
The Hadoop forensic evidence ecosystem
Running Hadoop
Summary
3
Identifying Big Data Evidence
Identifying Big Data Evidence
Identifying evidence
Locating sources of data
The chain of custody documentation
Summary
4
Collecting Hadoop Distributed File System Data
Collecting Hadoop Distributed File System Data
Forensically collecting a cluster system
Physical versus remote collections
HDFS collections through the host operating system
The Hadoop shell command collection
Collection via Sqoop
Other HDFS collection approaches
Summary
5
Collecting Hadoop Application Data
Collecting Hadoop Application Data
Application collection approaches
Validating application collections
Collecting Hive evidence
Collecting HBase evidence
Collecting other Hadoop application data and non-Hadoop data
Summary
6
Performing Hadoop Distributed File System Analysis
Performing Hadoop Distributed File System Analysis
The forensic analysis process
Analysis preparation
Analysis
Summary
7
Analyzing Hadoop Application Data
Analyzing Hadoop Application Data
Preparing the analysis environment
Pre-analysis steps
Analyzing data
Summary
8
Presenting Forensic Findings
Presenting Forensic Findings
Types of reports
Developing the report
Testimony and other presentations
Summary
Index
Index

The Hadoop shell command collection

Collecting HDFS data from within the Hadoop layer solves many of the problems that affect host operating system collections. First, the collection only has to be performed from a single machine. By accessing Hadoop through a Hadoop client's command line, all HDFS files are available, so the collection does not involve collecting data from each node individually. Second, the collected data does not require any piecing together or file carving in the analysis phase. The data that is collected is already pieced together as the logical Hadoop files, so no carving or data reconstruction is required.

The following is a list of limitations of collecting HDFS data from the Hadoop shell command line:

  • This method is only possible when Hadoop is online and its command line is accessible

  • Forensic tools such as dd and md5sum cannot easily be used during the collection of the data

  • Deleted data and data in memory that has not been written to disk may not be available

  • Hadoop...

Previous Section
End of Section 5
Next Section