Book Image

Big Data Forensics: Learning Hadoop Investigations

Book Image

Big Data Forensics: Learning Hadoop Investigations

Overview of this book

Related Content you might be interested in

Current Title:

Small book image
Big Data Forensics: Learning Hadoop Investigations
Aug, 2015 | 264 pages
No titles found
Table of Contents (15 chapters)
Big Data Forensics – Learning Hadoop Investigations
Big Data Forensics – Learning Hadoop Investigations
Credits
Credits
About the Author
About the Author
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Starting Out with Forensic Investigations and Big Data
Starting Out with Forensic Investigations and Big Data
An overview of computer forensics
What is Big Data?
Big Data forensics
Summary
2
Understanding Hadoop Internals and Architecture
Understanding Hadoop Internals and Architecture
The Hadoop architecture
Hadoop data analysis tools
Managing files in Hadoop
The Hadoop forensic evidence ecosystem
Running Hadoop
Summary
3
Identifying Big Data Evidence
Identifying Big Data Evidence
Identifying evidence
Locating sources of data
The chain of custody documentation
Summary
4
Collecting Hadoop Distributed File System Data
Collecting Hadoop Distributed File System Data
Forensically collecting a cluster system
Physical versus remote collections
HDFS collections through the host operating system
The Hadoop shell command collection
Collection via Sqoop
Other HDFS collection approaches
Summary
5
Collecting Hadoop Application Data
Collecting Hadoop Application Data
Application collection approaches
Validating application collections
Collecting Hive evidence
Collecting HBase evidence
Collecting other Hadoop application data and non-Hadoop data
Summary
6
Performing Hadoop Distributed File System Analysis
Performing Hadoop Distributed File System Analysis
The forensic analysis process
Analysis preparation
Analysis
Summary
7
Analyzing Hadoop Application Data
Analyzing Hadoop Application Data
Preparing the analysis environment
Pre-analysis steps
Analyzing data
Summary
8
Presenting Forensic Findings
Presenting Forensic Findings
Types of reports
Developing the report
Testimony and other presentations
Summary
Index
Index

Managing files in Hadoop

Hadoop has its own file management concepts that come with many different mechanisms for data storage and retrieval. Hadoop is designed to manage large volumes of data distributed across many nodes built with commodity hardware. As such, Hadoop manages the distribution of large volumes of data using techniques designed to divide, compress, and share the data all while dealing with the possibilities of node failures and numerous processes accessing the same data simultaneously. Many of the filesystem concepts in Hadoop are exactly the same as in other systems, such as directory structures. However, other concepts, such as MapFiles and Hadoop Archive Files, are unique to Hadoop. This section covers many of the file management concepts that are unique to Hadoop.

File permissions

HDFS uses a standard file permission approach. The three types of permissions for files and directories are:

  • Read (r): Read a file and list a directory's contents

  • Write (w): Write to a file and...

Previous Section
End of Section 4
Next Section