Name: Hadoop 2.x Administration Cookbook
Author: Gurmukh Singh
ISBN: 9781787126879

微信读书书城Hadoop 2.x Administration Cookbook

首页我的书架登录

翻译完成

切换

Hadoop 2.x Administration Cookbook

Gurmukh Singh

扉页
coverpage
+书签
Hadoop 2.x Administration Cookbook
Credits
About the Author
About the Reviewers
www.PacktPub.com
eBooks, discount offers, and more
Customer Feedback
Preface
What this book covers
What you need for this book
Who this book is for
Sections
Conventions
Reader feedback
Customer support
Chapter 1. Hadoop Architecture and Deployment
Introduction
Building and compiling Hadoop
Installation methods
Setting up host resolution
Installing a single-node cluster - HDFS components
Installing a single-node cluster - YARN components
Installing a multi-node cluster
Configuring the Hadoop Gateway node
Decommissioning nodes
Adding nodes to the cluster
Chapter 2. Maintaining Hadoop Cluster HDFS
Introduction
Configuring HDFS block size
Setting up Namenode metadata location
Loading data in HDFS
Configuring HDFS replication
HDFS balancer
Quota configuration
HDFS health and FSCK
Configuring rack awareness
Recycle or trash bin configuration
Distcp usage
Control block report storm
Configuring Datanode heartbeat
Chapter 3. Maintaining Hadoop Cluster – YARN and MapReduce
Introduction
Running a simple MapReduce program
Hadoop streaming
Configuring YARN history server
Job history web interface and metrics
Configuring ResourceManager components
YARN containers and resource allocations
ResourceManager Web UI and JMX metrics
Preserving ResourceManager states
Chapter 4. High Availability
Introduction
Namenode HA using shared storage
ZooKeeper configuration
Namenode HA using Journal node
Resourcemanager HA using ZooKeeper
Rolling upgrade with HA
Configure shared cache manager
Configure HDFS cache
HDFS snapshots
Configuring storage based policies
Configuring HA for Edge nodes
Chapter 5. Schedulers
Introduction
Configuring users and groups
Fair Scheduler configuration
Fair Scheduler pools
Configuring job queues
Job queue ACLs
Configuring Capacity Scheduler
Queuing mappings in Capacity Scheduler
YARN and Mapred commands
YARN label-based scheduling
YARN SLS
Chapter 6. Backup and Recovery
Introduction
Initiating Namenode saveNamespace
Using HDFS Image Viewer
Fetching parameters which are in-effect
Configuring HDFS and YARN logs
Backing up and recovering Namenode
Configuring Secondary Namenode
Promoting Secondary Namenode to Primary
Namenode recovery
Namenode roll edits – online mode
Namenode roll edits – offline mode
Datanode recovery – disk full
Configuring NFS gateway to serve HDFS
Recovering deleted files
Chapter 7. Data Ingestion and Workflow
Introduction
Hive server modes and setup
Using MySQL for Hive metastore
Operating Hive with ZooKeeper
Loading data into Hive
Partitioning and Bucketing in Hive
Hive metastore database
Designing Hive with credential store
Configuring Flume
Configure Oozie and workflows
Chapter 8. Performance Tuning
Tuning the operating system
Tuning the disk
Tuning the network
Tuning HDFS
Tuning Namenode
Tuning Datanode
Configuring YARN for performance
Configuring MapReduce for performance
Hive performance tuning
Benchmarking Hadoop cluster
Chapter 9. HBase Administration
Introduction
Setting up single node HBase cluster
Setting up multi-node HBase cluster
Inserting data into HBase
Integration with Hive
HBase administration commands
HBase backup and restore
Tuning HBase
HBase upgrade
Migrating data from MySQL to HBase using Sqoop
Chapter 10. Cluster Planning
Introduction
Disk space calculations
Nodes needed in the cluster
Memory requirements
Sizing the cluster as per SLA
Network design
Estimating the cost of the Hadoop cluster
Hardware and software options
Chapter 11. Troubleshooting, Diagnostics, and Best Practices
Introduction
Namenode troubleshooting
Datanode troubleshooting
Resourcemanager troubleshooting
Diagnose communication issues
Parse logs for errors
Hive troubleshooting
HBase troubleshooting
Hadoop best practices
Chapter 12. Security
Introduction
Encrypting disk using LUKS
Configuring Hadoop users
HDFS encryption at Rest
Configuring SSL in Hadoop
In-transit encryption
Enabling service level authorization
Securing ZooKeeper
Configuring auditing
Configuring Kerberos server
Configuring and enabling Kerberos for Hadoop
Index

简介

版权

Hadoop 2.x Administration Cookbook