Subscription

Explore Products

Best Sellers

New Releases

Books

Videos

Audiobooks

Learning Hub

Conferences

Free Learning

You're reading from Big Data Analytics with Hadoop 3 Build highly effective analytics solutions to gain valuable insight into your big data

Product type Paperback

Published in May 2018

Publisher Packt

ISBN-13 9781788628846

Length 482 pages

Edition 1st Edition

Languages

Python

Tools

Hadoop

Concepts

Big Data

Author (1):

Sridhar Alla

View More author details

Table of Contents (13) Chapters

Preface

1. Introduction to Hadoop FREE CHAPTER

2. Overview of Big Data Analytics

3. Big Data Processing with MapReduce

4. Scientific Computing and Big Data Analysis with Python and Hadoop

5. Statistical Big Data Computing with R and Hadoop

6. Batch Analytics with Apache Spark

7. Real-Time Analytics with Apache Spark

8. Batch Analytics with Apache Flink

9. Stream Processing with Apache Flink

10. Visualizing Big Data

11. Introduction to Cloud Computing

12. Using Amazon Web Services

Introduction to Hadoop

This chapter introduces the reader to the world of Hadoop and the core components of Hadoop, namely the Hadoop Distributed File System (HDFS) and MapReduce. We will start by introducing the changes and new features in the Hadoop 3 release. Particularly, we will talk about the new features of HDFS and Yet Another Resource Negotiator (YARN), and changes to client applications. Furthermore, we will also install a Hadoop cluster locally and demonstrate the new features such as erasure coding (EC) and the timeline service. As as quick note, Chapter 10, Visualizing Big Data shows you how to create a Hadoop cluster in AWS.

In a nutshell, the following topics will be covered throughout this chapter:

HDFS
- High availability
- Intra-DataNode balancer
- EC
- Port mapping
MapReduce
- Task-level optimization
YARN
- Opportunistic containers
- Timeline service v.2
- Docker containerization
Other changes
Installation of Hadoop 3.1
- HDFS
- YARN
- EC
- Timeline service v.2

You have been reading a chapter from

Big Data Analytics with Hadoop 3

Published in: May 2018

Publisher: Packt

ISBN-13: 9781788628846

A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.

Unlock this book and the full library FREE for 7 days

Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of

Start free trial

Renews at R$50/month. Cancel anytime

Authors (1)

Sridhar Alla

Sridhar?Alla?is the co-founder and CTO of Blue Whale Consulting and is expert at helping companies (big and small) define their vision for systems and capabilities that will allow them to establish a strategic execution plan to deal with the ever-growing data collected to support analytics and product teams. He has very experienced at dealing with all aspects of data collection, security, governance, and processing as part of end-to-end big data analytics and machine learning initiatives (including predictive modeling, deep learning, and ML automation). Sridhar?is a published book author and an avid presenter at numerous conferences, including Strata, Hadoop World, and Spark Summit.? He also has several patents filed with the US PTO on large-scale computing and distributed systems.? He has over 18 years' experience writing code in Scala, Java, C, C++, Python, R, and Go, and has extensive hands-on knowledge of Spark, Flink, TensorFlow, Keras, Hadoop, Cassandra, HBase, MongoDB, Riak, Redis, Zeppelin, Mesos, Docker, Kafka, ElasticSearch, Solr, H2O, machine learning, text analytics, distributed computing, and high-performance computing. Sridhar lives with his wife and daughter in New Jersey and in his spare time loves blogging and coaching organizations on next-generation advancements in technology and their alignment with business goals.

See other products by Sridhar Alla