Search icon CANCEL
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Mastering Hadoop 3

You're reading from   Mastering Hadoop 3 Big data processing at scale to unlock unique business insights

Arrow left icon
Product type Paperback
Published in Feb 2019
Publisher Packt
ISBN-13 9781788620444
Length 544 pages
Edition 1st Edition
Languages
Tools
Concepts
Arrow right icon
Authors (3):
Arrow left icon
Timothy Wong Timothy Wong
Author Profile Icon Timothy Wong
Timothy Wong
Manish Kumar Manish Kumar
Author Profile Icon Manish Kumar
Manish Kumar
Chanchal Singh Chanchal Singh
Author Profile Icon Chanchal Singh
Chanchal Singh
Arrow right icon
View More author details
Toc

Table of Contents (21) Chapters Close

Preface 1. Section 1: Introduction to Hadoop 3 FREE CHAPTER
2. Journey to Hadoop 3 3. Deep Dive into the Hadoop Distributed File System 4. YARN Resource Management in Hadoop 5. Internals of MapReduce 6. Section 2: Hadoop Ecosystem
7. SQL on Hadoop 8. Real-Time Processing Engines 9. Widely Used Hadoop Ecosystem Components 10. Section 3: Hadoop in the Real World
11. Designing Applications in Hadoop 12. Real-Time Stream Processing in Hadoop 13. Machine Learning in Hadoop 14. Hadoop in the Cloud 15. Hadoop Cluster Profiling 16. Section 4: Securing Hadoop
17. Who Can Do What in Hadoop 18. Network and Data Security 19. Monitoring Hadoop 20. Other Books You May Enjoy

MapReduce workflow in the Hadoop framework

The MapReduce execution goes through various steps and each step has scope for a little optimization. In the previous sections, we have covered the components of the MapReduce framework and now we will briefly look into the MapReduce execution flow, which will help us understand how each component interacts with each other. The following diagram gives a brief overview about the MapReduce execution flow. We have divided the diagram into smaller parts so that each step looks easier to understand. The step numbers are mentioned over arrow connectors and the last arrow in the diagram connects to the following diagram in the section: 

We will explain the different steps of the MapReduce internal flow here as follows:

  1. The InputFormat is the starting point of any MapReduce application. It is defined in the job configuration in the...
lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at R$50/month. Cancel anytime