Subscription

Explore Products

Best Sellers

New Releases

Books

Videos

Audiobooks

Learning Hub

Conferences

Free Learning

You're reading from Scala and Spark for Big Data Analytics Explore the concepts of functional programming, data streaming, and machine learning

Product type Paperback

Published in Jul 2017

Publisher Packt

ISBN-13 9781785280849

Length 796 pages

Edition 1st Edition

Languages

Scala

Tools

Apache Spark

Concepts

Big Data

Authors (2):

Sridhar Alla

Md. Rezaul Karim

View More author details

Table of Contents (19) Chapters

Preface

1. Introduction to Scala

2. Object-Oriented Scala FREE CHAPTER

3. Functional Programming Concepts

4. Collection APIs

5. Tackle Big Data – Spark Comes to the Party

6. Start Working with Spark – REPL and RDDs

7. Special RDD Operations

8. Introduce a Little Structure - Spark SQL

9. Stream Me Up, Scotty - Spark Streaming

10. Everything is Connected - GraphX

11. Learning Machine Learning - Spark MLlib and Spark ML

12. My Name is Bayes, Naive Bayes

13. Time to Put Some Order - Cluster Your Data with Spark MLlib

14. Text Analytics Using Spark ML

15. Spark Tuning

16. Time to Go to ClusterLand - Deploying Spark on a Cluster

17. Testing and Debugging Spark

18. PySpark and SparkR

Summary

In this chapter, we discussed how Spark works in a cluster mode with its underlying architecture. You also saw how to deploy a full Spark application on a cluster. You saw how to deploy cluster for running Spark application in different cluster modes such as local, standalone, YARN, and Mesos. Finally, you saw how to configure Spark cluster on AWS using EC2 script. We believe that this chapter will help you to gain some good understanding of Spark. Nevertheless, due to page limitation, we could not cover many APIs and their underlying functionalities.

If you face any issues, please don't forget to report this to Spark user mailing list at user@spark.apache.org. Before doing so, make sure that you have subscribed to it. In the next chapter, you will see how to test and debug Spark applications.

The rest of the chapter is locked

A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.

Unlock this book and the full library FREE for 7 days

Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of

Start free trial

Renews at €18.99/month. Cancel anytime

Authors (2)

Karim

Md. Rezaul Karim is a researcher, author, and data science enthusiast with a strong computer science background, coupled with 10 years of research and development experience in machine learning, deep learning, and data mining algorithms to solve emerging bioinformatics research problems by making them explainable. He is passionate about applied machine learning, knowledge graphs, and explainable artificial intelligence (XAI). Currently, he is working as a research scientist at Fraunhofer FIT, Germany. He is also a PhD candidate at RWTH Aachen University, Germany. Before joining FIT, he worked as a researcher at the Insight Centre for Data Analytics, Ireland. Previously, he worked as a lead software engineer at Samsung Electronics, Korea.

See other products by Karim

Sridhar Alla

Sridhar?Alla?is the co-founder and CTO of Blue Whale Consulting and is expert at helping companies (big and small) define their vision for systems and capabilities that will allow them to establish a strategic execution plan to deal with the ever-growing data collected to support analytics and product teams. He has very experienced at dealing with all aspects of data collection, security, governance, and processing as part of end-to-end big data analytics and machine learning initiatives (including predictive modeling, deep learning, and ML automation). Sridhar?is a published book author and an avid presenter at numerous conferences, including Strata, Hadoop World, and Spark Summit.? He also has several patents filed with the US PTO on large-scale computing and distributed systems.? He has over 18 years' experience writing code in Scala, Java, C, C++, Python, R, and Go, and has extensive hands-on knowledge of Spark, Flink, TensorFlow, Keras, Hadoop, Cassandra, HBase, MongoDB, Riak, Redis, Zeppelin, Mesos, Docker, Kafka, ElasticSearch, Solr, H2O, machine learning, text analytics, distributed computing, and high-performance computing. Sridhar lives with his wife and daughter in New Jersey and in his spare time loves blogging and coaching organizations on next-generation advancements in technology and their alignment with business goals.

See other products by Sridhar Alla