Search icon Close icon
Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases now! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.

Change country

Modal Close icon
Country selected Country selected
country flag United States
Country selected
country flag United Kingdom
Country selected
country flag India
Country selected
country flag Germany
Country selected
country flag France
Country selected
country flag Canada
Country selected
country flag Russia
Country selected
country flag Spain
Country selected
country flag Brazil
Country selected
country flag Australia
Country selected
country flag Argentina
Country selected
country flag Austria
Country selected
country flag Belgium
Country selected
country flag Bulgaria
Country selected
country flag Chile
Country selected
country flag Colombia
Country selected
country flag Cyprus
Country selected
country flag Czechia
Country selected
country flag Denmark
Country selected
country flag Ecuador
Country selected
country flag Egypt
Country selected
country flag Estonia
Country selected
country flag Finland
Country selected
country flag Greece
Country selected
country flag Hungary
Country selected
country flag Indonesia
Country selected
country flag Ireland
Country selected
country flag Italy
Country selected
country flag Japan
Country selected
country flag Latvia
Country selected
country flag Lithuania
Country selected
country flag Luxembourg
Country selected
country flag Malaysia
Country selected
country flag Malta
Country selected
country flag Mexico
Country selected
country flag Netherlands
Country selected
country flag New Zealand
Country selected
country flag Norway
Country selected
country flag Philippines
Country selected
country flag Poland
Country selected
country flag Portugal
Country selected
country flag Romania
Country selected
country flag Singapore
Country selected
country flag Slovakia
Country selected
country flag Slovenia
Country selected
country flag South Africa
Country selected
country flag South Korea
Country selected
country flag Sweden
Country selected
country flag Switzerland
Country selected
country flag Taiwan
Country selected
country flag Thailand
Country selected
country flag Turkey
Country selected
country flag Ukraine
Country selected
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Apache Spark 2: Data Processing and Real-Time Analytics

You're reading from   Apache Spark 2: Data Processing and Real-Time Analytics Master complex big data processing, stream analytics, and machine learning with Apache Spark

Arrow left icon
Product type Course
Published in Dec 2018
Publisher Packt
ISBN-13 9781789959208
Length 616 pages
Edition 1st Edition
Languages
Tools
Concepts
Arrow right icon
Authors (7):
Arrow left icon
Sridhar Alla Sridhar Alla
Author Profile Icon Sridhar Alla
Sridhar Alla
Sridhar?Alla?is the co-founder and CTO of Blue Whale Consulting and is expert at helping companies (big and small) define their vision for systems and capabilities that will allow them to establish a strategic execution plan to deal with the ever-growing data collected to support analytics and product teams. He has very experienced at dealing with all aspects of data collection, security, governance, and processing as part of end-to-end big data analytics and machine learning initiatives (including predictive modeling, deep learning, and ML automation).nSridhar?is a published book author and an avid presenter at numerous conferences, including Strata, Hadoop World, and Spark Summit.? He also has several patents filed with the US PTO on large-scale computing and distributed systems.?nHe has over 18 years' experience writing code in Scala, Java, C, C++, Python, R, and Go, and has extensive hands-on knowledge of Spark, Flink, TensorFlow, Keras, Hadoop, Cassandra, HBase, MongoDB, Riak, Redis, Zeppelin, Mesos, Docker, Kafka, ElasticSearch, Solr, H2O, machine learning, text analytics, distributed computing, and high-performance computing.nSridhar lives with his wife and daughter in New Jersey and in his spare time loves blogging and coaching organizations on next-generation advancements in technology and their alignment with business goals.
Read more
See other products by Sridhar Alla
Romeo Kienzler Romeo Kienzler
Author Profile Icon Romeo Kienzler
Romeo Kienzler
Romeo Kienzler works as the chief data scientist in the IBM Watson IoT worldwide team, helping clients to apply advanced machine learning at scale on their IoT sensor data. He holds a Master's degree in computer science from the Swiss Federal Institute of Technology, Zurich, with a specialization in information systems, bioinformatics, and applied statistics.
Read more
See other products by Romeo Kienzler
Siamak Amirghodsi Siamak Amirghodsi
Author Profile Icon Siamak Amirghodsi
Siamak Amirghodsi
Siamak Amirghodsi (Sammy) is interested in building advanced technical teams, executive management, Spark, Hadoop, big data analytics, AI, deep learning nets, TensorFlow, cognitive models, swarm algorithms, real-time streaming systems, quantum computing, financial risk management, trading signal discovery, econometrics, long-term financial cycles, IoT, blockchain, probabilistic graphical models, cryptography, and NLP.
Read more
See other products by Siamak Amirghodsi
Broderick Hall Broderick Hall
Author Profile Icon Broderick Hall
Broderick Hall
Broderick Hall is a hands-on big data analytics expert and holds a masters degree in computer science with 20 years of experience in designing and developing complex enterprise-wide software applications with real-time and regulatory requirements at a global scale. He is a deep learning early adopter and is currently working on a large-scale cloud-based data platform with deep learning net augmentation.
Read more
See other products by Broderick Hall
Md. Rezaul Karim Md. Rezaul Karim
Author Profile Icon Md. Rezaul Karim
Md. Rezaul Karim
LinkedIn
Md. Rezaul Karim is a researcher, author, and data science enthusiast with a strong computer science background, coupled with 10 years of research and development experience in machine learning, deep learning, and data mining algorithms to solve emerging bioinformatics research problems by making them explainable. He is passionate about applied machine learning, knowledge graphs, and explainable artificial intelligence (XAI).nCurrently, he is working as a research scientist at Fraunhofer FIT, Germany. He is also a PhD candidate at RWTH Aachen University, Germany. Before joining FIT, he worked as a researcher at the Insight Centre for Data Analytics, Ireland. Previously, he worked as a lead software engineer at Samsung Electronics, Korea.
Read more
See other products by Md. Rezaul Karim
Meenakshi Rajendran Meenakshi Rajendran
Author Profile Icon Meenakshi Rajendran
Meenakshi Rajendran
Meenakshi Rajendran is experienced in the end-to-end delivery of data analytics and data science products for leading financial institutions. Meenakshi holds a master's degree in business administration and is a certified PMP with over 13 years of experience in global software delivery environments. Her areas of research and interest are Apache Spark, cloud, regulatory data governance, machine learning, Cassandra, and managing global data teams at scale.
Read more
See other products by Meenakshi Rajendran
Shuen Mei Shuen Mei
Author Profile Icon Shuen Mei
Shuen Mei
Shuen Mei is a big data analytic platforms expert with 15+ years of experience in designing, building, and executing large-scale, enterprise-distributed financial systems with mission-critical low-latency requirements. He is certified in the Apache Spark, Cloudera Big Data platform, including Developer, Admin, and HBase. He is also a certified AWS solutions architect with emphasis on peta-byte range real-time data platform systems.
Read more
See other products by Shuen Mei
+3 more Show less
Arrow right icon
View More author details
Toc

Table of Contents (23) Chapters Close

Title Page
Copyright
About Packt
Contributors
Preface
1. A First Taste and What's New in Apache Spark V2 FREE CHAPTER 2. Apache Spark Streaming 3. Structured Streaming 4. Apache Spark MLlib 5. Apache SparkML 6. Apache SystemML 7. Apache Spark GraphX 8. Spark Tuning 9. Testing and Debugging Spark 10. Practical Machine Learning with Spark Using Scala 11. Spark's Three Data Musketeers for Machine Learning - Perfect Together 12. Common Recipes for Implementing a Robust Machine Learning System 13. Recommendation Engine that Scales with Spark 14. Unsupervised Clustering with Apache Spark 2.0 15. Implementing Text Analytics with Spark 2.0 ML Library 16. Spark Streaming and Machine Learning Library 1. Other Books You May Enjoy Index

Using the Scala Breeze library to do graphics in Spark 2.0

In this recipe, we will use the functions scatter() and plot() from the Scala Breeze linear algebra library (part of) to draw a scatter plot from a two-dimensional data. Once the results are computed on the Spark cluster, either the actionable data can be used in the driver for drawing or a JPEG or GIF can be generated in the backend and pushed forward for efficiency and speed (popular with GPU-based analytical databases such as MapD)

How to do it...

  1. First, we need to download the necessary ScalaNLP library. Download the JAR from the Maven repository available at https://repo1.maven.org/maven2/org/scalanlp/breeze-viz_2.11/0.12/breeze-viz_2.11-0.12.jar.
  1. Place the JAR in the C:\spark-2.0.0-bin-hadoop2.7\examples\jars directory on a Windows machine:
  2. In macOS, please put the JAR in its correct path. For our setting examples, the path is /Users/USERNAME/spark/spark-2.0.0-bin-hadoop2.7/examples/jars/.
  3. The following is the sample screenshot...
lock icon The rest of the chapter is locked
arrow left Previous Section
Section 18 of 18
Next Section arrow right
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Sign up now
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Start free trial
Renews at €18.99/month. Cancel anytime

Authors (7)

Left arrow icon
Profile icon Romeo Kienzler
Romeo Kienzler
Romeo Kienzler works as the chief data scientist in the IBM Watson IoT worldwide team, helping clients to apply advanced machine learning at scale on their IoT sensor data. He holds a Master's degree in computer science from the Swiss Federal Institute of Technology, Zurich, with a specialization in information systems, bioinformatics, and applied statistics.
Read more
See other products by Romeo Kienzler
Profile icon Shuen Mei
Shuen Mei
Shuen Mei is a big data analytic platforms expert with 15+ years of experience in designing, building, and executing large-scale, enterprise-distributed financial systems with mission-critical low-latency requirements. He is certified in the Apache Spark, Cloudera Big Data platform, including Developer, Admin, and HBase. He is also a certified AWS solutions architect with emphasis on peta-byte range real-time data platform systems.
Read more
See other products by Shuen Mei
Profile icon Broderick Hall
Broderick Hall
Broderick Hall is a hands-on big data analytics expert and holds a masters degree in computer science with 20 years of experience in designing and developing complex enterprise-wide software applications with real-time and regulatory requirements at a global scale. He is a deep learning early adopter and is currently working on a large-scale cloud-based data platform with deep learning net augmentation.
Read more
See other products by Broderick Hall
Profile icon Meenakshi Rajendran
Meenakshi Rajendran
Meenakshi Rajendran is experienced in the end-to-end delivery of data analytics and data science products for leading financial institutions. Meenakshi holds a master's degree in business administration and is a certified PMP with over 13 years of experience in global software delivery environments. Her areas of research and interest are Apache Spark, cloud, regulatory data governance, machine learning, Cassandra, and managing global data teams at scale.
Read more
See other products by Meenakshi Rajendran
Profile icon Siamak Amirghodsi
Siamak Amirghodsi
Siamak Amirghodsi (Sammy) is interested in building advanced technical teams, executive management, Spark, Hadoop, big data analytics, AI, deep learning nets, TensorFlow, cognitive models, swarm algorithms, real-time streaming systems, quantum computing, financial risk management, trading signal discovery, econometrics, long-term financial cycles, IoT, blockchain, probabilistic graphical models, cryptography, and NLP.
Read more
See other products by Siamak Amirghodsi
Profile icon Sridhar Alla
Sridhar Alla
Sridhar?Alla?is the co-founder and CTO of Blue Whale Consulting and is expert at helping companies (big and small) define their vision for systems and capabilities that will allow them to establish a strategic execution plan to deal with the ever-growing data collected to support analytics and product teams. He has very experienced at dealing with all aspects of data collection, security, governance, and processing as part of end-to-end big data analytics and machine learning initiatives (including predictive modeling, deep learning, and ML automation). Sridhar?is a published book author and an avid presenter at numerous conferences, including Strata, Hadoop World, and Spark Summit.? He also has several patents filed with the US PTO on large-scale computing and distributed systems.? He has over 18 years' experience writing code in Scala, Java, C, C++, Python, R, and Go, and has extensive hands-on knowledge of Spark, Flink, TensorFlow, Keras, Hadoop, Cassandra, HBase, MongoDB, Riak, Redis, Zeppelin, Mesos, Docker, Kafka, ElasticSearch, Solr, H2O, machine learning, text analytics, distributed computing, and high-performance computing. Sridhar lives with his wife and daughter in New Jersey and in his spare time loves blogging and coaching organizations on next-generation advancements in technology and their alignment with business goals.
Read more
See other products by Sridhar Alla
Profile icon Md. Rezaul Karim
Md. Rezaul Karim
Md. Rezaul Karim is a researcher, author, and data science enthusiast with a strong computer science background, coupled with 10 years of research and development experience in machine learning, deep learning, and data mining algorithms to solve emerging bioinformatics research problems by making them explainable. He is passionate about applied machine learning, knowledge graphs, and explainable artificial intelligence (XAI). Currently, he is working as a research scientist at Fraunhofer FIT, Germany. He is also a PhD candidate at RWTH Aachen University, Germany. Before joining FIT, he worked as a researcher at the Insight Centre for Data Analytics, Ireland. Previously, he worked as a lead software engineer at Samsung Electronics, Korea.
Read more
See other products by Md. Rezaul Karim
Right arrow icon

Personalised recommendations for you

Based on your interests and search pattern
Left arrow icon
Modern Computer Vision with PyTorch
Modern Computer Vision with PyTorch
Read more
This book provides a hands-on approach to solving over 30 prominent real-world computer vision problems using PyTorch 2.x on actual datasets. Here you'll learn to build a neural network from scratch and optimize hyperparameters, perform image classification, multi-object detection, segmentation, and more. You'll also explore facial expression manipulation and combining CV with NLP and RL techniques, build generative AI applications, and take your model to production on AWS. By the end of this book, you'll master modern NN architectures and confidently solve real-world CV problems.
Read more
Jun 2024 24h 52m
Modern Computer Vision with PyTorch
Modern Computer Vision with PyTorch
Read more
This book provides a hands-on approach to solving over 30 prominent real-world computer vision problems using PyTorch 2.x on actual datasets. Here you'll learn to build a neural network from scratch and optimize hyperparameters, perform image classification, multi-object detection, segmentation, and more. You'll also explore facial expression manipulation and combining CV with NLP and RL techniques, build generative AI applications, and take your model to production on AWS. By the end of this book, you'll master modern NN architectures and confidently solve real-world CV problems.
Read more
Jun 2024 24h 52m
Modern Computer Vision with PyTorch
Modern Computer Vision with PyTorch
Read more
This book provides a hands-on approach to solving over 30 prominent real-world computer vision problems using PyTorch 2.x on actual datasets. Here you'll learn to build a neural network from scratch and optimize hyperparameters, perform image classification, multi-object detection, segmentation, and more. You'll also explore facial expression manipulation and combining CV with NLP and RL techniques, build generative AI applications, and take your model to production on AWS. By the end of this book, you'll master modern NN architectures and confidently solve real-world CV problems.
Read more
Jun 2024 24h 52m
Modern Computer Vision with PyTorch
Modern Computer Vision with PyTorch
Read more
This book provides a hands-on approach to solving over 30 prominent real-world computer vision problems using PyTorch 2.x on actual datasets. Here you'll learn to build a neural network from scratch and optimize hyperparameters, perform image classification, multi-object detection, segmentation, and more. You'll also explore facial expression manipulation and combining CV with NLP and RL techniques, build generative AI applications, and take your model to production on AWS. By the end of this book, you'll master modern NN architectures and confidently solve real-world CV problems.
Read more
Jun 2024 24h 52m
Modern Computer Vision with PyTorch
Modern Computer Vision with PyTorch
Read more
This book provides a hands-on approach to solving over 30 prominent real-world computer vision problems using PyTorch 2.x on actual datasets. Here you'll learn to build a neural network from scratch and optimize hyperparameters, perform image classification, multi-object detection, segmentation, and more. You'll also explore facial expression manipulation and combining CV with NLP and RL techniques, build generative AI applications, and take your model to production on AWS. By the end of this book, you'll master modern NN architectures and confidently solve real-world CV problems.
Read more
Jun 2024 24h 52m
Modern Computer Vision with PyTorch
Modern Computer Vision with PyTorch
Read more
This book provides a hands-on approach to solving over 30 prominent real-world computer vision problems using PyTorch 2.x on actual datasets. Here you'll learn to build a neural network from scratch and optimize hyperparameters, perform image classification, multi-object detection, segmentation, and more. You'll also explore facial expression manipulation and combining CV with NLP and RL techniques, build generative AI applications, and take your model to production on AWS. By the end of this book, you'll master modern NN architectures and confidently solve real-world CV problems.
Read more
Jun 2024 24h 52m
Modern Computer Vision with PyTorch
Modern Computer Vision with PyTorch
Read more
This book provides a hands-on approach to solving over 30 prominent real-world computer vision problems using PyTorch 2.x on actual datasets. Here you'll learn to build a neural network from scratch and optimize hyperparameters, perform image classification, multi-object detection, segmentation, and more. You'll also explore facial expression manipulation and combining CV with NLP and RL techniques, build generative AI applications, and take your model to production on AWS. By the end of this book, you'll master modern NN architectures and confidently solve real-world CV problems.
Read more
Jun 2024 24h 52m
Modern Computer Vision with PyTorch
Modern Computer Vision with PyTorch
Read more
This book provides a hands-on approach to solving over 30 prominent real-world computer vision problems using PyTorch 2.x on actual datasets. Here you'll learn to build a neural network from scratch and optimize hyperparameters, perform image classification, multi-object detection, segmentation, and more. You'll also explore facial expression manipulation and combining CV with NLP and RL techniques, build generative AI applications, and take your model to production on AWS. By the end of this book, you'll master modern NN architectures and confidently solve real-world CV problems.
Read more
Jun 2024 24h 52m
Modern Computer Vision with PyTorch
Modern Computer Vision with PyTorch
Read more
This book provides a hands-on approach to solving over 30 prominent real-world computer vision problems using PyTorch 2.x on actual datasets. Here you'll learn to build a neural network from scratch and optimize hyperparameters, perform image classification, multi-object detection, segmentation, and more. You'll also explore facial expression manipulation and combining CV with NLP and RL techniques, build generative AI applications, and take your model to production on AWS. By the end of this book, you'll master modern NN architectures and confidently solve real-world CV problems.
Read more
Jun 2024 24h 52m
Modern Computer Vision with PyTorch
Modern Computer Vision with PyTorch
Read more
This book provides a hands-on approach to solving over 30 prominent real-world computer vision problems using PyTorch 2.x on actual datasets. Here you'll learn to build a neural network from scratch and optimize hyperparameters, perform image classification, multi-object detection, segmentation, and more. You'll also explore facial expression manipulation and combining CV with NLP and RL techniques, build generative AI applications, and take your model to production on AWS. By the end of this book, you'll master modern NN architectures and confidently solve real-world CV problems.
Read more
Jun 2024 24h 52m
Modern Computer Vision with PyTorch
Modern Computer Vision with PyTorch
Read more
This book provides a hands-on approach to solving over 30 prominent real-world computer vision problems using PyTorch 2.x on actual datasets. Here you'll learn to build a neural network from scratch and optimize hyperparameters, perform image classification, multi-object detection, segmentation, and more. You'll also explore facial expression manipulation and combining CV with NLP and RL techniques, build generative AI applications, and take your model to production on AWS. By the end of this book, you'll master modern NN architectures and confidently solve real-world CV problems.
Read more
Jun 2024 24h 52m
Modern Computer Vision with PyTorch
Modern Computer Vision with PyTorch
Read more
This book provides a hands-on approach to solving over 30 prominent real-world computer vision problems using PyTorch 2.x on actual datasets. Here you'll learn to build a neural network from scratch and optimize hyperparameters, perform image classification, multi-object detection, segmentation, and more. You'll also explore facial expression manipulation and combining CV with NLP and RL techniques, build generative AI applications, and take your model to production on AWS. By the end of this book, you'll master modern NN architectures and confidently solve real-world CV problems.
Read more
Jun 2024 24h 52m
Right arrow icon