Search icon CANCEL
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Scala Data Analysis Cookbook (new)

You're reading from   Scala Data Analysis Cookbook (new) Navigate the world of data analysis, visualization, and machine learning with over 100 hands-on Scala recipes

Arrow left icon
Product type Paperback
Published in Oct 2015
Publisher
ISBN-13 9781784396749
Length 254 pages
Edition 1st Edition
Languages
Arrow right icon
Author (1):
Arrow left icon
Arun Manivannan Arun Manivannan
Author Profile Icon Arun Manivannan
Arun Manivannan
Arrow right icon
View More author details
Toc

Table of Contents (9) Chapters Close

Preface 1. Getting Started with Breeze 2. Getting Started with Apache Spark DataFrames FREE CHAPTER 3. Loading and Preparing Data – DataFrame 4. Data Visualization 5. Learning from Data 6. Scaling Up 7. Going Further Index

Creating a DataFrame from Scala case classes

In this recipe, we'll see how to create a new DataFrame from Scala case classes.

How to do it...

  1. We create a new entity called Employee with the id and name fields, like this:
    case class Employee(id:Int, name:String)
    

    Similar to the previous recipe, we create SparkContext and SQLContext.

    val conf = new SparkConf().setAppName("colRowDataFrame").setMaster("local[2]")
    
    //Initialize Spark context with Spark configuration.  This is the core entry point to do anything with Spark
    val sc = new SparkContext(conf)
    
    //The easiest way to query data in Spark is to use SQL queries.
    val sqlContext=new SQLContext(sc)
    
  2. We can source these employee objects from a variety of sources, such as an RDBMS data source, but for the sake of this example...
You have been reading a chapter from
Scala Data Analysis Cookbook (new)
Published in: Oct 2015
Publisher:
ISBN-13: 9781784396749
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $19.99/month. Cancel anytime