Search icon CANCEL
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Scala Data Analysis Cookbook (new)

You're reading from   Scala Data Analysis Cookbook (new) Navigate the world of data analysis, visualization, and machine learning with over 100 hands-on Scala recipes

Arrow left icon
Product type Paperback
Published in Oct 2015
Publisher
ISBN-13 9781784396749
Length 254 pages
Edition 1st Edition
Languages
Arrow right icon
Author (1):
Arrow left icon
Arun Manivannan Arun Manivannan
Author Profile Icon Arun Manivannan
Arun Manivannan
Arrow right icon
View More author details
Toc

Table of Contents (9) Chapters Close

Preface 1. Getting Started with Breeze 2. Getting Started with Apache Spark DataFrames FREE CHAPTER 3. Loading and Preparing Data – DataFrame 4. Data Visualization 5. Learning from Data 6. Scaling Up 7. Going Further Index

Vectors and matrices with randomly distributed values

The breeze.stats.distributions package supplements the random number generator that is built into Scala. Scala's default generator just provides the ability to get the random values one by one using the "next" methods. Random number generators in Breeze provide the ability to build vectors and matrices out of these generators. In this recipe, we'll briefly see three of the most common distributions of random numbers.

In this recipe, we will cover at the following sub-recipes:

  • Creating vectors with uniformly distributed random values
  • Creating vectors with normally distributed random values
  • Creating vectors with random values that have a Poisson distribution
  • Creating a matrix with uniformly random values
  • Creating a matrix with normally distributed random values
  • Creating a matrix with random values that has a Poisson distribution

How it works...

Before we delve into how to create the vectors and matrices out of random numbers, let's create instances of the most common random number distribution. All these generators are under the breeze.stats.distributions package:

//Uniform distribution with low being 0 and high being 10
val uniformDist=Uniform(0,10)

//Gaussian distribution with mean being 5 and Standard deviation being 1
val gaussianDist=Gaussian(5,1)

//Poission distribution with mean being 5
val poissonDist=Poisson(5)

We could actually directly sample from these generators. Given any distribution we created previously, we could sample either a single value or a sequence of values:

//Samples a single value
println (uniformDist.sample())
//eg. 9.151191360491392

//Returns a sample vector of size that is passed in as parameter
println (uniformDist.sample(2))
//eg. Vector(6.001980062275654, 6.210874664967401)

Creating vectors with uniformly distributed random values

With no generator parameter, the DenseVector.rand method accepts a parameter for the length of the vector to be returned. The result is a vector (of length 10) with uniformly distributed values between 0 and 1:

val uniformWithoutSize=DenseVector.rand(10)
println ("uniformWithoutSize \n"+ uniformWithoutSize)
//DenseVector(0.1235038023750481, 0.3120595941786264, 0.3575638744660876, 0.5640844223813524, 0.5336149399548831, 0.1338053814330793, 0.9099684427908603, 0.38690724148973166, 0.22561993631651522, 0.45120359622713657)

The DenseVector.rand method optionally accepts a distribution object and generates random values using that input distribution. The following line generates a vector of 10 uniformly distributed random values that are within the range 0 and 10:

val uniformDist=Uniform(0,10)

val uniformVectInRange=DenseVector.rand(10, uniformDist)
println ("uniformVectInRange \n"+uniformVectInRange)
//DenseVector(1.5545833905907314, 6.172564377264846, 8.45578509265587, 7.683763574965107, 8.018688137742062, 4.5876187984930406, 3.274758584944064, 2.3873947264259954, 2.139988841403757, 8.314112884416943)

Creating vectors with normally distributed random values

In the place of the uniformDist generator, we could also pass the previously created Gaussian generator, which is configured to yield a distribution that has a mean of 5 and standard deviation of 1:

val gaussianVector=DenseVector.rand(10, gaussianDist)
println ("gaussianVector \n"+gaussianVector)
//DenseVector(4.235655596913547, 5.535011377545014, 6.201428236839494, 6.046289604188366, 4.319709374229152,
4.2379652913447154, 2.957868021601233, 3.96371080427211, 4.351274306757224, 5.445022658876723)

Creating vectors with random values that have a Poisson distribution

Similarly, by passing the previously created Poisson random number generator, a vector of values that has a mean of 5 could be generated:

val poissonVector=DenseVector.rand(10, poissonDist)
println ("poissonVector \n"+poissonVector)
//DenseVector(5, 5, 7, 11, 7, 6, 6, 6, 6, 6)

We saw how easy it is to create a vector of random values. Now, let's proceed to create a matrix of random values. Similar to DenseVector.rand to generate vectors with random values, we'll use the DenseMatrix.rand function to generate a matrix of random values.

Creating a matrix with uniformly random values

The DenseMatrix.rand defaults to the uniform distribution and generates a matrix of random values given the row and the column parameter. However, if we would like to have a distribution within a range, then as in vectors, we could use the optional parameter:.

//Uniform distribution, Creates a 3 * 3 Matrix with random values from 0 to 1
val uniformMat=DenseMatrix.rand(3, 3)
println ("uniformMat \n"+uniformMat)

0.4492155777289115  0.9098840386699856    0.8203022252988292
0.0888975848853315  0.009677790736892788  0.6058885905934237
0.6201415814136939  0.7017492438727635    0.08404147915159443

//Creates a 3 * 3 Matrix with uniformly distributed random values with low being 0 and high being 10
val uniformMatrixInRange=DenseMatrix.rand(3,3, uniformDist)
println ("uniformMatrixInRange \n"+uniformMatrixInRange)

7.592014659345548  8.164652560340933    6.966445294464401
8.35949395084735   3.442654641743763    3.6761640240938442
9.42626645215854   0.23658921372298636  7.327120138868571

Creating a matrix with normally distributed random values

Just as in vectors, in place of the uniformDist generator, we could also pass the previously created Gaussian generator to the rand function to generate a matrix of random values that has a mean of 5 and standard deviation of 1:

//Creates a 3 * 3 Matrix with normally distributed random values with mean being 5 and Standard deviation being 1
val gaussianMatrix=DenseMatrix.rand(3, 3,gaussianDist)
println ("gaussianMatrix \n"+gaussianMatrix)

5.724540885605018   5.647051873430568  5.337906135107098
6.2228893721489875  4.799561665187845  5.12469779489833
5.136960834730864   5.176410360757703  5.262707072950913

Creating a matrix with random values that has a Poisson distribution

Similarly, by passing the previously created Poisson random number generator, a matrix of random values that has a mean of 5 could be generated:

//Creates a 3 * 3 Matrix with Poisson distribution with mean being 5
val poissonMatrix=DenseMatrix.rand(3, 3,poissonDist)
println ("poissonMatrix \n"+poissonMatrix)
4  11  3
6  6   5
6  4   2
lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at €18.99/month. Cancel anytime