Search icon CANCEL
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Apache Spark Deep Learning Cookbook

You're reading from   Apache Spark Deep Learning Cookbook Over 80 best practice recipes for the distributed training and deployment of neural networks using Keras and TensorFlow

Arrow left icon
Product type Paperback
Published in Jul 2018
Publisher Packt
ISBN-13 9781788474221
Length 474 pages
Edition 1st Edition
Languages
Arrow right icon
Authors (2):
Arrow left icon
Ahmed Sherif Ahmed Sherif
Author Profile Icon Ahmed Sherif
Ahmed Sherif
Amrith Ravindra Amrith Ravindra
Author Profile Icon Amrith Ravindra
Amrith Ravindra
Arrow right icon
View More author details
Toc

Table of Contents (15) Chapters Close

Preface 1. Setting Up Spark for Deep Learning Development FREE CHAPTER 2. Creating a Neural Network in Spark 3. Pain Points of Convolutional Neural Networks 4. Pain Points of Recurrent Neural Networks 5. Predicting Fire Department Calls with Spark ML 6. Using LSTMs in Generative Networks 7. Natural Language Processing with TF-IDF 8. Real Estate Value Prediction Using XGBoost 9. Predicting Apple Stock Market Cost with LSTM 10. Face Recognition Using Deep Convolutional Networks 11. Creating and Visualizing Word Vectors Using Word2Vec 12. Creating a Movie Recommendation Engine with Keras 13. Image Classification with TensorFlow on Spark 14. Other Books You May Enjoy

Visualizing further 


This section will describe how to squash the dimensionality of all the trained words and put it all into one giant matrix for visualization purposes. Since each word is a 300-dimensional vector, it needs to be brought down to a lower dimension for us to visualize it in a 2D space.

Getting ready

Once the model is saved and checkpointed after training, begin by loading it into memory, as you did in the previous section. The libraries and modules that will be utilized in this section are: 

  • tSNE
  • pandas
  • Seaborn
  • numpy

How to do it...

The steps are as follows:

  1. Squash the dimensionality of the 300-dimensional word vectors by using the following command:
 tsne = sklearn.manifold.TSNE(n_components=2, random_state=0)
  1. Put all the word vectors into one giant matrix (named all_word_vectors_matrix), and view it using the following commands:
 all_word_vectors_matrix = got2vec.wv.syn0
 print (all_word_vectors_matrix)
  1. Use the tsne technique to fit all the learned representations into a two- dimensional...
lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at €18.99/month. Cancel anytime