Conventions

In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "Both spark-shell and PySpark are available in the bin directory of SPARK_HOME, that is, SPARK_HOME/bin"

A block of code is set as follows:

from pyspark  
import SparkContext

stocks = "hdfs://namenode:9000/stocks.txt"  
 
sc = SparkContext("<master URI>", "ApplicationName")
data = sc.textFile(stocks)

totalLines = data.count() 
print("Total Lines are: %i" % (totalLines))

Any command-line input or output is written as follows:

     $SPARK_HOME/bin/spark-shell --master <master type> 
     Spark context available as sc.

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.