Running the minimum temperature example and modifying it for maximums
Let's see this filter in action and find out the minimum temperature observed for each weather station in the year 1800. Go to the download package for this book and download two things: the min-temperatures
Python script and the 1800.csv
data file, which contains our weather information. Go ahead and download these now. When you're done, place them into your C:SparkCourse
folder or wherever you're storing all the stuff for this course:
When you're ready, go ahead and double-click on min-temperatures.py
and open that up in your editor. I think it makes a little bit more sense once you see this all together. Feel free to take some time to wrap your head around it and figure out what's going on here and then I'll walk you through it.
Examining the min-temperatures script
We start off with the usual boilerplate stuff, importing what we need from pyspark
and setting up a SparkContext
object that we're going to call MinTemperatures...