What do you get with Print?

Instant access to your digital eBook copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Download this book in EPUB and PDF formats

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

NumPy Beginner's Guide

Chapter 1. NumPy Quick Start

Let's get started. We will install NumPy and related software on different operating systems and have a look at some simple code that uses NumPy. The IPython interactive shell is introduced briefly. As mentioned in the Preface, SciPy is closely related to NumPy, so you will see the SciPy name appearing here and there. At the end of this chapter, you will find pointers on how to find additional information online if you get stuck or are uncertain about the best way to solve problems.

In this chapter, we shall:

Install Python, SciPy, Matplotlib, IPython, and NumPy on Windows, Linux, and Macintosh
Write simple NumPy code
Get to know IPython
Browse online documentation and resources

Time for action – installing NumPy, Matplotlib, SciPy, and IPython on Linux

Most Linux distributions have NumPy packages. We will go through the necessary steps for some of the popular Linux distros:

Run the following instructions from the command line for installing NumPy and Red Hat:
```
yum install python-numpy
```
To install NumPy on Mandriva, run the following command-line instruction:
```
urpmi python-numpy
```
To install NumPy on Gentoo run the following command-line instruction:
```
sudo emerge numpy
```
To install NumPy on Debian or Ubuntu, we need to type the following :
```
sudo apt-get install python-numpy
```

The following table gives an overview of the Linux distributions and corresponding package names for NumPy, SciPy, Matplotlib, and IPython.

Linux distribution	NumPy	SciPy	Matplotlib	IPython
Arch Linux	`python-numpy`	`python-scipy`	`python-matplotlib`	`ipython`
Debian	`python-numpy`	`python-scipy`	`python-matplotlib`	`ipython`
Fedora	`numpy`	`python-scipy`	`python-matplotlib`	`ipython`
Gentoo	`dev-python/numpy`	`scipy`	`matplotlib`	`ipython`
OpenSUSE	`python-numpy, python-numpy-devel`	`python-scipy`	`python-matplotlib`	`ipython`
Slackware	`numpy`	`scipy`	`matplotlib`	`ipython`

What just happened?

We installed NumPy, SciPy, Matplotlib, and IPython on various Linux distributions.

Time for action – adding vectors

Imagine that we want to add two vectors called a and b. Vector is used here in the mathematical sense meaning a one-dimensional array. We will learn in Chapter 5, Working with Matrices and ufuncs, about specialized NumPy arrays which represent matrices. The vector a holds the squares of integers 0 to n, for instance, if n is equal to 3, then a is equal to 0, 1, or 4. The vector b holds the cubes of integers 0 to n, so if n is equal to 3, then the vector b is equal to 0, 1, or 8. How would you do that using plain Python? After we come up with a solution, we will compare it with the NumPy equivalent.

The following function solves the vector addition problem using pure Python without NumPy:

def pythonsum(n):
   a = range(n)
   b = range(n)
   c = []

   for i in range(len(a)):
       a[i] = i ** 2
       b[i] = i ** 3
       c.append(a[i] + b[i])

   return c

The following is a function that achieves the same with NumPy:

def numpysum(n):
  a = numpy.arange(n) ** 2
  b = numpy.arange(n) ** 3
  c = a + b
  return c

Notice that numpysum() does not need a for loop. Also, we used the arange function from NumPy that creates a NumPy array for us with integers 0 to n. The arange function was imported; that is why it is prefixed with numpy.

Now comes the fun part. Remember that it is mentioned in the Preface that NumPy is faster when it comes to array operations. How much faster is Numpy, though? The following program will show us by measuring the elapsed time in microseconds, for the numpysum and pythonsum functions. It also prints the last two elements of the vector sum. Let's check that we get the same answers by using Python and NumPy:

#!/usr/bin/env/python

import sys
from datetime import datetime
import numpy as np

"""
 Chapter 1 of NumPy Beginners Guide.
 This program demonstrates vector addition the Python way.
 Run from the command line as follows
     
  python vectorsum.py n
 
 where n is an integer that specifies the size of the vectors.

 The first vector to be added contains the squares of 0 up to n. 
 The second vector contains the cubes of 0 up to n.
 The program prints the last 2 elements of the sum and the elapsed time.
"""

def numpysum(n):
   a = np.arange(n) ** 2
   b = np.arange(n) ** 3
   c = a + b

   return c

def pythonsum(n):
   a = range(n)
   b = range(n)
   c = []

   for i in range(len(a)):
       a[i] = i ** 2
       b[i] = i ** 3
       c.append(a[i] + b[i])

   return c
   
size = int(sys.argv[1])

start = datetime.now()
c = pythonsum(size)
delta = datetime.now() - start
print "The last 2 elements of the sum", c[-2:]
print "PythonSum elapsed time in microseconds", delta.microseconds

start = datetime.now()
c = numpysum(size)
delta = datetime.now() - start
print "The last 2 elements of the sum", c[-2:]
print "NumPySum elapsed time in microseconds", delta.microseconds

The output of the program for 1000, 2000, and 3000 vector elements is as follows:

$ python vectorsum.py 1000
The last 2 elements of the sum [995007996, 998001000]
PythonSum elapsed time in microseconds 707
The last 2 elements of the sum [995007996 998001000]
NumPySum elapsed time in microseconds 171

$ python vectorsum.py 2000
The last 2 elements of the sum [7980015996, 7992002000]
PythonSum elapsed time in microseconds 1420
The last 2 elements of the sum [7980015996 7992002000]
NumPySum elapsed time in microseconds 168

$ python vectorsum.py 4000
The last 2 elements of the sum [63920031996, 63968004000]
PythonSum elapsed time in microseconds 2829
The last 2 elements of the sum [63920031996 63968004000]
NumPySum elapsed time in microseconds 274

Tip

You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.

What just happened?

Clearly, NumPy is much faster than the equivalent normal Python code. One thing is certain; we get the same results whether we are using NumPy or not. However, the result that is printed differs in representation. Notice that the result from the numpysum function does not have any commas. How come? Obviously we are not dealing with a Python list but with a NumPy array. It was mentioned in the Preface that NumPy arrays are specialized data structures for numerical data. We will learn more about NumPy arrays in the next chapter.

Pop quiz Functioning of the arange function

Q1. What does arange(5) do?

Creates a Python list of 5 elements with values 1 to 5.
Creates a Python list of 5 elements with values 0 to 4.
Creates a NumPy array with values 1 to 5.
Creates a NumPy array with values 0 to 4.
None of the above.

Have a go hero – continue the analysis

The program we used here to compare the speed of NumPy and regular Python is not very scientific. We should at least repeat each measurement a couple of times. It would be nice to be able to calculate some statistics such as average times, and so on. Also, you might want to show plots of the measurements to friends and colleagues.

Tip

Hints to help can be found in the online documentation and resources listed at the end of this chapter. NumPy has, by the way, statistical functions that can calculate averages for you. I recommend using Matplotlib to produce plots. Chapter 9, Plotting with Matplotlib, gives a quick overview of Matplotlib.

IPython—an interactive shell

Scientists and engineers are used to experimenting. IPython was created by scientists with experimentation in mind. The interactive environment that IPython provides is viewed by many as a direct answer to Matlab, Mathematica, and Maple. You can find more information, including installation instructions, at: http://ipython.org/.

IPython is free, open source, and available for Linux, Unix, Mac OS X, and Windows. The IPython authors only request that you cite IPython in scientific work where IPython was used. Here is the list of basic IPython features:

Tab completion
History mechanism
Inline editing
Ability to call external Python scripts with %run
Access to system commands
Pylab switch
Access to Python debugger and profiler

The Pylab switch imports all the Scipy, NumPy, and Matplotlib packages. Without this switch, we would have to import every package we need, ourselves.

All we need to do is enter the following instruction on the command line:

$ ipython --pylab
Python 2.7.2 (default, Jun 20 2012, 16:23:33) 
Type "copyright", "credits" or "license" for more information.

IPython 0.14.dev -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

Welcome to pylab, a matplotlib-based Python environment [backend: MacOSX].
For more information, type 'help(pylab)'.
In [1]: quit()

The quit() function or Ctrl + D quits the IPython shell. We might want to be able to go back to our experiments. In IPython, it is easy to save a session for later:

In [1]: %logstart
Activating auto-logging. Current session state plus future input saved.
Filename       : ipython_log.py
Mode           : rotate
Output logging : False
Raw input log  : False
Timestamping   : False
State          : active

Let's say we have the vector addition program that we made in the current directory. We can run the script as follows:

In [1]: ls
README        vectorsum.py
In [2]: %run -i vectorsum.py 1000

As you probably remember, 1000 specifies the number of elements in a vector. The -d switch of %run starts an ipdb debugger and on typing c, the script is started. n steps through the code. Typing quit at the ipdb prompt exits the debugger.

In [2]: %run -d vectorsum.py 1000
*** Blank or comment
*** Blank or comment
Breakpoint 1 at: /Users/…/vectorsum.py:3

><string>(1)<module>()
ipdb> c
> /Users/…/vectorsum.py(3)<module>()
      2
1---> 3 import sys
      4 from datetime import datetime
ipdb> n
>
/Users/…/vectorsum.py(4)<module>()
1     3 import sys
----> 4 from datetime import datetime
      5 import numpy
ipdb> n
> /Users/…/vectorsum.py(5)<module>()
      4 from datetime import datetime
----> 5 import numpy
      6 
ipdb> quit

We can also profile our script by passing the -p option to %run.

In [4]: %run -p vectorsum.py 1000
         1058 function calls (1054 primitive calls) in 0.002 CPU seconds
   Ordered by: internal time
ncallstottimepercallcumtimepercallfilename:lineno(function)
1 0.001    0.001   0.001    0.001 vectorsum.py:28(pythonsum)
1 0.001    0.001   0.002    0.002 {execfile}
1000 0.000    0.0000.0000.000 {method 'append' of 'list' objects}
1 0.000    0.000    0.002    0.002 vectorsum.py:3(<module>)
1 0.000    0.0000.0000.000 vectorsum.py:21(numpysum)
3    0.000    0.0000.0000.000 {range}
1    0.000    0.0000.0000.000 arrayprint.py:175(_array2string)
3/1    0.000    0.0000.0000.000 arrayprint.py:246(array2string)
2    0.000    0.0000.0000.000 {method 'reduce' of 'numpy.ufunc' objects}
4    0.000    0.0000.0000.000 {built-in method now}
2    0.000    0.0000.0000.000 arrayprint.py:486(_formatInteger)
2    0.000    0.0000.0000.000 {numpy.core.multiarray.arange}
1    0.000    0.0000.0000.000 arrayprint.py:320(_formatArray)
3/1    0.000    0.0000.0000.000 numeric.py:1390(array_str)
1    0.000    0.0000.0000.000 numeric.py:216(asarray)
2    0.000    0.0000.0000.000 arrayprint.py:312(_extendLine)
1    0.000    0.0000.0000.000 fromnumeric.py:1043(ravel)
2    0.000    0.0000.0000.000 arrayprint.py:208(<lambda>)
1    0.000    0.000    0.002    0.002<string>:1(<module>)
11    0.000    0.0000.0000.000 {len}
2    0.000    0.0000.0000.000 {isinstance}
1    0.000    0.0000.0000.000 {reduce}
1    0.000    0.0000.0000.000 {method 'ravel' of 'numpy.ndarray' objects}
4    0.000    0.0000.0000.000 {method 'rstrip' of 'str' objects}
3    0.000    0.0000.0000.000 {issubclass}
2    0.000    0.0000.0000.000 {method 'item' of 'numpy.ndarray' objects}
1    0.000    0.0000.0000.000 {max}
1    0.000    0.0000.0000.000 {method 'disable' of '_lsprof.Profiler' objects}

This gives us a bit more insight into the workings of our program. In addition, we can now identify performance bottlenecks. The %hist command shows the commands history.

In [2]: a=2+2
In [3]: a
Out[3]: 4
In [4]: %hist
1: _ip.magic("hist ")
2: a=2+2
3: a

I hope you agree that IPython is a really useful tool!

Key benefits

Perform high performance calculations with clean and efficient NumPy code.

Analyze large data sets with statistical functions

Execute complex linear algebra and mathematical computations

Description

NumPy is an extension to, and the fundamental package for scientific computing with Python. In today's world of science and technology, it is all about speed and flexibility. When it comes to scientific computing, NumPy is on the top of the list. NumPy Beginner's Guide will teach you about NumPy, a leading scientific computing library. NumPy replaces a lot of the functionality of Matlab and Mathematica, but in contrast to those products, is free and open source. Write readable, efficient, and fast code, which is as close to the language of mathematics as is currently possible with the cutting edge open source NumPy software library. Learn all the ins and outs of NumPy that requires you to know basic Python only. Save thousands of dollars on expensive software, while keeping all the flexibility and power of your favourite programming language.You will learn about installing and using NumPy and related concepts. At the end of the book we will explore some related scientific computing projects. This book will give you a solid foundation in NumPy arrays and universal functions. Through examples, you will also learn about plotting with Matplotlib and the related SciPy project. NumPy Beginner's Guide will help you be productive with NumPy and have you writing clean and fast code in no time at all.

What you will learn

Install NumPy

NumPy arrays

Universal functions

NumPy matrices

NumPy modules

Plot with Matplotlib

Test NumPy code

Relation to SciPy

What do you get with Print?

Instant access to your digital eBook copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Download this book in EPUB and PDF formats

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

Frequently bought together

€37.99

Building Machine Learning Systems with Python

€41.99

€37.99

Total € 117.97

Filter reviews by

All

Amazon verified reviews

nipun Jun 13, 2013

I had written a review for this book on my blog here: http://nipunbatra.wordpress.com/2013/06/11/book-review-numpy-beginners-guide-second-edition/ I would recommend this book highly. It is very good for newcomers. For Python oldies it might be a little slow. NumPy Beginner's Guide - Second Edition

Amazon Verified review

Tamir Lousky Jun 15, 2013

The book is very well written, and tries (quite successfully) to address this specialized, complicated subject in a light, simplified manner without losing depth and oversimplifying. The book includes many down to earth examples and sample code, and many interesting exercises that help implement the material.It starts with a nice, gradual introduction to NumPy, its functions, data structures and uses (and of course, how to install it). Then, the book addresses how to handle, analyze and manipulate larger datasets with examples from stock market data. Around half way, the book focuses on matrices and linear systems. The last few subjects cover advanced topics which I didn't delve into yet.All in all, a very good book, fun to read and useful. Recommended to anyone interested in manipulating arrays, matrices and large datasets in python efficiently.For further information, please see the mini-review on my blog:http://bioblog3d.wordpress.com/2013/06/15/numpy-beginners-guide-review/

ReaddyEddy Jul 03, 2013

Who is it for:This is an excellent book for the Python programmer who wants to extend their knowledge into mathematical programming and those with a mathematical or engineering background who want to leverage open source alternatives to commercial tools by using the NumPy with Python.Installation and other packages:Installation on the main platforms and the relationship between NumPy and SciPy, and using the library with Matplotlib and other Python modules is well covered.What's covered:The coverage is goes from creating vectors and multi-dimensional matrices through calculating Eigenvectors, the FFT, complex numbers, polynomial fitting and many others.Example Code:The examples I've followed are well thought through and illustrate the use the relevant parts of the NumPy API required in a clear and concise manner. An increasing amount of mathematical background is needed and for a beginner the book should be read in conjunction with a text book covering the relevant material.Recommendation:I would heartily recommend the book both as tool for learning and as a working reference for the NumPy library and wish it had been available when I was going up the learning curve.

Paul A. Courchene Jul 02, 2013

I wish to comment on the excellent book by Ivan Idris. The text entitled NumPy Beginner's Guide, Second Edition is an outstanding book for a broad range of computer enthusiasts. While not all computer nerds are necessarily interested in Programming per se', in light of the growth and momentum of digital media, it is now a fact of life that many fields of employment require some basic introduction to computer programming. Whether you are a business or financial analyst, or perhaps a serious computer wanabee, or even a College student, it is now time to take the first step. With the momentum seen in the computer language of Python, many people who would otherwise not be inclined to learn this skill, never the less are posing some interest and saying "I wonder if I could do that?" This book will show you how. Since this book covers programming with Python from A to Z, and provides an easy path of self-learning, a number of people with a broad range of computer and math skills can accomplish these tasks.Yes, sitting in the classroom somewhere and listening to someone lecture on the fine points of computer programming may be the road to success for lots of people, but it is not beyond achieving this same goal of learning new skills by self-study ... In the first few pages of this book, the notion of interactive python is introduced (called IPython). That is, one would take a canned script or snippet of sample code and run it in a command line interpreter or interactive shell. That is a fancy name for entering code at a prompt, essentially using a cut and paste routine. How simple can it be ... Since the subject text has a bazillion examples or python routines and coding structure, one can view a broad range of examples from the simplistic to the quite advanced level. With this book, following along step by step or page by page, you can almost become a computer (read python) programmer, overnight. I. E., one enters the example code, then follows the book and reviews the resulting output, along with subtle hints about the interpreter operation. The shell (or debugger) will allow you to step into the code line by line and view the resultant computer operations. Thus, not only do you have visual feedback (to check your work) but one of the books highlights are the numerous "On-line resources and help" that will provide you with support material at a click of a mouse. As an aside, I regularly read the Yahoo email Group entitled "the Python Community"([...] each day where someone asks the basic question about "Where can I learn Python by way of self-study?" This book provides that answer and opportunity. So Chapter 1 of this book is appropriately entitled"NumPy Quick Start". As you begin Chapter 2, you will soon recognize the format that follows, and gives the reader (or student) an introduction to key Python topics. For example, to program a machine to organize and otherwise manipulate data, be it text or a set of integers, we need to know some basic objects such as data structures. So after a few words about a data structure like an "array", one needs to create an array using examples (code) from the book. With further comments about "What happened?", while creating an array, you will be provided a mini quiz or "Pop quiz" that will test your comprehension and understanding of the concept at hand. Obviously, the quiz is a good reinforcement tool to aid learning. Next, the author uses additional examples to lead you through some more variations of the objects that you are studying. That is: the "Time for Action" is now an opportunity to alter or create alternate arrays by manipulating their dimensions etc. This Time for Action is actually a number of requests (or examples) for "you to try it". Now, not only are you learning new skills and techniques but in fact are "gaining experience" with Python. As a computer wanabee, I now recommend that you take a pencil in hand and copy some of the examples on paper (that you may be interested in). This may be seen as a proactive learning experience and will help you understand some rudimentary things like the syntax (the ways words combine to form phrases or code) of the Python language.Let me clarify a point here, about this text. It is true that this text is focused on mathematical techniques that the average beginner may not be quite ready for, I. E. basic Statistics and Linear Models as seen in Chapter 3, but I would argue that as we discussed above, the layout of the book and its associated (read: excellent) format allows one to learn and gain experience, one step at a time. Chapter 4 of the book lays out additional functions that start to exhibit the power of NumPy. For example (page 95) starts a discussion of Polynomials, in layman's terms, that of finding a function the fits a set of data. I am of course assuming that you have some interest in math and "numbers" per se', if in fact you have searched for this book at Amazon.com Other Chapters go on to highlight such programming techniques as related to:(3). CSV files, Simple Statistics, Linear Models and Trend Lines, (4). Convenient Functions,(5). Matrices and Universal Functions, (6) Linear Algebra, Random Numbers, Distributions,(7). Special Routines for Sorting, Searching, Financial Functions. (8) Quality and Testing [of software],(9). Plotting (results such as Graphs, Histograms, Scatter Plots etc.),(10.) SciPy., a Python based system for Scientists, Engineering and Mathematics. and(11). PyGames. What could be more fun than learning to program games in Python. As you inspect this book, you will find something of interest to everyone who is acquainted with the word Python.It is an excellent book in a format that is easy to find a keyword or idea and will remain a valuable resource to you, for a long time.

Roberto Avilés Jun 09, 2013

I have been reviewing Ivan Idris Numpy `beginner's guide', second edition of a great book. While in the first edition he started explaining iPython and Matplotlib, this second is an even more practical guide: "less theory, more results". The first chapter deals with essentials as Python, iPython and libraries as NumPy, Matplotlib and Scipy and ends with applications to vectors and arrays using those tools. Just then in second chapter he goes deep into NumPy fundamentals with many exercises using arrays, including passing from an array to a list (exercise that helps to understand what are the differences and advantages of each data type.)Chapter 3 relates to common functions and applications to, example, finances and times. Chapter 4 deals with how to fit polynomials and also Smoothing and as an example, the Hanning function. Chapter 5 is related to matrices and "ufuncs" (short by universal functions) including how to create it and use with a code. These ideas are applied to a classic, Fibonacci numbers but also to Lissajous curves and the drawing of sawtooth and triangle waves, and also how to work with bitwise comparison operations. Up to this point this is the perfect companion for a crash course on the uses of Python and libraries like NumPy; however is also strong enough to be used as part of a college lecture or as the tool that should not be absent in your tool box and desk. Is hard to summarize further chapters; is a very good book because Idris shows his ability to explain in clear and concise sentences and then he applies immediately to exercises in a variety of areas. If you can't find what you need in this book, the only solution might be a do it yourself solution, based in the' big book', the web! In chapter 6 he explains applications to Linear Algebra including eigenvalues and eigenvectors, Fourier Transform, random numbers and applications to a gambling show. In chapter 7 he works with complex numbers, including sorting and searching, and even plays with Bessel functions. Chapter 8 deals on Assuring Quality with Testing and how to assert and compare arrays. Chapter 9 is one of my favorites, because Idris shows excellent examples on how to plot using Matplotlib, including animations! If you cannot impress your audience with these tools, likely they are dead. In Chapter 10 he goes deeper on libraries like Scipy (NumPy + SciPy are essentials on dealing with big arrays of data, like images) but this time Idris also give us hints in how to deal with audio processing. Finally in Chapter 11, he talks about PyGame along with Matplotlib, Numpy and applications to ... Artificial Intelligence, and simulations of life.Certainly one book can hardly cover ALL `what we need'. However, is up to us to get the most of a tool. If you work on Engineering or Science and are in need to deal with data, vectors, matrices, arrays, sounds or images, if you need to plot and to make things evident to others, if you need a data handling Swiss tool in your toolkit, this is it, clear, concise, full of many examples in so many areas that you will always have something to talk with others even from areas different to yours. Read it, use it, enjoy it.

NumPy Beginner's Guide: An action packed guide using real world examples of the easy to use, high performance, free open source NumPy mathematical library. , Second Edition

What do you get with Print?

Contact Details

Shipping Address

Billing Address

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with Print?

Contact Details

Shipping Address

Billing Address

Product Details

Packt Subscriptions

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

Filter reviews by

People who bought this also bought

About the author

FAQs