You're reading from Python High Performance, Second Edition Build high-performing, concurrent, and distributed applications

Product type Paperback

Published in May 2017

Publisher Packt

ISBN-13 9781787282896

Length 270 pages

Edition 2nd Edition

Languages

Python

Tools

Dask

Concepts

High Performance Programming

Author (1):

Dr. Gabriele Lanaro

View More author details

Table of Contents (10) Chapters

Preface

Benchmarking and Profiling FREE CHAPTER

Pure Python Optimizations

Fast Array Operations with NumPy and Pandas

C Performance with Cython

Exploring Compilers

Implementing Concurrency

Parallel Processing

Distributed Processing

Designing for High Performance

Using PySpark

Nowadays, Apache Spark is one of the most popular projects for distributed computing. Developed in Scala, Spark was released in 2014, and integrates with HDFS and provides several advantages and improvements over the Hadoop MapReduce framework.

Contrary to Hadoop MapReduce, Spark is designed to process data interactively and supports APIs for the Java, Scala, and Python programming languages. Given its different architecture, especially by the fact that Spark keep results in memory, Spark is generally much faster than Hadoop MapReduce.

Setting up Spark and PySpark

Setting up PySpark from scratch requires the installation of the Java and Scala runtimes, the compilation of the project from source, and the configuration of Python and Jupyter notebook...

The rest of the chapter is locked

A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.

Unlock this book and the full library FREE for 7 days

Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of

Start free trial

Renews at $19.99/month. Cancel anytime

Authors (1)

Dr. Gabriele Lanaro

Dr. Gabriele Lanaro is passionate about good software and is the author of the chemlab and chemview open source packages. His interests span machine learning, numerical computing visualization, and web technologies. In 2013, he authored the first edition of the book High Performance Python Programming. He has been conducting research to study the formation and growth of crystals using medium and large-scale computer simulations. In 2017, he obtained his PhD in theoretical chemistry.

See other products by Dr. Gabriele Lanaro