Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletter Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds
Learning Cython Programming (Second Edition)
Learning Cython Programming (Second Edition)

Learning Cython Programming (Second Edition): Expand your existing legacy applications in C using Python , Second Edition

eBook
€8.99 €23.99
Paperback
€29.99
Subscription
Free Trial
Renews at €18.99p/m

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Billing Address

Table of content icon View table of contents Preview book icon Preview Book

Learning Cython Programming (Second Edition)

Chapter 2. Understanding Cython

As I mentioned previously, there are a number of methods of using Cython. As the basics are very familiar to any Python programmer, it's important to review the linking models before getting into the programming language. This is what drives the design of applications when using Cython.

Next, we will get more familiar with the Cython programming language constructs, namely, the distinction between cdef and cpdef. Then, we will look at getting the most out of Cython by interfacing directly with native C types. Later in this book, we will see that it's possible to use native C++ STL container types. This is where you will gain the optimizations in execution, as no Python runtime is required to work with native types.

Finally, we will see how easy it is to work with callbacks to and from C and Python code. This is an interesting technique whereby you can offload logic from C code to Python.

Therefore, in this chapter, we will be diving into...

Linking models

Linking models are extremely important when considering how we can extend or embed things in native applications. There are two main linking models for Cython:

Fully embedded Python within C/C++ code, which looks like the following screenshot:

Linking models

Using this method of embedding the Python runtime into a native application means you initiate execution of code directly from any point in your C/C++ code, as opposed to the Chapter 1, Cython Won't Bite where we had to run the Python interpreter and call an import to execute native code.

For the sake of completeness, here is the import model of using Cython:

Linking models

This would be a more Pythonic approach to Cython, and will be helpful if your code base is mostly Python. We will review an example of the Python lxml module, which provides a Cython backend, later in this book, and we can compare it to the native Python backend to review the speed and execution of both code bases to perform the same task.

Cython keyword – cdef

The cdef keyword tells the compiler that this statement is a native C type or native function. Remember from Chapter 1, Cython Won't Bite that we used this line to declare the C prototype function:

cdef int AddFunction(int, int)

This is the line that let us wrap the native C function into a Python callable using the Python def keyword. We can use this in many contexts, for example, we can declare normal variables for use within a function to speed up execution:

def square(int x):
    return x ** 2

This is a trivial example, but it will tell the compiler that we will always be squaring an integer. However, for normal Python code, it's a little more complicated as Python has to worry a lot more about losing precision when it comes to handling many different types. But in this case, we know exactly what the type is and how it can be handled.

You might also have noticed that this is a simple def function, but because it will be fed to the Cython compiler, this...

Typedef and function pointers

The typedef in C/C++ code allows the programmer to give a new name or alias to any type. For example, one could typedef an int to myint. Or you can just simply typedef a struct so that you don't have to refer to the struct with the keyword struct every time. For example, consider this C struct and typedef:

struct foobar {
  int x;
  char * y;
};
typedef struct foobar foobar_t;

In Cython, this can be described by the following:

cdef struct foobar:
    int x
    char * y
ctypedef foobar foobar_t

Note we can also typedef pointer types as below:

ctypedef int * int_ptr

We can also typedef function C/C++ pointers, as follows:

typedef void (*cfptr) (int)

In Cython, this will be as follows:

ctypedef void (*cfptr)(int)

Using the function pointer is just as you would expect:

cdef cfptr myfunctionptr = &myfunc

There is some magic going on here with function pointers as it's simply not safe for raw Python code to directly call a Python function or vice versa. Cython...

The public keyword

This is a very powerful keyword in Cython. It allows any cdef declaration with the public modifier to output a respective C/C++ header with the relative declaration accessible from C/C++. For example, we can declare:

cdef public struct CythonStruct:
    size_t number_of_elements;
    char ** elements;

Once the compiler handles this, you will have an output of cython_input.h:

 struct CythonStruct {
    size_t number_of_elements;
    char ** elements;
};

The main caveat, if you're going to call the Python public declarations directly from C, is that, if your link model is fully embedded and linked against libpython.so, you need to use some boilerplate code to initialize Python correctly:

#include <Python.h>

int main(int argc, char **argv) {
    Py_Initialize ();
    // code in here
    Py_Finalize ();
    return 0;
}

And before calling anything with the function, you need to initialize the Python module example if you have a cythonfile.pyx file, and compile it with...

Keyword cpdef

So far, we have seen two different function declarations in Cython, def and cdef, to define functions. There is one more declaration—cpdef. The def is a Python-only function, so it is only callable from Python or Cython code blocks; calling from C does not work. The cdef is the opposite; this means that it's callable from C and not from Python. For example, if we create a function such as:

cpdef public test (int x):
   …
  return 1

It will generate the following function prototype:

__PYX_EXTERN_C DL_IMPORT(PyObject) *test(int, int __pyx_skip_dispatch);

The public keyword will make sure we generate the needed header so that we can call it from C. Calling from pure Python, we can work with this as if it was just any other Python function. The drawback of using cpdef is that the native return type is PyObject *, which requires you to know exactly what the return type is and consult the Python API documentation to access the data. I prefer keeping bindings between...

Linking models


Linking models are extremely important when considering how we can extend or embed things in native applications. There are two main linking models for Cython:

Fully embedded Python within C/C++ code, which looks like the following screenshot:

Using this method of embedding the Python runtime into a native application means you initiate execution of code directly from any point in your C/C++ code, as opposed to the Chapter 1, Cython Won't Bite where we had to run the Python interpreter and call an import to execute native code.

For the sake of completeness, here is the import model of using Cython:

This would be a more Pythonic approach to Cython, and will be helpful if your code base is mostly Python. We will review an example of the Python lxml module, which provides a Cython backend, later in this book, and we can compare it to the native Python backend to review the speed and execution of both code bases to perform the same task.

Cython keyword – cdef


The cdef keyword tells the compiler that this statement is a native C type or native function. Remember from Chapter 1, Cython Won't Bite that we used this line to declare the C prototype function:

cdef int AddFunction(int, int)

This is the line that let us wrap the native C function into a Python callable using the Python def keyword. We can use this in many contexts, for example, we can declare normal variables for use within a function to speed up execution:

def square(int x):
    return x ** 2

This is a trivial example, but it will tell the compiler that we will always be squaring an integer. However, for normal Python code, it's a little more complicated as Python has to worry a lot more about losing precision when it comes to handling many different types. But in this case, we know exactly what the type is and how it can be handled.

You might also have noticed that this is a simple def function, but because it will be fed to the Cython compiler, this will work just...

Typedef and function pointers


The typedef in C/C++ code allows the programmer to give a new name or alias to any type. For example, one could typedef an int to myint. Or you can just simply typedef a struct so that you don't have to refer to the struct with the keyword struct every time. For example, consider this C struct and typedef:

struct foobar {
  int x;
  char * y;
};
typedef struct foobar foobar_t;

In Cython, this can be described by the following:

cdef struct foobar:
    int x
    char * y
ctypedef foobar foobar_t

Note we can also typedef pointer types as below:

ctypedef int * int_ptr

We can also typedef function C/C++ pointers, as follows:

typedef void (*cfptr) (int)

In Cython, this will be as follows:

ctypedef void (*cfptr)(int)

Using the function pointer is just as you would expect:

cdef cfptr myfunctionptr = &myfunc

There is some magic going on here with function pointers as it's simply not safe for raw Python code to directly call a Python function or vice versa. Cython understands...

The public keyword


This is a very powerful keyword in Cython. It allows any cdef declaration with the public modifier to output a respective C/C++ header with the relative declaration accessible from C/C++. For example, we can declare:

cdef public struct CythonStruct:
    size_t number_of_elements;
    char ** elements;

Once the compiler handles this, you will have an output of cython_input.h:

 struct CythonStruct {
    size_t number_of_elements;
    char ** elements;
};

The main caveat, if you're going to call the Python public declarations directly from C, is that, if your link model is fully embedded and linked against libpython.so, you need to use some boilerplate code to initialize Python correctly:

#include <Python.h>

int main(int argc, char **argv) {
    Py_Initialize ();
    // code in here
    Py_Finalize ();
    return 0;
}

And before calling anything with the function, you need to initialize the Python module example if you have a cythonfile.pyx file, and compile it with the...

Keyword cpdef


So far, we have seen two different function declarations in Cython, def and cdef, to define functions. There is one more declaration—cpdef. The def is a Python-only function, so it is only callable from Python or Cython code blocks; calling from C does not work. The cdef is the opposite; this means that it's callable from C and not from Python. For example, if we create a function such as:

cpdef public test (int x):
   …
  return 1

It will generate the following function prototype:

__PYX_EXTERN_C DL_IMPORT(PyObject) *test(int, int __pyx_skip_dispatch);

The public keyword will make sure we generate the needed header so that we can call it from C. Calling from pure Python, we can work with this as if it was just any other Python function. The drawback of using cpdef is that the native return type is PyObject *, which requires you to know exactly what the return type is and consult the Python API documentation to access the data. I prefer keeping bindings between the languages simpler...

Logging from C/C++ into Python


An example of everything brought together is reusing the Python logging module directly from C. We want a few macros, such as info, error, and debug that can all handle a variable number of arguments and works as if we are calling a simple printf method.

To achieve this, we must make a Python logging backend for our C/C++ code. We need an initialization function to tell Python about our output logfile, and some wrappers for each info, error, and debug. We can simply write the public cdef wrappers as:

import logging

cdef public void initLoggingWithLogFile(const char * logfile):
    logging.basicConfig(filename = logfile,
                        level = logging.DEBUG,
                        format = '%(levelname)s %(asctime)s: %(message)s',
                        datefmt = '%m/%d/%Y %I:%M:%S')

cdef public void python_info(char * message):
    logging.info(message)

cdef public void python_debug(char * message):
    logging.debug(message)

cdef public void...
Left arrow icon Right arrow icon

Key benefits

  • Learn how to extend C applications with pure Python code
  • Get more from Python – you’ll not only learn Cython, you’ll also unlock a greater understanding of how to harness Python
  • Packed with tips and tricks that make Cython look easy, dive into this accessible programming guide and find out what happens when you bring C and Python together!

Description

Cython is a hybrid programming language used to write C extensions for Python language. Combining the practicality of Python and speed and ease of the C language it’s an exciting language worth learning if you want to build fast applications with ease. This new edition of Learning Cython Programming shows you how to get started, taking you through the fundamentals so you can begin to experience its unique powers. You’ll find out how to get set up, before exploring the relationship between Python and Cython. You’ll also look at debugging Cython, before moving on to C++ constructs, Caveat on C++ usage, Python threading and GIL in Cython. Finally, you’ll learn object initialization and compile time, and gain a deeper insight into Python 3, which will help you not only become a confident Cython developer, but a much more fluent Python developer too.

Who is this book for?

This book is for developers who are familiar with the basics of C and Python programming and wish to learn Cython programming to extend their applications.

What you will learn

  • Reuse Python logging in C
  • Make an IRC bot out of your C application
  • Extend an application so you have a web server for rest calls
  • Practice Cython against your C++ code
  • Discover tricks to work with Python ConfigParser in C
  • Create Python bindings for native libraries
  • Find out about threading and concurrency related to GIL
  • Expand Terminal Multiplexer Tmux with Cython

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Feb 22, 2016
Length: 110 pages
Edition : 2nd
Language : English
ISBN-13 : 9781785289125
Category :
Languages :
Tools :

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Billing Address

Product Details

Publication date : Feb 22, 2016
Length: 110 pages
Edition : 2nd
Language : English
ISBN-13 : 9781785289125
Category :
Languages :
Tools :

Packt Subscriptions

See our plans and pricing
Modal Close icon
€18.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
€189.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just €5 each
Feature tick icon Exclusive print discounts
€264.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just €5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total 95.97
Learning Python Design Patterns - Second Edition
€32.99
Learning Cython Programming (Second Edition)
€29.99
Modular Programming with Python
€32.99
Total 95.97 Stars icon
Banner background image

Table of Contents

7 Chapters
1. Cython Won't Bite Chevron down icon Chevron up icon
2. Understanding Cython Chevron down icon Chevron up icon
3. Extending Applications Chevron down icon Chevron up icon
4. Debugging Cython Chevron down icon Chevron up icon
5. Advanced Cython Chevron down icon Chevron up icon
6. Further Reading Chevron down icon Chevron up icon
Index Chevron down icon Chevron up icon

Customer reviews

Rating distribution
Full star icon Empty star icon Empty star icon Empty star icon Empty star icon 1
(1 Ratings)
5 star 0%
4 star 0%
3 star 0%
2 star 0%
1 star 100%
Todd Leonhardt Dec 25, 2016
Full star icon Empty star icon Empty star icon Empty star icon Empty star icon 1
For the vast majority of people looking to learn Cython, this book is absolutely horrible. It is poorly written and does a truly bad job of covering the basics of Python. If you want to learn Cython, there are vastly better resources: the online docs at cython.org are excellent, as is the book "Cython" by Kurt Smith. There is also a good 3.5 hour training video on Youtube from SciPy 2015.However, this book does have a few hidden gems for someone who is already a seasoned C, Python, and Cython developer. It does have a couple excellent examples of how to get C applications to easily call into Python code by using Cython.The book really should have a different title. As a "Learning Cython Programming" book, it deserves 1 star. But as a "Learning how to easily call Python code from a C application by using Cython" book, it would deserve 3 or 4 stars. But I would estimate that less than 5% of the people interested in learning Cython fish to learn it for this purpose - probably about 80% want to learn it to optimize existing Python code and maybe 15 to 20% want to learn it to wrap existing C/C++ code so it can be called from Python.
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

How do I buy and download an eBook? Chevron down icon Chevron up icon

Where there is an eBook version of a title available, you can buy it from the book details for that title. Add either the standalone eBook or the eBook and print book bundle to your shopping cart. Your eBook will show in your cart as a product on its own. After completing checkout and payment in the normal way, you will receive your receipt on the screen containing a link to a personalised PDF download file. This link will remain active for 30 days. You can download backup copies of the file by logging in to your account at any time.

If you already have Adobe reader installed, then clicking on the link will download and open the PDF file directly. If you don't, then save the PDF file on your machine and download the Reader to view it.

Please Note: Packt eBooks are non-returnable and non-refundable.

Packt eBook and Licensing When you buy an eBook from Packt Publishing, completing your purchase means you accept the terms of our licence agreement. Please read the full text of the agreement. In it we have tried to balance the need for the ebook to be usable for you the reader with our needs to protect the rights of us as Publishers and of our authors. In summary, the agreement says:

  • You may make copies of your eBook for your own use onto any machine
  • You may not pass copies of the eBook on to anyone else
How can I make a purchase on your website? Chevron down icon Chevron up icon

If you want to purchase a video course, eBook or Bundle (Print+eBook) please follow below steps:

  1. Register on our website using your email address and the password.
  2. Search for the title by name or ISBN using the search option.
  3. Select the title you want to purchase.
  4. Choose the format you wish to purchase the title in; if you order the Print Book, you get a free eBook copy of the same title. 
  5. Proceed with the checkout process (payment to be made using Credit Card, Debit Cart, or PayPal)
Where can I access support around an eBook? Chevron down icon Chevron up icon
  • If you experience a problem with using or installing Adobe Reader, the contact Adobe directly.
  • To view the errata for the book, see www.packtpub.com/support and view the pages for the title you have.
  • To view your account details or to download a new copy of the book go to www.packtpub.com/account
  • To contact us directly if a problem is not resolved, use www.packtpub.com/contact-us
What eBook formats do Packt support? Chevron down icon Chevron up icon

Our eBooks are currently available in a variety of formats such as PDF and ePubs. In the future, this may well change with trends and development in technology, but please note that our PDFs are not Adobe eBook Reader format, which has greater restrictions on security.

You will need to use Adobe Reader v9 or later in order to read Packt's PDF eBooks.

What are the benefits of eBooks? Chevron down icon Chevron up icon
  • You can get the information you need immediately
  • You can easily take them with you on a laptop
  • You can download them an unlimited number of times
  • You can print them out
  • They are copy-paste enabled
  • They are searchable
  • There is no password protection
  • They are lower price than print
  • They save resources and space
What is an eBook? Chevron down icon Chevron up icon

Packt eBooks are a complete electronic version of the print edition, available in PDF and ePub formats. Every piece of content down to the page numbering is the same. Because we save the costs of printing and shipping the book to you, we are able to offer eBooks at a lower cost than print editions.

When you have purchased an eBook, simply login to your account and click on the link in Your Download Area. We recommend you saving the file to your hard drive before opening it.

For optimal viewing of our eBooks, we recommend you download and install the free Adobe Reader version 9.