2.1 Functions as first-class objects

Functional programming is often succinct and expressive. One way to achieve this is by providing functions as arguments and return values for other functions. We’ll look at numerous examples of manipulating functions.

For this to work, functions must be first-class objects in the runtime environment. In programming languages such as C, a function is not a runtime object; because the compiled C code generally lacks internal attributes and methods, there’s little runtime introspection that can be performed on a function. In Python, however, functions are objects that are created (usually) by def statements and can be manipulated by other Python functions. We can also create a function as a callable object or by assigning a lambda object to a variable.

Here’s how a function definition creates an object with attributes:

>>> def example(a, b, **kw): 
...     return a*b 
... 
>>> type(example) ...

2.2 Immutable data

Since we’re not using variables to track the state of a computation, our focus needs to stay on immutable objects. We can make extensive use of tuples, typing.NamedTuples, and frozen @dataclass to provide more complex data structures that are also immutable. We’ll look at these class definitions in detail in Chapter 7, Complex Stateless Objects.

The idea of immutable objects is not foreign to Python. Strings and tuples are two widely-used immutable objects. There can be a performance advantage to using immutable tuples instead of more complex mutable objects. In some cases, the benefits come from rethinking the algorithm to avoid the costs of object mutation.

As an example, here’s a common design pattern that works well with immutable objects: the wrapper() function. A list of tuples is a fairly common data structure. We will often process this list of tuples in one of the two following ways:

• Using higher-order functions: As shown earlier...

2.3 Strict and non-strict evaluation

Functional programming’s efficiency stems, in part, from being able to defer a computation until it’s required. There are two similar concepts for avoiding computation. These are:

• Strictness: Python operators are generally strict and evaluate all sub-expressions from left to right. This means an expression like f(a)+f(b)+f(c) is evaluated as if it was (f(a)+f(b))+f(c). An optimizing compiler might avoid strict ordering to improve performance. Python doesn’t optimize and code is mostly strict. We’ll look at cases where Python is not strict below.

• Eagerness and laziness: Python operators are generally eager and evaluate all sub-expressions to compute the final answer. This means (3-3) * f(d) is fully evaluated even though the first part of the multiplication—the (3-3) sub-expression—is always zero, meaning the result is always zero, no matter what value is computed by the expression f(d). Generator...

2.4 Lazy and eager evaluation

Python’s generator expressions and generator functions are lazy. These expressions don’t create all possible results immediately. It’s difficult to see this without explicitly logging the details of a calculation. Here is an example of the version of the range() function that has the side effect of showing the numbers it creates:

from collections.abc import Iterator 
def numbers(stop: int) -> Iterator[int]: 
    for i in range(stop): 
        print(f"{i=}") 
        yield i

To provide some debugging hints, this function prints each value as the value is yielded. If this function were eager, evaluating numbers(1024) would take the time (and storage) to create all 1,024 numbers. Since the numbers() function is lazy, it only creates a number as it is requested.

We can use this noisy numbers() function in a...

2.5 Recursion instead of an explicit loop state

Functional programs don’t rely on loops and the associated overhead of tracking the state of loops. Instead, functional programs try to rely on the much simpler approach of recursive functions. In some languages, the programs are written as recursions, but Tail-Call Optimization (TCO) in the compiler changes them to loops. We’ll introduce some recursion here and examine it closely in Chapter 6, Recursions and Reductions.

We’ll look at an iteration to test whether a number is a prime number. Here’s a definition from https://mathworld.wolfram.com/PrimeNumber.html: “A prime number ... is a positive integer p > 1 that has no positive integer divisors other than 1 and p itself.” We can create a naive and poorly performing algorithm to determine whether a number has any factors between 2 and the number. This is called the Trial Division algorithm. It has the advantage of simplicity; it works acceptably...

2.6 Functional type systems

Some functional programming languages, such as Haskell and Scala, are statically compiled, and depend on declared types for functions and their arguments. To provide the kind of flexibility Python already has, these languages have sophisticated type-matching rules allowing a generic function to work for a variety of related types.

In object-oriented Python, we often use the class inheritance hierarchy instead of sophisticated function type matching. We rely on Python to dispatch an operator to a proper method based on simple name-matching rules.

Python’s built-in ”duck typing” rules offer a great deal of type flexibility. The more complex type matching rules for a compiled functional language aren’t relevant. It’s common to define a typing.Protocol to specify the features an object must have. The actual class hierarchy doesn’t matter; what matters is the presence of the appropriate methods and attributes.

Python’...

2.7 Familiar territory

One of the ideas that emerges from the previous list of topics is that many functional programming constructs are already present in Python. Indeed, elements of functional programming are already a very typical and common part of OOP.

As a very specific example, a fluent Application Program Interface (API) is a very clear example of functional programming. If we take time to create a class with return self in each method, we can use it as follows:

some_object.foo().bar().yet_more()

We can just as easily write several closely related functions that work as follows:

yet_more(bar(foo(some_object)))

We’ve switched the syntax from traditional object-oriented suffix notation to a more functional prefix notation. Python uses both notations freely, often using a prefix version of a special method name. For example, the len() function is generally implemented by the __len__() class special method.

Of course, the implementation of the preceding class might involve...

2.8 Learning some advanced concepts

We will set some more advanced concepts aside for consideration in later chapters. These concepts are part of the implementation of a purely functional language. Since Python isn’t purely functional, our hybrid approach won’t require deep consideration of these topics.

We will identify these here for the benefit of readers who already know a functional language such as Haskell and are learning Python. The underlying concerns are present in all programming languages, but we’ll tackle them differently in Python. In many cases, we can and will drop into imperative programming rather than use a strictly functional approach.

The topics are as follows:

• Referential transparency: When looking at lazy evaluation and the various kinds of optimizations that are possible in a compiled language, the idea of multiple routes to the same object is important. In Python, this isn’t as important because there aren’t any relevant compile...

2.9 Summary

In this chapter, we’ve identified a number of features that characterize the functional programming paradigm. We started with first-class and higher-order functions. The idea is that a function can be an argument to a function or the result of a function. When functions become the object of additional programming, we can write some extremely flexible and generic algorithms.

The idea of immutable data is sometimes odd in an imperative and object-oriented programming language such as Python. When we start to focus on functional programming, however, we see a number of ways that state changes can be confusing or unhelpful. Using immutable objects can be a helpful simplification.

Python focuses on strict evaluation: all sub-expressions are evaluated from left to right through the statement. Python, however, does perform some non-strict evaluation. The or, and, and if-else logical operators are non-strict: all sub-expressions are not necessarily evaluated.

Generator functions...

2.10 Exercises

This chapter’s exercises are based on code available from Packt Publishing on GitHub. See https://github.com/PacktPublishing/Functional-Python-Programming-3rd-Edition.

In some cases, the reader will notice that the code provided on GitHub includes partial solutions to some of the exercises. These serve as hints, allowing the reader to explore alternative solutions.

In many cases, exercises will need unit test cases to confirm they actually solve the problem. These are often identical to the unit test cases already provided in the GitHub repository. The reader should replace the book’s example function name with their own solution to confirm that it works.