It is generally the tradition that the first program one learns to write in any computer language is "hello world." We maintain the convention in this book! Let's begin with a Hello World program:

import tensorflow as tf
message = tf.constant('Welcome to the exciting world of Deep Neural Networks!')
with tf.Session() as sess:
    print(sess.run(message).decode())

Let us go in depth into this simple code. The first line imports tensorflow. The second line defines the message using tf.constant. The third line defines the Session() using with, and the fourth runs the session using run(). Note that this tells us that the result is a "byte string." In order to remove string quotes and b (for byte) we use the method decode().

TensorFlow 1.x computational graph program structure

TensorFlow 1.x is unlike other programming languages. We first need to build a blueprint of whatever neural network we want...

As discussed, TensorFlow 2.x recommends using a high-level API such as tf.keras, but leaves low-level APIs typical of TensorFlow 1.x for when there is a need to have more control on internal details. tf.keras and TensorFlow 2.x come with some great benefits. Let's review them.

Eager execution

TensorFlow 1.x defines static computational graphs. This type of declarative programming might be confusing for many people. However, Python is typically more dynamic. So, following the Python spirit, PyTorch, another popular deep learning package, defines things in a more imperative and dynamic way: you still have a graph, but you can define, change, and execute nodes on-the-fly, with no special session interfaces or placeholders. This is what is called eager execution, meaning that the model definitions are dynamic, and the execution is immediate. Graphs and sessions should be considered as implementation details.

Both PyTorch and TensorFlow 2 styles...

Today, TensorFlow 2.x is a rich learning ecosystem where, in addition to the core learning engine, there is a large collection of tools that can be freely used. In particular:

• TensorFlow.js (https://www.tensorflow.org/js) is a collection of APIs to train and run inference directly in browsers or in Node.js.
• TensorFlow Lite (https://www.tensorflow.org/lite) is a lightweight version of TensorFlow for embedded and mobile devices. Currently, both Android and iOS are supported in Java and C++.
• TensorFlow Hub (https://www.tensorflow.org/hub) is a complete library supporting the most common machine learning architectures. As of April 2019, Hub only partially supports the tf.Keras API but this issue (https://github.com/tensorflow/tensorflow/issues/25362) is going to be solved soon. We will see an example of Hub in Chapter 5, Advanced Convolutional Neural Networks.
• TensorFlow Extended (TFX) (https://github.com/tensorflow/tfx) is a complete end...

Another legitimate question is whether you should use Keras with TensorFlow as a backend or, instead, use the APIs in tf.keras directly available in TensorFlow. Note that there is not a 1:1 correspondence between Keras and tf.keras. Many endpoints in tf.keras are not implemented in Keras and tf.Keras does not support multiple backends as Keras. So, Keras or tf.keras? My suggestion is the second option rather than the first one. tf.keras has multiple advantages over Keras, consisting of TensorFlow enhancements discussed in this chapter (eager execution; native support for distributed training, including training on TPUs; and support for the TensorFlow SavedModel exchange format). However, the first option is still the most relevant one if you plan to write highly portable code that can run on multiple backends, including Google TensorFlow, Microsoft CNTK, Amazon MXnet, and Theano. Note that Keras is an independent open source project, and its development is not dependent...

TensorFlow 2.0 is a rich development ecosystem composed of two main parts: Training and Serving. Training consists of a set of libraries for dealing with datasets (tf.data), a set of libraries for building models, including high-level libraries (tf.Keras and Estimators), low-level libraries (tf.*), and a collection of pretrained models (tf.Hub), which will be discussed in Chapter 5, Advanced Convolutional Neural Networks. Training can happen on CPUs, GPUs, and TPUs via distribution strategies and the result can be saved using the appropriate libraries. Serving can happen on multiple platforms, including on-prem, cloud, Android, iOS, Raspberry Pi, any browser supporting JavaScript, and Node.js. Many language bindings are supported, including Python, C, C#, Java, Swift, R, and others. The following diagram summarizes the architecture of TensorFlow 2.0 as discussed in this chapter:



Figure 6: Summary of TensorFlow 2.0 architecture

• tf.data can be used to load...