Have you ever played the game Fizz buzz? Don't worry if you haven't. The following is a simple explanation of what the game is about.

Fizz buzz has been used in a fun example by Joel Grus, one of the research engineers at the Allen Institute of Artificial Intelligence (AI2), while writing a blog post [3] on TensorFlow. Although this particular example doesn't solve any practical problems, the blog post got quite a lot of traction and it is fun to see how a neural network learns to find a mathematical pattern from a number stream.

Building a data pipeline is as important as the architecture of your network, especially when you train your network in real time. The data that you get from the wild is never going to be clean, and you'll have to process it before throwing it at your network. For example, if we were to collect data for predicting whether a person buys a product or not, we would end up having outliers. Outliers could be of any kind and unpredictable. Somebody could have made an order accidently, for example, or they could have given access to their friends who then made the order, and so on.

Theoretically, deep neural networks are ideal for finding patterns and solutions from your dataset because they are supposed to mimic the human brain. However, in practice, this is often not quite the case. Your network will be able to solve problems easily by finding the pattern if your data is clean and properly formatted. PyTorch gives data preprocessing wrappers out of the box...

Now we are going to build a novice, NumPy-like model, not using any PyTorch-specific approach. Then, in the next session, we'll convert the same model to PyTorch's method. If you come from a NumPy background, you'll feel at home, but if you are an advanced deep learning practitioner who has used other frameworks, please take the liberty of skipping this session.

x = torch.from_numpy(trX).to(device=device, dtype=dtype)
y = torch.from_numpy(trY).to(device=device, dtype=dtype)
w1 = torch.randn(input_size, hidden_size, requires_grad=True, device=device, dtype=dtype)
w2 = torch.randn(hidden_size, output_size, requires_grad=True, device=device, dtype=dtype)
b1 = torch.zeros(1, hidden_size, requires_grad=True, device=device, dtype=dtype)
b2 = torch.zeros(1, output_size, requires_grad=True, device=device, dtype=dtype...

So far, we have developed a simple two-layer neural network in a hybrid NumPy-PyTorch style. We have coded each operation line by line, like how we do it in NumPy, and we have adopted automatic differentiation from PyTorch so that we don't have to code the backward pass.

On the way, we have learned how to wrap matrices (or tensors) in PyTorch, and that helps us with backpropagation. The PyTorch way of doing the same thing is a bit more convenient and that is what we are going to discuss in this section. PyTorch gives access to almost all the functionality required for a deep learning project inbuilt. Since PyTorch supports all the mathematical functions available in Python, it's not a tough task to build one function if it's not available in the core. You can not only build any functionality you need, but PyTorch defines the derivative function of the functionality you build implicitly.

PyTorch is helpful for people who need to know the low-level...

In this chapter, we have learned how to build a simple neural network in the most basic way and convert that to a PyTorch's way. The basic building block of deep learning starts here. Once we know how and why the methodology we follow exists, we'll be able to take the big steps. Any deep learning model, regardless of the size, usage, or algorithm, can be built with the concepts we have learned in this chapter. Because of that, understanding this chapter thoroughly is critical for going through future chapters. In the next chapter, we will dive into the deep learning workflow.