Media and news use AI as a substitute buzzword for any technology related to data analysis. In fact, AI is a sub-field of computer science and mathematics. It all started in the 1950s, when several researchers started asking whether computers can learn, think, and reason. 70 years later, we still do not know the answer. However, we have made significant progress in a specific kind of AI that solves thoroughly specified narrow tasks: weak AI.

Science fiction novels tell about machines that can reason and think like humans. In scientific language, they are described as strong AI. Strong AI can think like a human, and its intellectual abilities may be much more advanced. The creation of strong AI remains the main long-term dream of the scientific community. However, practical applications are all about weak AI. While strong AI tries to solve the problem of general intelligence...

Machine learning is by far the most important tool of a data scientist. It allows us to create algorithms that discover patterns in data with thousands of variables. We will now explore different types and capabilities of machine learning algorithms.

Machine learning is a scientific field that studies algorithms that can learn to perform tasks without specific instructions, relying on patterns discovered in data. For example, we can use algorithms to predict the likelihood of having a disease or assess the risk of failure in complex manufacturing equipment. Every machine learning algorithm follows a simple formula. In the following diagram, you can see a high-level decision process that is based on a machine learning algorithm. Each machine learning model consumes data to produce information that can support human decisions or fully automate them...

Before writing this section, I was thinking about the many ways we can draw a line between machine learning and deep learning. Each of them was contradictory in some way. In truth, you can't separate deep learning from machine learning because deep learning is a subfield of machine learning. Deep learning studies a specific set of models called neural networks. The first mentions of the mathematical foundations of neural networks date back to the 1980s, and the theory behind modern neural networks originated in 1958. Still, they failed to show good results until the 2010s. Why?

The answer is simple: hardware. Training big neural networks uses a great amount of computation power. But not any computation power will suffice. It turns out that neural networks do a lot of matrix operations under the hood. Strangely, rendering computer graphics also...

To show how deep learning may work in practical settings, we will explore product matching.

Up-to-date pricing is very important for large internet retailers. In situations where your competitor lowers the price of a popular product, late reaction leads to large profit losses. If you know the correct market price distributions for your product catalog, you can always remain a step ahead of your competitors. To create such a distribution for a single product, you first need to find this product description on a competitor's site. While automated collection of product descriptions is easy, product matching is the hard part.

Once we have a large volume of unstructured text, we need to extract product attributes from it. To do this, we first need to tell whether two descriptions refer to the same product. Suppose that we have collected a large dataset of...

Up to this point, we have talked about predictive models. The main purpose of a predictive model is to recognize and forecast. The explanation behind the model's reasoning is of lower priority. On the contrary, causal inference tries to explain relationships in the data rather than to make predictions about the future events. In causal inference, we check whether an outcome of some action was not caused by so-called confounding variables. Those variables can indirectly influence action through the outcome. Let's compare causal inference and predictive models through several questions that they can help to answer:

• Prediction models:
  • When will our sales double?
  • What is the probability of this client buying a certain product?
• Causal inference models:
  • Was this cancer treatment effective? Or is the effect apparent only because of the...

In this chapter, we have explored the practical applications of AI, data science, machine learning, deep learning, and causal inference. We have defined machine learning as a field that studies algorithms that use data to support decisions and give insights without specific instructions. There are three main machine learning methodologies: supervised, unsupervised, and reinforcement learning. In practice, the most common types of task we solve using machine learning are regression and classification. Next, we described deep learning as a subset of machine learning devoted to studying neural network algorithms. The main application domains of deep learning are computer vision and NLP. We have also touched on the important topic of causal inference: the field that studies a set of methods for discovering causal relationships in data. You now know a lot about general data...