Introduction to One-shot Learning

Humans can learn new things with a small set of examples. When presented with stimuli, humans seem to be able to understand new concepts quickly and then recognize variations of those concepts in the future. A child can learn to recognize a dog from a single picture, but a machine learning system needs a lot of examples to learn the features of a dog and recognize them in the future. Machine learning, as a field, has been highly successful at a variety of tasks, such as classification and web searching, as well as image and speech recognition. Often, however, these models do not perform well without a large amount of data (examples) to learn from. The primary motivation behind this book is to train a model with very few examples that is capable of generalizing to unfamiliar categories without extensive retraining.

Deep learning has played an important...

Technical requirements

The following libraries will be required to learn and execute the project in this chapter:

Python
Anaconda
Jupyter Notebook
PyTorch
Matplotlib
Scikit-learn

You can find the code files for this chapter in the GitHub repository of this book: https://github.com/PacktPublishing/Hands-on-One-Shot-Learning-with-Python.

The human brain – overview

The human brain has been a subject of research since the beginning of civilization. If we look into the development of a child, we will observe that as they grow, their ability to learn also grows. First, they learn about food, then they learn to identify faces. Every time a child learns something, information is encoded into some portion of the brain. Still, the real question remains, how does information get stored in our brains? Why is some information hardcoded, yet other information is easily forgotten?

How the human brain learns

Most of the information on how the brain trains itself to process data is unknown, but there are various theories that explore it. If we look into the structure...

Machine learning – historical overview

Machine learning is a program that, given a task (loss function), learns through experience (training data). With experience, that program learns to perform the given task to a desirable standard. During the 1960s, machine learning was majorly focused on creating different forms of data preprocessing filters. With the introduction of image filters, the focus then shifted toward computer vision, and major research work was undertaken in this domain during the 1990s and 2000s. After some stability in terms of traditional machine learning algorithms being developed, researchers moved to the probabilistic domain, as it became more promising with the introduction of high-dimensional data. Deep learning bloomed when it won the ImageNet Challenge in 2012, and has since taken on an important role in the field of data science.

Machine learning...

One-shot learning – overview

One-shot learning can be seen as an approach to train machines in a way that is similar to how humans learn. One-shot learning is an approach to learn a new task using limited supervised data with the help of strong prior knowledge. The first work published that resulted in high accuracy for the image classification problem dates back to the 2000s by Dr. Fei Fei Li—although, in recent years, researchers have made good progress tackling it through different deep learning architectures and optimization algorithms, such as matching networks, model agnostic meta-learning, and memory-augmented neural networks. One-shot learning has a lot of applications in several industries—the medical and manufacturing industries in particular. In medicine, we can use one-shot learning when there is limited data available, for example, when working...

Setting up your environment

In this section, we will set up a virtual environment for our coding exercise and questions using the following steps:

Clone the repository by going into the directory of your choice and running the following command in the Git Bash command line:

git clone https://github.com/Packt-Publishing/Hands-on-One-Shot-Learning.git

Go to the Chapter01 directory of the cloned repository:

cd Hands-on-One-Shot-Learning/Chapter01

Then, open a Terminal and use the following command to install Anaconda for Python, version 3.6 (https://docs.anaconda.com/anaconda/install/), and create a virtual environment:

conda create --name environment_name python=3.6

In steps 3 and 4, you can replace environment_name with an easy name to remember, such as one_shot, or a name of your choice.

Activate the environment using the following command:

source activate environment_name...

Coding exercise

In this section, we will explore a basic one-shot learning approach. As humans, we have a hierarchical way of thinking. For example, if we see something unknown to us, we look for its similarity to objects we already know. Similarly, in this exercise, we will use a nonparametric kNN approach to find classes. We will also compare its performance to the basic neural network architecture.

kNN – basic one-shot learning

In this exercise, we will compare kNN to neural networks where we have a small dataset. We will be using the iris dataset imported from the scikit-learn library.

To begin, we will first discuss the basics of kNN. The kNN classifier is a nonparametric classifier that simply stores the training...

Summary

Deep learning has revolutionized the field of data science and it is still making progress, but there are still major industries that are yet to experience all of the advantages of deep learning, such as the medical and manufacturing industries. The zenith of human achievement will be to create a machine that can learn as humans do and that can become an expert in the same way humans can. Successful deep learning, though, usually comes with the prerequisite of having very large datasets to work from. Fortunately, this book focuses on architectures that can do away with this prerequisite.

In this chapter, we learned about the human brain and how the structure of an artificial neural network is close to the structure of our brain. We introduced the basic concepts of machine learning and deep learning, along with their challenges. We also discussed one-shot learning and its...

Questions

Why does a kNN work better than a newly trained artificial neural network for a one-shot learning task?
What are nonparametric machine learning algorithms?
Are decision trees a parametric or nonparametric algorithm?
Experiment with other classification algorithms as a coding exercise and compare the results.