Machine Learning Solutions: Expert techniques to tackle complex machine learning problems using Python

The stock market is a place where you can buy and sell units of ownership in the company, which we call stocks. If the company performs well and increases its profit, then you will earn some profit as well because you have the stocks of the company, but if the company's profit goes down, then you will lose the money you have with the company. So if you invest your money in the right company at the right time, it could lead to you earning quite a lot of money. The question is which company's stock should you buy? Is there any way we can predict the future prices of the stock of any company given the historical prices of the company's stock so that we can have higher chances of getting good returns? The answer is yes. This is what we will explore in this chapter.

If you invest in the stock market, then you may have heard that stock prices are completely random and unpredictable. This is called the efficient market hypothesis, but a majority of the big financial...

In order to build the model, first we need to collect the data. We will use the following two data points:

DJIA index prices give us an overall idea about the stock market's movements on a particular day, whereas news articles help us find out how news affects the stock prices. We will build our model using these two data points. Now let's collect the data.

Collecting DJIA index prices

In order to collect the DJIA index prices, we will use Yahoo Finance. You can visit this link: https://finance.yahoo.com/quote/%5EDJI/history?period1=1196706600&period2=1512325800&interval=1d&filter=history&frequency=1d. Once you click on this link, you can see that the price data shows up. You can change the time period and click on the Download Data link and that's it; you can have all the data in .csv file format. Refer to the following screenshot of the Yahoo finance DJIA index price page:

Figure 2.1: Yahoo...

In this section, we will understand the meaning of data attributes, which will help us understand what kind of dataset we are going to deal with and what kind of preprocessing is needed for the dataset. We understand our dataset in two sections, and those sections are given as follows:

In this section, we will mainly cover data preprocessing and data analysis. As a part of data preprocessing, we are preparing our training dataset. You may be wondering what kind of data preparation I'm talking about, considering we already have the data. Allow me to tell you that we have two different datasets and both datasets are independent. So, we need to merge the DJIA dataset and NYTimes news article dataset in order to get meaningful insights from these datasets. Once we prepare our training dataset, we can train the data using different machine learning (ML) algorithms.

Now let's start the coding to prepare the training dataset. We will be using numpy, csv, JSON, and pandas as our dependency libraries. Here, our code is divided into two parts. First, we will prepare the dataset for the DJIA index dataset and then we will move to the next part, which is preparing the NYTimes news article dataset. During the preparation of the training dataset,...

As discussed earlier, we want to predict the close price for the DJIA index for a particular trading day. In this section, we will do feature selection based on our intuition for our basic prediction model for stock prices. We have already generated the training dataset. So, now we will load the saved .pkl format dataset and perform feature selection as well as minor data processing. We will also generate the sentiment score for each of the filtered NYTimes news articles and will use this sentiment score to train our baseline model. We will use the following Python dependencies:

This section has the following steps:

So, let's begin coding!

Loading the dataset

We have saved the data in the pickle format, and now we need to load data from it. You can refer to the following code snippet:

Figure 2.16: Code snippet for loading the dataset from the pickle file

You can refer to the code by clicking...

In this section, we will choose the Machine Learning (ML) algorithm based on our intuition and then perform training using our training dataset. This is the first model for this particular chapter, so the trained model is our baseline model, which we will improve later on. So, let's decide which kind of ML algorithm suits this stock price prediction application.

The stock price prediction application is a time-series analysis problem, where we need to predict the next point in the time series. This prediction activity is similar to linear regression, so we can say that this application is a kind of regression problem and any algorithm from the regression family should work. Let's select the ensemble algorithm, which is RandomForestRegressor, in order to develop our baseline model. So let's train our baseline model, and, based on the result of that model, we will modify our approach.

As you know, we have selected the RandomForestRegressor algorithm. We will be using the scikit-learn library to train the model. These are the steps we need to follow:

So, let's implement each of these steps one by one.

Splitting the training and testing dataset

We have 10 years of data values. So for training purposes, we will be using 8 years of the data, which means the dataset from 2007 to 2014. For testing purposes, we will be using 2 years of the data, which means data from 2015 and 2016. You can refer to the code snippet in the following screenshot to implement this:

Figure 2.22: Splitting the training and testing dataset

As you can see from the preceding screenshot, our training dataset has been stored in the train dataframe and our testing dataset has been stored in the test dataframe...

In this section, we will understand the testing matrix and visualization approaches to evaluate the performance of the trained ML model. So let's understand both approaches, which are as follows:

The default testing matrix

We are using the default score API of scikit-learn to check how well the ML is performing. In this application, the score function is the coefficient of the sum of the squared error. It is also called the coefficient of R2, which is defined by the following equation:

Here, u indicates the residual sum of squares. The equation for u is as follows:

The variable v indicates the total sum of squares. The equation for v is as follows:

The best possible score is 1.0, and it can be a negative score as well. A negative score indicates that the trained model can be arbitrarily worse. A constant model that always predicts the expected value for label y, disregarding the input features, will produce an...

In this section, we will be implementing our testing approach so that we can evaluate our model's accuracy. We will first generate the output prediction and then we'll start testing it. We will be implementing the following steps here:

Generating and interpreting the output

To generate the prediction, we are using the treeinterpreter library. We are predicting the price value for each of our testing dataset records using the following code:

Figure 2.26: Code snippet for generating the prediction

Here, prediction is the array in which we have elements that are the corresponding predicted adj close price for all records of the testing dataset. Now, we will compare this predicted output with the actual adj close price of the testing dataset. By doing this, we will get to know how accurately our first model is predicting the adj close price. In order to evaluate further, we will generate...

In this section, we will be discussing the problems of the existing approach. There are mainly three errors we could have possibly committed, which are listed as follows:

Let's discuss each of the points one by one.

In this section, we will be looking at the key concepts and approaches for alignment and smoothing. It is not that difficult to implement the Logistic Regression algorithm; we will be using the scikit-learn API. So, we will start with understanding the concepts and approaches for implementation.

In this section, we will discuss the three parts of implementation, which are as follows:

Implementation

Here, we are implementing the following:

• Alignment

• Smoothing

• Logistic Regression

We have already discussed the approach and key concepts, so now we just focus on the code part here. You can find all the code at this GitHub link: https://github.com/jalajthanaki/stock_price_prediction/blob/master/Stock_Price_Prediction.ipynb.

Implementing alignment

The alignment is performed on the testing dataset. You can refer to the following code snippet:

Figure 2.30: Code snippet for alignment on the test dataset

As you can see in the preceding code snippet, we obtain a difference of 10 days adj close price using the average price of the last 5 days and the average price of the predicted upcoming 5 days in order to align the test data. Here, we also convert the date from the string into...

Here, we are going to implement the neural network-based algorithm multilayer perceptron (MLP). You can refer to the following code snippet:

Figure 2.37: Code snippet for multilayer perceptron

Here, you can see that we are using the Relu activation function, and the gradient descent solver function is ADAM. We are using a learning rate of 0.0001. You can evaluate the result by referring to the following graph:

Figure 2.38: Code snippet for generating the graph for the actual and predicted prices

This graph shows that all the data records' predicted prices follow the actual price pattern. You can say that our MLP model works well to predict the stock market prices. You can find the code at this GitHub link: https://github.com/jalajthanaki/stock_price_prediction/blob/master/Stock_Price_Prediction.ipynb.

In this chapter, you learned how to predict stock prices. We covered the different machine learning algorithms that can help us in this. We tried Random Forest Regressor, Logistic Regression, and multilayer perceptron. We found out that the multilayer perceptron works really well. I really want to discuss something beyond what we have done so far. If you are under the impression that using the sentiment analysis of news and predictive methods, we can now correctly predict the stock market price with a hundred percent accuracy, then you would be wrong. We can't predict stock prices with a hundred percent accuracy. Many communities, financial organizations, and academic researchers are working in this direction in order to make a stock market price predictive model that is highly accurate. This is an active research area.

So if you are interested in research and freelancing, then you can join some pretty cool communities. There are two communities that are quite popular. One of these...