Decision trees are a family of non-parametric supervised learning methods. In the decision tree algorithm, we start with the complete dataset and split it into two partitions based on a simple rule. The splitting continues until a specified criterion is met. The nodes at which the split is made are called interior nodes and the final endpoints are called terminal or leaf nodes.

As an example, let us look at the following tree:

Here, we are assuming that the exoplanet data has only two properties: flux.1 and flux.2. First, we make a decision if flux.1 > 400 and then divide the data into two partitions. Then we divide the data again based on flux.2 feature, and that division decides whether the planet is an exoplanet or not. How did we decide that condition flux.1 > 400? We did not. This was just to demonstrate a decision...

Decision trees are prone to overfitting training data and suffer from high variance, thus, providing poor predictions from new unseen data. However, using an ensemble of decision trees helps alleviate the shortcoming of using a single decision tree model. In an ensemble, many weak learners come together to create a strong learner.

Among the many ways that we can combine decision trees to make ensembles, the two methods that have been popular due to their performance for predictive modeling are:

• Gradient boosting (also known as gradient tree boosting)
• Random decision trees (also known as random forests)

In this section let us explore briefly two kinds of ensemble methods for decision trees: random forests and gradient boosting.

Random forests is a technique where you construct multiple trees, and then use those trees to learn the classification and regression models, but the results are aggregated from the trees to produce a final result.

Random forests are an ensemble of random, uncorrelated, and fully-grown decision trees. The decision trees used in the random forest model are fully grown, thus, having low bias and high variance. The trees are uncorrelated in nature, which results in a maximum decrease in the variance. By uncorrelated...

In this chapter, we shall use the gradient boosted trees and random forest implementation as pre-made estimators in TensorFlow from the Google TensorFlow team. Let us learn the details of their implementation in the upcoming sections.

TensorForest is a highly scalable implementation of random forests built by combining a variety of online HoeffdingTree algorithms with the extremely randomized approach.

For the project explained in this chapter, we use the Kepler labeled time series data from Kaggle: https://www.kaggle.com/keplersmachines/kepler-labelled-time-series-data/home. This dataset is derived mainly from the Campaign 3 observations of the mission by NASA's Kepler space telescope.

In the dataset, column 1 values are the labels and columns 2 to 3198 values are the flux values over time. The training set has 5087 data points, 37 confirmed exoplanets, and 5050 non-exoplanet stars. The test set has 570 data points, 5 confirmed exoplanets, and 565 non-exoplanet stars.

We will carry out the following steps to download, and then preprocess our data to create the train and test datasets: 

1. Download the dataset using the Kaggle API. The following code will be used for the same:

armando@librenix:~/datasets/kaggle...

In this section, we shall build the gradient boosted trees model for detecting exoplanets using the Kepler dataset. Let us follow these steps in the Jupyter Notebook to build and train the exoplanet finder model:

1. We will save the names of all the features in a vector with the following code:

numeric_column_headers = x_train.columns.values.tolist()

2. We will then bucketize the feature columns into two buckets around the mean since the TFBT estimator only takes bucketed features with the following code:

bc_fn = tf.feature_column.bucketized_column
nc_fn = tf.feature_column.numeric_column
bucketized_features = [bc_fn(source_column=nc_fn(key=column),
                             boundaries=[x_train[column].mean()])
                       for column in numeric_column_headers]

3. Since we only have numeric bucketized features and no...

In this chapter, we learned what a decision tree is and two broad classes of creating ensembles from the decision trees. The ensembles we took a look at were random forests and gradient boosting trees.

We also learned about the Kepler dataset from Kaggle competitions. We used the Kepler dataset to build an exoplanet detection model using TensorFlow's prebuilt estimator for gradient boosting trees known as the BoostedTreesClassifier. The BoostedTreesClassifier estimator is part of the machine learning toolkit recently released by the TensorFlow team. As for now, the TensorFlow team is working on releasing prebuilt estimators based on support vector machine (SVM) and extreme random forests as part of the tf.estimators API.

In the next chapter, we shall learn how to use TensorFlow in the browser using the TensorFlow.js API for sentiment analysis.