Packt+ | Advance your knowledge in tech

You're reading from R Data Mining Implement data mining techniques through practical use cases and real-world datasets

Product type Paperback

Published in Nov 2017

Publisher Packt

ISBN-13 9781787124462

Length 442 pages

Edition 1st Edition

Languages

Tools

ggplot

Concepts

Data Mining

Author (1):

Andrea Cirillo

View More author details

Table of Contents (16) Chapters

Preface

1. Why to Choose R for Your Data Mining and Where to Start FREE CHAPTER

2. A First Primer on Data Mining Analysing Your Bank Account Data

3. The Data Mining Process - CRISP-DM Methodology

4. Keeping the House Clean – The Data Mining Architecture

5. How to Address a Data Mining Problem – Data Cleaning and Validation

6. Looking into Your Data Eyes – Exploratory Data Analysis

7. Our First Guess – a Linear Regression

8. A Gentle Introduction to Model Performance Evaluation

9. Don't Give up – Power up Your Regression Including Multiple Variables

10. A Different Outlook to Problems with Classification Models

11. The Final Clash – Random Forests and Ensemble Learning

12. Looking for the Culprit – Text Data Mining with R

13. Sharing Your Stories with Your Stakeholders through R Markdown

14. Epilogue

15. Dealing with Dates, Relative Paths and Functions

A final general warning – training versus test datasets

I know I said we were going to move on to new models estimation, but let me clarify a concept before that—the difference between the training and test datasets.

When you estimate a model, you usually have at least two different datases:

The training dataset: This is the one on which you actually estimate the model. To be clear, the one over which you apply the lm() function, or whatever algorithm you want to employ.
The testing dataset: This is a separate dataset you use to validate your model's performance. It can also be a new dataset that becomes available after you first estimate your model.

Why are there two different datasets and why do we need a separate dataset to test our model? This is because of the danger of overfitting, that is, fitting a model that is really good for the dataset it was estimated for, but underperforms when it is applied to new data.

To understand it, you can think of your high school or university tests and...