The importance of validation in competitions
If you think about a competition carefully, you can imagine it as a huge system of experiments. Whoever can create the most systematic and efficient way to run these experiments wins.
In fact, in spite of all your theoretical knowledge, you will be in competition with the hundreds or thousands of data professionals who have more or less the same competencies as you.
In addition, they will be using exactly the same data as you and roughly the same tools for learning from the data (TensorFlow, PyTorch, Scikit-learn, and so on). Some will surely have better access to computational resources, although the availability of Kaggle Notebooks and generally decreasing cloud computing prices mean the gap is no longer so wide. Consequently, if you look at differences in knowledge, data, models, and available computers, you won’t find many discriminating factors between you and the other competitors that could explain huge performance...
