Understanding Metis
Metis is one of the variants of BO that has several algorithm modifications compared to the BO method in general. Metis utilizes GP and GMM in its algorithm. GP is used as the surrogate model and outliers detector, while GMM is used as part of the acquisition function, similar to TPE.
What makes Metis different from other BO methods, in general, is that it can balance exploration and exploitation more data-efficiently than the EI acquisition function. It can also handle noise in the data that doesn’t follow the Gaussian distribution, and this is the case most of the time. Unlike most of the methods that perform random sampling to initialize the set of hyperparameters and cross-validation score, D, Metis utilizes Latin Hypercube Sampling (LHS), which is a stratified sampling procedure based on the equal interval of each hyperparameter. This sampling method is believed to be more data-efficient compared to random sampling to achieve the same exploration...
