Large-scale data from release 3 of the 1000 Genomes project contributes to 820 GB of data. Therefore, ADAM and Spark are used to pre-process and prepare the data (that is, training, testing, and validation sets) for the MLP and K-means models in a scalable way. Sparkling water transforms the data between H2O and Spark.
Then, K-means clustering, the MLP (using H2O) are trained. For the clustering and classification analysis, the genotypic information from each sample is required using the sample ID, variation ID, and the count of the alternate alleles where the majority of variants that we used were SNPs and indels.
Now, we should know the minimum info about each tool used such as ADAM, H2O, and some background information on the algorithms such as K-means, MLP for clustering, and classifying the population groups.
