Creating a database interface
In the previous section, we looked at how we can use a configuration manager library such as PureConfig
to read environment variables. However, the getDBParams
function is not very useful as this needs to be repeated each time we need to access the host, port, and so on for a given database. One possible option would be to update getDBParams
by passing a Database
object instead, as shown here:
package com.packt.dewithscala.chapter4 import org.apache.spark.sql.SparkSession import com.packt.dewithscala.Config import com.packt.dewithscala._ @SuppressWarnings(Array("org.wartremover.warts.OptionPartial")) object UpdatedDBParamMethod extends App {   private val session = SparkSession     .builder()     .appName("de-with-scala")     .master("local[*]")     .getOrCreate()   private val mysqlDB = Config.getDB("my_db").get...