Loading and saving data from relational databases

Loading data into Spark from relational databases is very common. As the Spark-based big data lake is becoming a replacement for traditional Enterprise Data warehouses, Spark needs to connect to support for JDBC.

Getting ready

Make sure that the JDBC driver JAR is visible on the client node and all the slave nodes on which the executor will run.

How to do it...

1. Create a table named person in MySQL using the following DDL:

        CREATE TABLE 'person' ( 
          'person_id' int(11) NOT NULL AUTO_INCREMENT, 
          'first_name' varchar(30) DEFAULT NULL, 
          'last_name' varchar(30) DEFAULT NULL, 
          'gender' char(1) DEFAULT NULL, 
          'age' tinyint(4) DEFAULT NULL, 
          PRIMARY KEY ('person_id') 
        )

2. Insert some data:

        Insert into person values('Barack','Obama','M',55); 
        Insert into person values('Bill','Clinton','M',70); 
        Insert into person values('Hillary','Clinton','F',69); 
        Insert...