SPICE - overview and architecture

SPICE is an abbreviation for Super-fast, Parallel, In-memory, Calculation, Engine and is the brain of QuickSight. It can source data from S3, RDS databases, Redshift, RDBMS databases, and, in future, EMR, Kinesis, and DynamoDB. SPICE has a great toolset of functions that can do lightweight transformation of data in memory, thereby eliminating the need for traditional ETL (Extract Transform and Load).

The following diagram shows the position of SPICE in an overall BI solution. On the left-hand side you see the various sources that can push data to SPICE. Data from them is cached by SPICE, which uses a combination of columnar storage, in-memory technologies, machine code generation, data compression, and auto-replication for high availability. The results from SPICE serve QuickSight analysis and, in future, SPICE will integrate with AWS BI partners such as Tableau, TIBCO, Qlick, and Domo using its SQL-like interface.

Figure 3.1: SPICE architecture

For...