Building the solution

This section outlines the construction of a proof-of-concept application that will implement the functionality required for this application. The details of data warehousing, OLAP, and data security each would require large, detailed books in and of themselves. At a minimum, we would suggest you consider studying the following topics:

• Partitioning strategies for data storage and loading

• Compression strategies

• Transparent data encryption

• Data access auditing for PID

• Multi-dimensional data structures

• Encryption strategies for data in motion

• SAN and physical infrastructure design

From a very high level (and shown below), we will be taking data already encrypted on the source system using agreed upon encryption algorithms, decrypting it, debatching it, and loading it into our database, or, as an alternative, loading it into a staging table then debatching it using SQL:

Encryption

We are dealing with people's credit card numbers so we must encrypt this data, particularly if we...