AWS big data ecosystem

Amazon's big data ecosystem has several software services that enable business insights from data. These services can be broadly classified into four major categories - Collect, Store, Analyze, and Orchestrate, as shown in the following diagram:

Figure 2.1: AWS big data ecosystem

Let's look at each category in detail.

Collect

The first step for any BI initiative is to collect data from external systems to Amazon for which AWS has the following services:

• Direct connect: With direct connect, you can establish private connectivity between AWS and your enterprise data center and provide an easy way to move data files from your applications to AWS for analysis

• Snowball: Snowball (also known as Import/Export) lets you import hundreds of terabytes of data quickly into AWS using Amazon-provided, secure appliances for secure transport

• Kinesis and Kinesis Firehose: Kinesis services enable building custom applications that process or analyze streaming data

Store

The data collected...