Discussion – Building flexible applications in AWS

Before moving on to AWS applications used to help automate ETL development, we wanted to pause and discuss how to use AWS. S3 and EC2 instances are two core services offered by AWS that are frequently used together to build scalable and flexible applications in the cloud.

Leveraging S3 and EC2

Together, S3 and EC2 can be used in tandem to create a powerful and flexible platform for instantiating an application that is easily and reliably scalable in the cloud. S3 can be used as a storage backend for EC2 instances, where data can be stored and retrieved using S3 APIs, HTTP, or a CLI. By using S3 as the data source or target for ETL workflows, data can be ingested, processed, and stored in intermittent locations within different S3 buckets (such as staging and archive buckets). EC2 instances can then access the transformed output data from the S3 data directly over your network, without the need to copy or move data across...