File and storage migration
A lot of data still resides in files for many reasons. When the data resides in files, we just need an easy transfer mechanism to bring it over into the raw layer of the data lake in S3. In this section, we will explore some of the AWS services that make it easy to transfer files into the AWS ecosystem.
AWS DataSync
AWS DataSync makes it easy to continuously migrate on-prem data into many AWS storages, including Amazon S3. DataSync has an agent that needs to be deployed that will help do all the heavy lifting for the data migration. Before we look at the usage patterns, let’s look at a use case at GreatFin that makes DataSync very appealing.
Use case for data migration using AWS DataSync
Multiple LOBs at GreatFin want to save costs by retiring multi-terabyte data stored on their on-prem storage systems. They want to continuously replicate new data as it arrives on their on-prem storage. Also, because of regulatory requirements, they have...
