Chapter 12: Tuning, Debugging, and Troubleshooting

In the previous chapter, we explored some of the fundamental concepts involved in monitoring AWS Glue workloads, such as defining and monitoring service-level agreements (SLAs) of the data platform, as well as monitoring components of the data platform such as overall statistics, state changes, delay, performance, common failures, and log messages. We also explored how we can analyze usage using logs emitted by different AWS services based on the use case.

Based on the insights gained by monitoring AWS Glue workloads or downstream applications, we will be able to understand whether our workload is running optimally and whether we have over- or under-provisioned resources, and determine whether there is room for improvement. We can tune and enhance our workflows to obtain better performance and thereby save time and resources required by the components of our data integration workflow.

Upon completing this chapter, you will be...