Maintaining metadata
There’s rarely a scenario in which Glue Data Catalog tables will be static entities defined once and never updated again. Whether your tables use partitioning and they need to be updated with new partition values, or you have a changing stream of incoming data that adds or modifies data types, you’ll want to keep updating and refining your Data Catalog entities.
Glue provides several mechanisms to do so automatically without user interaction, although any of the methods described before can be used to update tables or partitions manually. Metadata can be automatically updated using crawlers or extract, transform, load (ETL) jobs, which we will discuss in this section.
Glue crawlers
Similar to how crawlers can define tables and partitions in the Data Catalog, they can also update them. Any subsequent runs of a successfully completed crawler will update objects the crawler initially defined as per the configuration options selected. There...
