Let's get into the architecture of Kafka Connect. The following figure gives a good idea of Kafka Connect:
Kafka Connect architecture
Kafka Connect has three major models in its design:
- Connector: A Connector is configured by defining the Connector class and configuration. The Connector class is defined based on the source or target of the data, which means that it will be different for the Database source and File source. It is then followed by setting up the configuration for these classes. For example, configuration for the Database source could be the IP of the database, the username and password to connect to the database, and so on. The Connector creates a set of tasks, which is actually responsible for copying data from the source or copying data to the target. Connectors are of two types:
- Source Connector: This is responsible for ingesting...
