Queues are used to collect intra-system messages and/or external events. Processing of these messages can happen sequentially or in parallel, depending on how many consumers are available to process them. Messages can also be routed to different parts of your system based on various criteria (as dictated by the queuing engine), to make sure that it is handled appropriately.
AWS provides different queues and queue-like services for different purposes:
- Managed Streaming for Kafka (MSK)
- Kinesis (data streams and firehose)
- Amazon MQ
- Amazon Simple Queue Service (SQS)
We will be discussing Kafka in this section.