Rate Limiter

Problem

Individual services in Microservice architecture have different Quality of Service (QoS) guarantees. If the processing requirements of the system exceed the capacity of the resources that are available, it will suffer from poor performance and may even fail. The system may be obliged to meet an agreed level of service, and such failure could be unacceptable.

Using auto-scaling pattern helps provision more resources with an increase in demand. This pattern not only consistently meets user demand, but also optimizes running costs. However, auto-scaling is not the optimal solution in the following scenarios:

• The backend service or data store has throttling limits which affect the limit of scale the application...