The Storm isolation scheduler
The Storm isolation scheduler was released in Storm Version 0.8.2. This was a very handy feature that is very actively being used ever since its release, in the case of the shared Storm cluster. Let's understand its working and capability through an example; say, we have a four supervisor node Storm cluster with four slots each, so in total I have 16 slots. Now I want to employ three Storm topologies here, say, Topo1, Topo2, and Topo3; each has four workers allocated to it.
So by probable default, the scheduling behavior of the Storm distribution will be as follows:
|
Supervisor 1 |
Supervisor 2 |
Supervisor 3 |
Supervisor 4 | |
|---|---|---|---|---|
|
Topo1 |
Worker 1 |
Worker 2 |
Worker 3 |
Worker 4 |
|
Topo2 |
Worker 2 |
Worker 1 |
Worker 1 |
Worker 1 |
|
Topo3 |
Worker 3 |
Worker 3 |
Worker 2 |
Worker 2 |
Storm will respect load distribution and will spawn one worker of each topology on each node.
Now let's tweak the scenario a bit and introduce a requirement that Topo1 is a very resource-intensive topology...
