Improving performance on your AKS cluster

Sometimes you will deploy an endpoint on AKS and it doesn't perform how you'd like. Maybe it times out, maybe it's too slow, maybe an endpoint that was previously working fine suddenly gets a lot more traffic that it cannot handle. These situations happen, and you must be prepared to face them.

Thankfully, AKS deployments have a lot of additional configurations that you can take advantage of to solve these problems. This section covers some of the more common situations as follows:

• Depending on how complex your model is, how many data points you are trying to score, and the size of your VMs, AKS models can sometimes take a while to score or even timeout. In this situation, there are many things you can do.

  First, you can try increasing the size of your VM, selecting one with more RAM. Next, you can add an additional setting to your deployment configuration, scoring_timeout_ms. This setting defaults to 60000 milliseconds...