Model interoperability with ONNX
In the previous chapter, we learned how to deploy ML models as web services for online and batch scoring. However, many real-world use cases require you to embed a trained ML model directly into an application without the use of a separate scoring service. The target service is likely written in a different language than the language used for training the ML model. A common example is that a simple model trained in Python using scikit-learn needs to be embedded into a Java application.
Model interoperability gives you the flexibility to train your model with your language and framework of choice, export it to a common format, and then score it in a different language and platform using the shared format. In some cases, using a native runtime optimized for scoring on the target environment even achieves a better scoring performance than running the original model.
First, we will take a look at the ONNX initiative, consisting of the specification...