Amazon open-sources SageMaker Neo to help developers optimize the training and deployment of machine learning models

Amazon announced last week that it’s making Amazon SageMaker Neo, a new machine learning feature in Amazon Sagemaker, available as open source. Amazon has released the code as Neo-AI project under the Apache software license. Neo-AI’s open source release will help processor vendors, device makers, and AI developers to bring and extend the latest machine learning innovations to a wide variety of hardware platforms.

“with the Neo-AI project, processor vendors can quickly integrate their custom code into the compiler..and.. enables device makers to customize the Neo-AI runtime for the particular software and hardware configuration of their devices”, states the AWS team.

Amazon SageMaker Neo was announced at AWS re:Invent 2018 as a newly added capability to Amazon SageMaker, its popular Machine learning Platform as a service. Neo-AI offers developers with a capability to train their machine learning models once and to run it anywhere in the cloud. It can deploy the machine learning models on multiple platforms by automatically optimizing TensorFlow, MXNet, PyTorch, ONNX, and XGBoost models.

Moreover, it can also convert the machine learning models into a common format to get rid of the software compatibility problems. It currently supports platforms from Intel, NVIDIA, and ARM. There’ll also be an added support for Xilinx, Cadence, and Qualcomm in the near future in Neo-AI.

Amazon states that Neo-AI is a machine learning compiler and runtime at its core, built on traditional compiler technologies like LLVM and Halide. It also uses TVM (to compile deep learning models) and Treelite (to compile decision tree models), which had started off as open source research projects at the University of Washington. Other than these, it also performs platform-specific optimizations from different contributors.

The Neo-AI project will receive contributions from several organizations such as AWS, ARM, Intel, Qualcomm, Xilinx, Cadence, and others. Also, the Neo-AI runtime is deployed currently on devices such as ADLINK, Lenovo, Leopard Imaging, Panasonic, and others. “Xilinx provides the FPGA hardware and software capabilities that accelerate machine learning inference applications in the cloud..we are pleased to support developers using Neo to optimize models for deployment on Xilinx FPGAs”, said Sudip Nag, Corporate Vice President at Xilinx.

For more information, check out the official Neo-AI GitHub repository.