ONNX was established in December 2017 as an open source format for machine learning models (deep learning and traditional ML). Backed by support from over 20 industry leading companies including Microsoft, Facebook, Amazon, Intel, NVIDIA, and more, ONNX provides data scientists with the choice to select the right tools for their task and offers software and hardware developers a common standard to build optimizations on.
Henry Zeng, Klein Hu, and Emma Ning discuss the scenarios that ONNX enables with a technical overview of the format itself. You can obtain an ONNX model in several ways, including selecting popular pretrained models from the ONNX Model Zoo, exporting or converting an existing model trained on another framework (including PyTorch/Caffe2, CNTK, Keras, sci-kit learn, TensorFlow, Chainer, and more), or training a new model using services such as Azure Machine Learning or Azure Custom Vision Service. Henry, Klein, and Emma demystify the process and show several examples of how this can be done easily.
The ONNX model can then be operationalized using an inference runtime such as ONNX Runtime on a variety of hardware endpoints. Hardware companies are plugging in accelerators to provide maximum efficiency in latency and resource utilization on cloud and edge. Henry, Klein, and Emma discuss how Intel, NVidia, and others are participating and the performance gains they’re seeing on the own models at Microsoft.
Henry Zeng is a principal program manager on the AI platform team at Microsoft, where he works with the engineering team, partners, and customers to ensure AzureML is the best ML platform in the cloud. He’s been in the AI and data area for more than 14 years in areas such as database, big data, machine learning, and deep learning. Previously, he was the lead AI solution architect at Microsoft China, where he worked with partners and customers to land AI solutions in manufactory, retail, finance, education, and public service. Henry holds an MS in computer science from Wuhan University.
Klein Hu is the senior software engineer on the Microsoft Azure machine learning team, focusing on the AI model inferencing area, especially ONNX model operationalization and acceleration with ONNX Runtime. Klein holds an MS in computer science from Beijing Normal University.
Emma Ning is a senior Product manager in AI Framework team under Microsoft Cloud + AI Group, focusing on AI model operationalization and acceleration with ONNX/ONNX Runtime in support of Microsoft’s strategic investment for open and interoperable AI. She has more than five years of product experience in search engine taking advantage of machine learning techniques and spent more than three years exploring AI adoption among various businesses. She is passionate about bringing AI solutions to solve business problems as well as enhance product experience.
©2019, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org