Code generation for fast inference of Deep Learning models ROOT/TMVA SOFIE (“System for Optimized Fast Inference code Emit”) is a new package introduced in this release that generates C++ functions easily invokable for the fast inference of trained neural network models. It takes ONNX model files as inputs and produces C++ header files that can be included and utilized in a “plug-and-go” style. This is a new development and it is currently still in experimental stage. SOFIE can take your trained ONNX model and generate blazingly fast C++ code from it, depending only on BLAS.
- Announcement https://root.cern/doc/v626/release-notes.html#sofie-code-generation-for-fast-inference-of-deep-learning-models
- SOFIE was created by Sitong An https://sitongan.github.io/ a Marie Curie Fellow at CERN
- Supported ONNX operators: https://github.com/root-project/root/blob/master/tmva/sofie/inc/TMVA/OperatorList.hxx
- SOFIE is part of TMVA, the ROOT Machine Learning library https://root.cern/manual/tmva/