Open Source Summit Japan 2022

KServe ModelMesh comes with a few built-in serving runtimes to serve models in common ML frameworks. However, at times there is a need to serve models in custom runtimes, specially built to satisfy various requirements. In this talk, we will introduce ModelMesh key features, and then share hands-on experience of building a new container image with support for the desired framework. The following will be covered: - Overview of ModelMesh - Demo of loading models in existing model server - Metrics and dashboard - How to create a new custom serving runtime - Demo of running prediction using custom serving runtime