Open Source Summit Japan 2022

MLSecOps is the intersection of machine learning, DevOps, infrastructure, and security. Machine Learning models can be easily reversed, leading to invaluable loss of Data. Having a robust MLSecOps infrastructure in place is absolutely necessary. It's incredibly complex for data scientists to monitor model security since it is hard to access and process model payloads on Kubernetes in scale. Therefore, it’s crucial to automate online real-time evaluations and detailed offline analysis on Kubernetes. Real-time evaluations such as model explanations can provide immediate feedback for each model prediction, while offline analysis such as fairness and adversarial detection can examine the model security over a period of time in order to visualize and report any potential threats in the model. This talk covers how to use KServe, Knative, Apache Kafka, and Trusted-AI tools to serve ML models, persist payloads, and automate both online and offline model evaluations in a production environment.