Towards Observability for ML Pipelines // Shreya Shankar // MLOps Coffee Sessions #75

MLOps Coffee Sessions #75 with Shreya Shankar, Towards Observability for ML Pipelines. // Abstract Achieving observability in ML pipelines is a mess right now. We are tracking thousands of means, percentiles, and KL divergences of features and outputs in a haphazard attempt to figure out when and how to retrain models. In this session, we break down current unsuccessful approaches and discuss the path towards effectively maintaining ML models in production. Along the way, we introduce mltrace -- a preliminary open source project striving towards “bolt-on“ observability in ML pipelines. // Bio Shreya Shankar is a computer scientist living in the Bay Area. She’s interested in building systems to operationalize machine learning workflows. Shreya’s research focus is on end-to-end observability for ML systems, particularly in the context of heterogeneous stacks of tools. Currently, Shreya is doing her Ph.D. in the RISE lab at UC Berkeley. Previously, she was t