Lightrun Launches Support for Python, Giving Developers a Simpler Way to Debug Live Machine Learning Pipelines

By AIT News Desk On Aug 8, 2021

IDE-native observability platform now supports Python language, Apache Spark and other popular Big Data frameworks, all natively within the PyCharm IDE

Lightrun, the leader in IDE-native observability, announced support for the Python programming language and its ecosystem of deep learning and data science libraries. With support for Python (the second most popular programming language, according to analyst firm Redmonk’s 2021 Programming Language Rankings), Lightrun brings “shift-left” observability to the algorithmically complex data science realm, where troubleshooting production code is essential to troubleshooting machine learning pipelines.

SysAdmin Appreciation Day: Top Industry Leaders Share their Insights on IT and Data Ops

“Discovering the root cause of failures in a machine learning pipeline is challenging because problems can come from many different sources, including bugs in the code, input data, and improper parameter settings”

With the release, Lightrun is supporting the Python language and the PyCharm IDE, allowing Python developers to debug many of the most popular frameworks in artificial intelligence and machine learning, including: Apache Airflow, Luigi DAGs, PyTorch / Tensorflow models, and PySpark jobs. Where traditional debugging solutions focus on debugging logical bugs in a single process application — Lightrun extends shift-left observability to Python developers whose machine learning pipelines require computation across multiple processes and machines, and where input data is always changing in unforeseen ways.

“Discovering the root cause of failures in a machine learning pipeline is challenging because problems can come from many different sources, including bugs in the code, input data, and improper parameter settings,” said Leonid Blouvshtein, co-founder and CTO at Lightrun. “With Lightrun’s support for Python and related frameworks, now developers can have live probes into their algorithms and models and observe what’s happening in production code, all without leaving the IDE. This is a dramatically more efficient way to evolve machine learning pipelines over previous approaches.”