Frontline Systems Releases Analytic Solver V2023 with Patent-Pending Risk Analysis for Machine Learning Models

Frontline Systems is shipping Analytic Solver V2023, a new version of its advanced analytics toolset for Excel (Web, Windows, and Macintosh), that enables business analysts to easily build models using business rules, machine learning, mathematical optimization, and Monte Carlo simulation, and easily deploy those models in cloud-based applications.

Analytic Solver is not new – it’s a market-leading analytics tool upward compatible from the Solver in Excel, which Frontline originally developed for Microsoft. But now it’s the first and only tool with fully automated methods for risk analysis of previously trained and validated machine learning (ML) models.

As an “Excel Solver upgrade”, Analytic Solver can handle virtually any type or size of optimization problem, ranging from a few to millions of inter-related decisions in a single model. And for years, Analytic Solver has offered powerful features for risk analysis using Monte Carlo simulation, and powerful features for training, validating, and deploying predictive models using machine learning.

Now, Analytic Solver V2023 includes an innovative capability for risk analysis of machine learning models that leverages multiple capabilities of the software. Risk analysis changes the focus from how accurately a ML model will predict a single new case, to how it will perform in aggregate over thousands or millions of new cases, what the business consequences will be, and the (quantified) risk that this will be different than expected from the ML model’s training and validation.

“With a patent application now on file to preserve invention rights, Analytic Solver users are the first to benefit from these innovative methods”, said Daniel Fylstra, Frontline’s President and CEO. Frontline Systems is concurrently releasing new versions of RASON®, its cloud platform for analytics, and Solver SDK, its toolkit for software developers, with support for the same innovative methods.

Recommended AI News: Pega Launches Pega Process Extender for Salesforce Lightning on Salesforce App Exchange and Pega Marketplace

How and Why Machine Learning has Lacked Risk Analysis

For a decade, data science and machine learning (DSML) tools – including Analytic Solver – have offered facilities for ‘training’ a model on one set of data, ‘validating’ its performance on another set of data, and ‘testing’ it versus other ML models on a third set of data. But this is not risk analysis: based on known data, it doesn’t assess the risk that the ML model will perform differently on new data when put into production use. While it’s common to assess a ML model’s performance in use, and move to re-train the model if its performance is unexpectedly poor, by that time those risks have occurred, often with adverse financial consequences. Quantification of such risks “ahead of time” has been missing in practice.

There are many reasons for this state of affairs: Data scientists with expertise in ML methods often are not trained in risk analysis; they think of “features” and even predicted output values as data, not as “random variables” with sampled instances. Even if known, conventional risk analysis methods are expensive and time-consuming to apply to machine learning: ML data sets include many (sometimes hundreds) of features, with limited “provenance” of the data’s origins. There are hundreds of classical probability distributions that could be ‘candidates’ to fit each feature. Only some of the features are typically found, after ML model training, to have predictive value; many are found to be correlated with other features and hence ‘redundant’. And in typical projects, a great many ML models are built.

How Analytic Solver Performs Automated Risk Analysis

Unlike most other DSML software, Analytic Solver includes powerful algorithms for risk analysis in the same package: Probability distribution fitting, correlation fitting, stratified sample generation, and Monte Carlo analysis. But asking business analysts – let alone data scientists – to “quickly master risk analysis” is asking too much. So Frontline Systems has invented ways to automate the entire risk analysis process. Using the new capability is as simple as checking one extra box in a dialog, with some further “point and click” options for analysis and reporting – and the risk analysis typically adds just seconds to a minute to the existing process of training, validating, and testing a ML model.

Behind the scenes, for each feature, Analytic Solver tests an entire family of probability distributions – drawing on its first-mover support for the new Metalog family of distributions, created by Dr. Tom Keelin; optimizes all the parameters of each distribution; detects and models correlations among features, using rank order and copula methods; performs synthetic data generation, using Monte Carlo methods for stratified sampling and correlation; computes the ML model’s predictions, as well as user-specified financial consequences, for each simulated case; and importantly, assesses and quantifies the differences in performance of the ML model on this simulated data versus the training, validation and test data.

The user sees results of the risk analysis in automatically-generated charts, statistics, and risk measures – drawing from the Monte Carlo simulation features of Analytic Solver, proven in use over many years.

Recommended AI News: Tyson & Blake makes Follow-on Investment in StrongRoom AI “SRAI”

Synthetic Data Generation as a Side Benefit

Synthetic Data Generation (SDG) has become topical in machine learning in recent years, with a number of companies founded just to supply software and services around this technology. SDG is used when there isn’t enough original data, or when use of the original data is restricted by law or regulation. But until now (in a patent and literature search), SDG has simply been used to better train ML models.

Analytic Solver V2023 includes a powerful, general-purpose, easy to use Synthetic Data Generation facility, accessible from the Excel Ribbon. Unlike some special-purpose SDG offerings, this facility can accurately model the behavior of nearly any combination of features with continuous values. But Analytic Solver also uses synthetic data in an entirely new way, to analyze the risk that a ML model will yield unexpected results “large enough to matter” when deployed for production use.

Works with Already-Available ‘Augmented Machine Learning’

Analytic Solver’s V2021.5 release featured “augmented machine learning” features found only in other sophisticated machine learning tools. The user simply supplies data, and selects a menu option “Find Best Model”: Analytic Solver will automatically test and fit parameters for multiple types of machine learning models – classification and regression trees, neural networks, linear and logistic regression, discriminant analysis, naïve Bayes, k-nearest neighbors and more – then validate and compare them according to user-chosen criteria, and deliver the model that best fits the data.

Analytic Solver V2021.5 also featured enhancements that enable multi-stage “data science workflows” including machine learning models built and tested in Excel, to be deployed automatically to RASON® Decision Services, Frontline Systems’ comprehensive cloud platform for decision intelligence.

But now that it’s possible and even easy, users will want to assess the risk of a ML model before it is deployed. With a few mouse clicks, Analytic Solver V2023’s automated risk analysis can be applied to the model delivered by “Find Best Model”. When the analyst or decision-maker is satisfied, with a few more mouse clicks, the model can be deployed as a cloud service with an easy-to-use REST API.

Recommended AI News: Datanomix Announces Automated Downtime Insights

[To share your insights with us, please write to sghosh@martechseries.com]