Unveiling SnapGPT: Impact of GPT-4 on Data and App Integration

OpenAI recently introduced GPT-4, the latest version of its language model. This iteration provides significant improvement in generation and reasoning capability over the GPT-3 models. We believe GPT-4 and successor LLMs will transform the automation of data and application integration.

Large language models (LLMs) like GPT-4 can assist organizations to simplify and automate the integration of business processes, and unify data across applications into data warehouses for reporting and for descriptive and predictive analytics.

At SnapLogic, our vision is to deliver frictionless integration and automation. We have helped pioneer the incorporation of AI and ML technology to streamline the integration, automation, and orchestration of data flows across the enterprises. Our Iris AI solution, trained against over 7 petabytes of metadata, has already provided tremendous value to our customers over the last six years.  We are now actively exploring the use of GPT-4 to further enhance our platform capabilities.

What we’ve seen so far in GPT-4

At SnapLogic we have been prototyping a variety of integration and automation use cases using ChatGPT (GPT-3.5) to connect intent expressed in business language with technical implementation to realize the business goal. We wanted to see how GPT-4’s increased reasoning and longer prompt support improved our integration-based use cases.

We have been rapidly testing and comparing our previous prompts against both GPT-3.5 and GPT-4. In addition, we have been experimenting with completely new and more demanding use cases. So far we are extremely impressed with the improved accuracy of the GPT-4 responses. It is worth noting that GPT-4 is currently more expensive per token (prompt length) and slightly slower in response generation. That said, the cost and time seem justified for the improved response accuracy.

Our interest is driven by the enormous pressure that enterprises face to increase agility while reducing costs. In our work, we find that process and data fragmentation from the simultaneous proliferation of departmental SaaS apps, the growing use of public and homegrown APIs, and the explosion of heterogeneous data from departmental business systems are some of the biggest obstacles to enterprise agility.

Introducing SnapGPT

SnapLogic recently announced the release of SnapGPT, the first-ever generative AI solution for enterprise applications and the modern data stack. The new addition to the SnapLogic platform leverages AI to quickly integrate and automate business processes using natural language prompts.

With SnapGPT, IT and business users will be able to accelerate business innovation by automating and integrating data flows at speeds never seen before. Integrations that used to take weeks to deliver can now be generated to achieve rapid results in minutes. Business users simply specify their integration requirements in natural language and SnapGPT does the rest by creating all the necessary processing flows, expressions, or scripts needed to complete the task faster and more accurately than ever before.

With SnapGPT, IT and business users will be able to automate and integrate data flows and build integration pipelines with ease.

For example, a “Quote to Cash” product owner could specify their business intent in natural language to automatically create an invoice in NetSuite when an opportunity is marked “Closed Won” in Salesforce.com. SnapGPT can translate the intent into a complete pipeline with suggested filters, mapping, and transformation required to complete the task.

Use cases for AI in data integration and automation

We believe that LLMs can simplify data and application integrations in the following areas:

Extraction

SQL is the language of business data in both databases and cloud data warehouses. 65% of our customers’ pipelines include at least one SQL Snap. Authoring the queries within these Snaps can be a challenging process, requiring in-depth understanding of not only SQL but also the target database structures and nuances specific to each data store. GPT-4’s advanced NLP capabilities can simplify the formulation of SQL queries. With the ability to understand complex data structures and relationships, GPT-4 can help users generate more accurate and effective queries using natural language prompts. This can significantly reduce the time and technical expertise required to extract data from various sources.

Transformation

GPT-4 can simplify the data preparation process by automating the transformation of data from one format to another. With advanced machine learning capabilities, GPT-4 can identify patterns and relationships within the data and generate the logic required to automatically transform it into the desired format. This can significantly reduce the time and effort required for data preparation and increase the accuracy of the data.

Data pipeline creation

Using natural language prompts, GPT-4 has the ability to simplify the creation of data pipelines by guiding users through the process. By specifying their requirements using conversational language, users can minimize manual intervention, as GPT-4 generates the necessary processing flow, expressions, or scripts needed to create the pipeline. This can save organizations time and reduce errors in the pipeline creation process. Furthermore, GPT-4 can suggest the appropriate configurations for data connections, thereby eliminating the challenge of connecting to enterprise data.

Quality assurance

GPT-4 can help automate testing and quality assurance of data and app integration and orchestration processes. We have found that GPT-4 can summarize small data set samples and identify data quality issues such as data errors and conflicts. These responses can then be used to apply data cleaning transformations on a larger data set to produce high quality data for the target database or cloud data warehouse.

Troubleshooting

In case of any issues or errors in the data and app integration and orchestration processes, GPT-4 can help troubleshoot the problem. With the ability to summarize complex data structures and relationships as well as transform complicated system error messages into plain language, GPT-4 can quickly identify the root cause of the issue and suggest potential solutions. This can significantly reduce the time and effort required for troubleshooting and increase the efficiency of the system.

Insights

GPT-4’s advanced NLP capabilities can help users get insights against a subset of enterprise data by allowing them to ask questions of the data in natural language.  The ability for non-technical subject matter experts to pose questions in their terms can significantly reduce the time and effort required for data analysis and increase the accuracy of the insights.

We believe that GPT-4’s advanced NLP, machine learning, and data processing capabilities offer significant opportunities to simplify and automate data and app integration and orchestration processes.

Continuing the conversation: what’s next in GPT-4?

At SnapLogic, we are excited to see the potential of GPT-4 and look forward to exploring its capabilities further as we continue to deliver our long-standing vision of frictionless integration and automation. We are moving fast, and are confident that by collaborating with the SnapLogic community we will be able to deliver a new experience that is transformational in terms of time to value — delivering results to the business at both the top and bottom line.

Businesses wanting to accelerate their technical workflows can join the SnapGPT waitlist at https://www.snaplogic.com/snapgpt-waitlist.

 

[To share your insights with us, please write to sghosh@martechseries.com]