Snowflake Extends Programmability for Developers with New Snowpark Container Services to Run Secure Generative AI

• Snowpark Container Services expands Snowflake’s compute infrastructure to run a variety of workloads, including full-stack applications, the hosting of LLMs, robust model training, and more securely within Snowflake

• Snowflake has partnered with NVIDIA, Alteryx, Astronomer, Dataiku, Hex, SAS, and more to give customers secure, easy, and governed access to an expansive lineup of products and solutions within their Snowflake account using Snowpark Container Services

• Snowpark expands support for more efficient machine learning development and execution

Snowflake , the Data Cloud company,  announced at its annual user conference, Snowflake Summit 2023, new innovations that extend data programmability for data scientists, data engineers, and application developers so they can build faster and more efficiently in the Data Cloud. With the launch of Snowpark Container Services (private preview), Snowflake is expanding the scope of Snowpark so developers can unlock broader infrastructure options such as accelerated computing with NVIDIA GPUs and AI software to run more workloads within Snowflake’s secure and governed platform without complexity, including a wider range of AI and machine learning (ML) models, APIs, internally-developed applications, and more. Using Snowpark Container Services, Snowflake customers also get access to an expansive catalog of third-party software and apps including large language models (LLMs), Notebooks, MLOps tools, and more within their account. In addition, Snowflake is simplifying and scaling how users develop, operationalize, and consume ML models, unveiling new innovations so more organizations can bring their data and ML models to life. These advancements include a set of new Snowpark ML APIs for more efficient model development (public preview), a Snowpark Model Registry (private preview) for scalable MLOps, Streamlit in Snowflake (public preview soon) to turn models into interactive apps, and advanced streaming capabilities.

AiThority Interview Insights: How to Get Started with Prompt Engineering in Generative AI Projects

“Our continued investments in Snowpark, alongside our machine learning and streaming capabilities accelerate how users put their data to work, unlocking new ways to drive impact across their organizations with increased flexibility.”

“Snowflake’s product advancements are revolutionizing how customers build in the Data Cloud, enabling data scientists, data engineers, and application developers with extended programmability and a wide range of use cases so they can build, test, and deploy anything they can dream up, without tradeoffs,” said Christian Kleinerman, SVP of Product, Snowflake. “Our continued investments in Snowpark, alongside our machine learning and streaming capabilities accelerate how users put their data to work, unlocking new ways to drive impact across their organizations with increased flexibility.”

Snowpark Empowers Developers with Broader Programmability, Without Governance or Security Tradeoffs

Snowpark continues to serve as Snowflake’s secure deployment and processing of non-SQL code with various runtimes and libraries — expanding who can build and what gets built in the Data Cloud. It lets builders work with data more effectively in their programming languages and tools of choice, while providing organizations with the automation, governance, and security guarantees missing in legacy data lakes and big data environments. Since launching in June 2021, Snowpark has helped data engineers migrate pipelines and run them faster and more efficiently, enabled data scientists to build and train models, and unlocked Snowflake as a powerful platform for application development.

Snowpark Container Services further expands the scope of workloads that can be brought to customers’ data. It provides users with the flexibility to build in any programming language and deploy on broader infrastructure choices, including the NVIDIA AI platform for optimized acceleration, with the same ease of use, scalability, and unified governance of the Snowflake Data Cloud. In addition, Snowpark Container Services can be used as part of a Snowflake Native App (public preview on AWS), enabling developers to distribute sophisticated apps that run entirely in their end-customer’s Snowflake account. Snowpark Container Services will also enable users to securely run leading third-party generative model providers like Reka directly within their Snowflake account, removing the need to expose proprietary data to accelerate innovation.

Snowflake has partnered with dozens of third-party software and application providers to deliver world-class products that can run within their end-customer’s Snowflake account using Snowpark Container Services. For example, customers can run Hex’s industry-leading Notebooks for analytics and data science, use popular AI platforms and ML features from Alteryx, Dataiku, and SAS to run more advanced AI and ML processing, and manage these data workflows with Astronomer’s platform powered by Apache Airflow — all entirely within Snowflake. These are just a few examples, with AI21 Labs, Amplitude, CARTO, H2O.ai, Kumo AI, Pinecone, RelationalAI, Weights & Biases, and more also delivering their products and services with Snowpark Container Services.

Furthermore, NVIDIA and Snowflake are building transformative accelerated computing and software integrations for Snowpark Container Services. Yesterday, the companies announced a partnership that aims to make advanced generative AI capabilities available to enterprises everywhere.

The collaboration also brings NVIDIA AI Enterprise — the software pillar of the NVIDIA AI platform — to Snowpark Container Services, along with support for NVIDIA accelerated computing. NVIDIA AI Enterprise includes over 100 frameworks, pretrained models, and development tools including PyTorch for training, NVIDIA RAPIDS for data science, and NVIDIA Triton Inference Server for production AI deployments.

“Data is the foundation for custom generative AI applications built with the unique business and brand requirements of companies in every industry,” said Manuvir Das, Vice President, Enterprise Computing, NVIDIA. “The Snowpark Container Service and NVIDIA AI Enterprise integration brings NVIDIA’s full suite of AI frameworks, pretrained models, and development tools to the data platform used by thousands of companies worldwide to support today’s most advanced workloads.”

Read More about AiThority Interview: AiThority Interview with Brigette McInnis-Day, Chief People Officer at UiPath

Snowflake Helps Bring ML Models to Life, Delivers Improved Developer Experiences, and Expands Streaming Capabilities

To streamline and scale machine learning model operations (MLOps), Snowflake is announcing the new Snowpark Model Registry, a unified repository for organizations’ ML models. The registry enables users to centralize the publishing and discovery of models, further streamlining collaboration across data scientists and ML engineers to seamlessly deploy models into production.

Snowflake is also advancing its integration of Streamlit in Snowflake, empowering data scientists and other Python developers to increase the impact of their work by building apps that bridge the gap between data and business action. With Streamlit in Snowflake, builders can use familiar Python code to develop their apps, transforming an idea into an enterprise-ready app with just a few lines of code, and then quickly deploy and share these apps securely in the Data Cloud.

In addition, Snowflake is making development within its unified platform easier and more familiar through new capabilities including native Git integration (private preview) to support seamless CI/CD workflows, and native Command Line Interface (CLI) (private preview) for optimized development and testing within Snowflake. New innovations also make it easier and more cost effective for data engineers to work with low latency data, without having to stitch together solutions or build additional data pipelines. Snowflake is eliminating boundaries between batch and streaming pipelines with Snowpipe Streaming (general availability soon) and Dynamic Tables (public preview), delivering a simplified and cost effective solution for data engineers to ingest streaming data and easily build complex declarative pipelines.

 Latest AiThority Interview Insights : AiThority Interview with Abhay Parasnis, Founder and CEO at Typeface

 [To share your insights with us, please write to sghosh@martechseries.com]