AiThority Interview with Yuhong Sun, Co-Founder of Onyx

Yuhong Sun, Co-Founder of Onyx, discusses about enterprise search in the post-GenAI era and open-source AI solutions, describes the concept of “white-box AI” and talks more about AI-powered systems in this Q&A:

———

Hi Yuhong, what inspired you to launch Onyx, and how does it differentiate itself in the open-source enterprise search and GenAI space?

Chris and I started Onyx to solve a problem that we faced as engineers. A huge portion of the week would be spent answering questions from other teams so we wanted to automate that to give engineers time back. We quickly found out that this is not unique to engineering and that in fact, a ton of productivity is lost across all the departments, whether it’s customer support, sales, RnD, or any of the other teams. So we decided to make Onyx the best solution for finding information at work by using the latest developments in NLP and information retrieval.

Product-wise, we’re differentiated in our unique focus on search quality. I can call out some specific features to give a better sense of this. We’re the first ones to use AI agents to do deep research across internal documents. We’re also the first to incorporate contextual retrieval. We’ve also trained custom models in-house to handle retrieval challenges unique to enterprise search. Finally, we’re currently building knowledge graphs using LLMs which is a very new technique and that no large-scale enterprise software has productionized yet. We’re looking to be the first on that too.

We’re also the most extensible and secure option as well as a result of being open source. Teams can and have built extensions on top of Onyx that just is not possible with closed source alternatives. We’re also built to be easily self hostable with all the data processed locally and stored locally as well.

Also Read: Building Scalable AI-as-a-Service: The Architecture of Managed AI Solutions

Open-source AI solutions are gaining traction. How do you see the balance between openness and enterprise-grade security evolving in the coming years?

We believe that every modern team will be adopting GenAI in the next 5 years, and more specifically, GenAI that is enhanced with the unique knowledge of the team. Being open source lets our users have complete trust that their internal knowledge is safe with Onyx. This is extremely important in our space because we’re asking folks to connect up a lot of sensitive knowledge like sales calls, eng docs, internal discussions, etc. But the alternative to just sit on their hands as GenAI is empowering all the other teams in the industry which is not really a viable option either. So we feel that open source will be increasingly more prevalent in our space.

What, according to you, is evolving in the enterprise search post-GenAI era? What role does Onyx play in shaping that future?

GenAI is a paradigm shift for enterprise search. The fundamental technology behind the search has completely changed. Before our era, the technology was based on indexing document word counts, page rank, and a bit of ML sprinkled in. Now the documents are processed with LLMs to capture way more information and context. LLMs can be used to create better knowledge graphs or provide additional context when generating vector representations of documents. Without getting too technical, basically what I’m trying to say is that the new approaches are dramatically improving the quality of the search results and the user experience, and we’re looking to be the most cutting edge when it comes to giving the most reliable answers.

The user experience is changing, too. Instead of having to go through search results manually, the LLM can easily read the top 50 documents and just point the user to exactly what they want. This makes Onyx much more intuitive to us,e and that alone makes a big difference.

The concept of “white-box AI” is gaining importance in the enterprise world. How does Onyx’s fully transparent model benefit businesses compared to black-box solutions?

This year and moving forward, teams are looking into AI agents. To build the best agent experience, a lot of teams need the flexibility of an open-source model. It’s always a balance. Small teams might be ok with “no code”, “drag and drop” workflow builders, but the enterprise teams out there often need a ton more flexibility that just isn’t feasible with the simple UIs.

Being whitebox/open source means that our users can build and customize every part of the solution. Even for the teams building on top of the APIs, the added visibility makes the development much easier. We hear this repeatedly from teams like Ramp, who have built agents on top of our APIs to achieve a 93% auto-resolution rate on inbound customer requests.

In an era where AI biases and security risks are under scrutiny, how does Onyx’s open architecture help enterprises maintain control and trust?

Security is one of the biggest benefits of Onyx. We’re working with defense contractors like Thales, cybersecurity teams like Bitwarden, and fintech companies like Ramp/Brex. They know exactly how their data is being processed and stored, and many teams have actually audited the code as well. again, the benefits of open source. It also makes it possible to use the cloud provider and deployment stack of their choice. Some teams deploy it on AWS EKS, some on ECS, some on a single instance, and some use Onyx Cloud.

Also Read: Edge Computing vs. Cloud AI: Striking the Right Balance for Enterprise AI Workloads

With enterprises rapidly adopting AI assistants, what are the key considerations when building and deploying AI-powered systems at scale?

We’ve designed the architecture of Onyx to be horizontally scalable. We’ve tested scales of up to a hundred million documents and actually have live deployments that have approximately 50,000 users.

The technical aspects of scaling are just engineering best practices. I would say the challenge for AI systems in particular is the quality of the experience at scale. For example, older methods that rely strongly on pagerank and links between documents would definitely struggle for the long tail of documents that aren’t linked to actively, and these documents just become harder to surface when the scale increases. A lot of this is solved by replacing the old-school heuristics with newer LLM-based approaches. And of course, AI agents built on top of poor knowledge retrieval will inevitably make wrong decisions. So search quality as scale is paramount.

What’s your take on the future of AI-driven search—will proprietary models dominate, or will open-source solutions become the industry standard?

The space is moving very quickly, and predictions are hard to make. With Onyx, users can plug in any LLM, so regardless of what the future looks like with the foundation models, we’ll be able to provide the best experience to our users.

If I had to make a guess, and this is just my personal take, I think open-source models will become very similar in functionality to proprietary models. The models are pretty easy to dissect and reproduce so it feels hard for the research labs to stay ahead of each other in meaningful ways.

[To share your insights with us as part of editorial or sponsored content, please write to psen@itechseries.com]