The Risks of Self-Service AI Software in Customer Service
Since its launch in November, ChatGPT has ushered in a new wild west when it comes to leveraging technological innovation in business applications. The powers of this technology are clear: Profound improvements in productivity, unlocking new creative potential, and in my industry, the opportunity to change customer experience for the better, are all now on the table.
However the rapid introduction of self-service LLM-enabled chatbot software has also opened up numerous risks and liabilities. Especially in customer-facing situations, ChatGPT and other generative AI technologies are too prone to hallucinations and customer manipulation to be put in front of any customer. Yet many self-service software providers have simply incorporated generative AI technology into their solutions like it’s just another product upgrade, and therefore most business users operate under the assumption that these technologies act like other tools they use.
This approach is dangerous and can potentially set back the development and progress of harnessing AI in a safe and effective manner. Looking again at its applications within customer service, without proper knowledge and oversight generative AI can open up brands to both potential PR disasters (through hallucinations) and security and privacy concerns (when the proper security and PII protocols aren’t put into place).
Generative AI is an unprecedented technology and requires a special type of oversight that lends itself to fully managed service solutions. There are a few reasons for this. First, few business technologies have been adopted as rapidly as generative AI. Best practices and guidelines are being developed on the fly and moving at a breakneck pace. It is unreasonable to assume that business users can keep up with the pace of innovation. (It’s tough enough for us AI experts!) Meanwhile, fully managed service providers are able to quickly learn what’s working–both from what they are seeing within their customer base and in the industry at large. Additionally, LLMs can behave inconsistently and can be very hard to control, hence the need for those that have experience in generating the right safeguards and testing infrastructure. It is therefore incumbent on the vendors providing these solutions to offer services, counsel, and ongoing oversight to ensure that their customers are able to be agile and nimble in continuing to use these tools in the most effective way possible.
Second, in the particular case of generative AI, business users are actually able to exert more control and customization through a fully managed service than they would be able to through self-service software. Historically, the main benefit of SaaS was that business users could use it without needing to rely on IT management or services, meaning the business user felt like they had more control and freedom over how to use the tool.
However, when generative AI is applied in a SaaS model, the result is actually limiting. Take, for example, generative AI-powered customer service chatbots. A lot of self-service vendors have taken the route of offering their customers general templates they can use to create customer personas, generate knowledgebase data sets, set up basic conversational flows, etc. But as we all know, generative AI is capable of so much more than that! Working through a fully managed service, business users can ensure incredibly precise customization for their unique set of customers; they can identify the exact sets of knowledgebase content and data they need to see that no customer inquiry goes unresolved, and they have access to a host of expert resources to update and refine conversational flows so that they only become more effective overtime. The end result is a wholly more powerful experience for their customers.
Finally, for most businesses, fully managed services are a much cheaper option. There are a number of essential aspects of overseeing AI tools that business units are not properly resourced for right now, such as data analysis and ensuring that the data being used by the AI is of high quality, ongoing process improvements, and identifying new use cases. Until we are at a place where every business unit within an organization has dedicated AI-specific headcount (which is an expensive investment), the most cost efficient way to make the most out of AI is through fully managed services.
Generative AI is the most profoundly transformative technology since at least the advent of social media. And while the SaaS model was revolutionary in its own way, a paradigm shifting innovation like generative AI requires a lot of specialized effort, knowledge, and testing to get consistent, safe results, which would be very difficult to replicate in a self-service manner. For the foreseeable future, buyers of generative AI solutions need to first and foremost identify providers that offer robust fully managed services if they want to be at the forefront of this powerful new technology.