Nearly every company leader in the world is thinking about the potential opportunity—and risk—of generative AI for their business and customers. To better understand what his customers are thinking, Coveo CEO Louis Têtu recently toured the world to speak with over 50 CIOs and IT leaders.
In an interview with BetaKit, Têtu shared the takeaways from his world tour, the overall growth in AI-enabled experiences, and how CIOs can prepare for the “tectonic change” of generative AI.
The evolution of digital experience with AI
Despite the current AI craze, tech companies have been leveraging machine learning (ML) in their products to improve user experience since around 2010, Têtu said. Early adopters like Amazon, Netflix, and Wayfair analyzed user actions to suggest the next action a user might want to take—for instance, a movie you might like based on a TV show you just watched on Netflix or promoting chairs if you recently purchased a dining table on Wayfair.
“It turned intent detection and the ability to personalize on its head because machine learning could understand who’s at the other end and triangulate in real-time,” said Têtu.
Coveo’s CEO noted that the significant growth in AI capabilities in the past decade have brought with it three novel outcomes. The first is a coding autopilot to accelerate development timelines and debugging. Second is what Têtu calls “creative, non-consequential applications” such as creating videos, marketing messages, or account summaries—things that can add value but don’t require the strictest level of security or efficacy.
The third outcome is the one that Têtu feels is the most important: the ability to generate advice. This will have wide-ranging applications, from customer service to employee learning, and even potentially generating whole new products or services.
“That takes you into a world where suddenly the digital experience becomes more advisory,” said Têtu.
Generative AI’s confidence is one of many new headaches for CIOs
The promise of generative AI as an advisory engine is huge for businesses of all sizes. However, making business decisions from that generated advice, like any other type, requires a traceable source of truth. For generative AI, that’s currently no small feat.
“ChatGPT lies with extreme confidence.”
– Louis Têtu,
Coveo CEO
“ChatGPT lies with extreme confidence,” Têtu said. “It’s wonderful when you read the thing and it’s semantically rich and beautiful and impressive. But it’s factually incorrect often. So you need to be factually correct. And in fact, not only that, many companies have compliance; it needs to be verifiable.”
Data traceability and accuracy is just one of the nine issues Têtu identified from his world tour discussions with over 50 CIOs of major corporations about generative AI. Another key challenge is that many generative AI tools like ChatGPT are public; by using them, you technically consent to your information being made publicly available (or at least used for further training the AI engines).
Security and permissions then become another obstacle: CIOs need to ensure platforms can securely handle relevant proprietary information for the question at hand without divulging anything else. Multiple corporations have already responded to this challenge by banning ChatGPT at work because of sensitive data leaks. But there are internal implications too. For example, an internal generative AI tool meant to help employees learn about the organization might share employee salaries or performance reviews if it’s not set up with the right sharing and permission settings.
Then comes data source connectivity. For generative AI to provide value, CIOs need to allocate the resources required to tag and connect disparate and often-siloed data sources within the organization. Even when these sources are connected, CIOs then need to consider data recency and continued accuracy at scale as data is automatically collected, in addition to consistency across all output channels (e.g. search and chat).
All of these challenges boil down to a single, significant consideration for any business: cost. If corporations want to leverage generative AI without the security or compliance risks associated with public large language models (LLMs), they have to spend a lot of money and time to build or buy a customized platform.
“The cost of generating answers is about a thousand times the cost of processing a query right now in search,” said Têtu.
When generating answers becomes a commodity, inputs and infrastructure become more valuable
As CIOs continue to think about how best to implement generative AI tools, Têtu said they have to keep in mind that people use websites, fundamentally, for one of six intents: to learn, watch, listen, fix a problem, make purchases, or connect with someone.
“Generating an answer is not necessarily the only thing that people want,” said Têtu. “People still want to discover.”
“Generating an answer is not necessarily the only thing that people want. People still want to discover.”
The next thing CIOs need to realize, Têtu added, is that answer generation is a commodity. Certain methods of generating answers might be novel to end-users and more or less expensive to produce, but generating an answer is not, by itself, valuable or rare. What makes an answer valuable is the data sources that make it accurate and actionable.
“The science and the challenge is in the data infrastructure, the security, and the unified index and relevance as well as in the prompt engineering,” Têtu said. “It is not in the LLM.”
Getting this “boring plumbing” right starts with what Têtu calls the “traditional mechanisms of relevance”: ensuring the AI can understand who is using it, what question the person is trying to answer, and what that person’s preferences and access permissions are. Then the AI, leveraging accurate and traceable data from the right sources, can deliver an answer that will genuinely add value or address a query.
As this infrastructure gets established, Têtu added a third caution: don’t waste a user’s time. He explained that generative AI will exponentially increase the amount of content available online, meaning customers and internal employees alike are at risk of getting lost in the content and not being able to fulfill their intent. This is a risk CIOs must be aware of, ensuring that any tools not only drive toward a user’s intent but also help them wade through the ocean of content that gets created every microsecond by AI.
“Digital patience is an oxymoron,” said Têtu. “Companies have to adapt to this new set of expectations that can only be delivered with AI. And now generative AI raises the bar again to experiences that are conversational and advisory in nature. And that’s why companies take this very, very seriously.”