Why harnessing the power of generative AI begins with search

During the past year and a half, generative AI (GAI) has embedded itself in the fabric of everyday life. Instantly disruptive, ChatGPT surpassed more than 100 million users within months of launching . Since then, GAI features have steadily appeared in familiar tools for editing images or summarizing and writing documents. With the initial buzz and early experiments behind us, business leaders are looking for ways to harness generative AI across their entire organizations, rather than as a plug-in for existing software. They’re seeking areas where tools such as large language models (LLMs) can not only add value today but eventually transform how their companies operate. The big question on everyone’s minds: When and how will generative AI transcend being a mere feature or chatbot?



The answer is hiding on the other side of the screen, deep in the software and IT systems underpinning modern organizations. For example, Microsoft’s code completion tool GitHub Copilot predates ChatGPT by more than a year, and programmers consistently report writing faster, cleaner code because of it . Now Cognition AI aims to transform the field again with “Devin,” an agent it describes as “the world’s first fully autonomous AI software engineer.” IT professionals are watching the AI revolution unfold in areas such as cybersecurity, system resiliency, and information discovery. Their firsthand experiences highlight what’s possible today, while illuminating the underlying capabilities needed given where the tech is going.



For [GAI] to work, you need a search engine to bridge the gap between public domain information and internal private data.” Matt Riley, Elastic



To this end, Elastic , the leading Search AI company, commissioned a survey of more than 3,000 IT professionals around the world asking how generative AI could drive change in their organizations . Half of all respondents (50%) saw external opportunities for improving customer experience and engagement, and more than half (57%) saw internal possibilities for increasing operational efficiencies and individual productivity. In both cases, value is derived from a paradigm shift away from simple search results, alerts, and notifications toward receiving exact answers to a problem . For generative AI to serve up answers from data it wasn’t directly trained on, it needs contextual aid. And the key to the best contextual aid? It begins with search.



THE RIGHT DATA AT THE RIGHT TIME



Foundation models such as GPT-4, Llama 2, and Gemini are typically trained on vast training sets of data culled from the open Web. Their ability to learn new information is limited by their context window, which can increasingly range in size from short documents to large codebases or multiple novels. This puts the impetus on organizations to bring together the right data at the right time for generative AI systems to generate context-specific answers. “For generative AI to work, you really need a search engine to bridge the gap between information in the public domain and internal private data that’s changing very rapidly,” says Matt Riley, GVP and general manager of search at Elastic.



For example, asking a generative AI tool to assist you with a customer calling for support might require rapidly pulling your organization’s proprietary data—the customer’s order history, call logs, and other relevant information—into its context window for analysis. Robust search is essential for maximizing the accuracy of answers generated from that proprietary data. With those answers at hand, generative AI helps provide personalized experiences at scale, transforming customer support and similar interactions from a page of search results to a one-to-one conversation.



This is just as true for IT managers and other leaders within the firm striving to understand how operational performance issues might impact business performance. By combining data for both measures within the context window, users can save precious time and resources by asking generative AI directly rather than hunting for correlations themselves. “You could ask it, ‘Over the past five hours, has something anomalous happened?’ ” explains Abhishek Singh, general manager of observability at Elastic. “Maybe it replies, ‘Here’s a set of logs where I see problems.’ Now you can ask it which logs have had an impact on the business—without having to know how to connect that data.”



IMMEDIATE BENEFITS FOR CYBERSECURITY



Not only do these feedback loops help teams become more productive by identifying problems faster and shortening the time spent searching for answers, but they also train the AI over time to connect operational data with business performance data, thus continuously improving its results. Rather than issue alerts and notifications for users to resolve, generative AI tools will learn from and eventually write their own scripts to automatically detect, diagnose, and remediate problems before they escalate to human attention.



Such capabilities will be especially important when it comes to cybersecurity, where the number and severity of threats mount daily while many organizations struggle to fill critical positions. More than half (53%) of organizations surveyed by Elastic anticipate using AI to improve automatic threat detection. “Using generative AI to help with that is already a huge boost,” says James Spiteri, director of product management for security at Elastic. “It’s a very time-consuming process requiring a certain level of skills,” ones better suited to thwarting attacks than simply keeping a lookout.



Generative AI tools have proven adept at both identifying threats and spotting correlations between intrusions and making it easier for analysts to understand, respond to, and document incidents. This, in turn, frees up senior staff while helping to address the cybersecurity skills gap by making complex tasks more accessible to junior analysts. In this regard, generative AI is delivering immediate value by bolstering defenses that would otherwise go unmanned.



Tying all these threads together—key among them, leveraging proprietary data through search—is generative AI’s ability to reduce complexity while learning as it grows, adding new capabilities and functions that will help organizations respond to threats and disruptions faster, more efficiently, and more effectively. That’s why 88% of those surveyed by Elastic report they plan to increase their budgets for AI during the next three years. (And they’re not just building chatbots.)







Learn how you can  integrate generative AI into your organization .