top of page
Writer's pictureRam Bala

Responsible Enterprise AGI: Why LLMs flatter to deceive (Part One)



Imagine this: you are the CEO of an e-commerce company and just had difficult conversations with one of your customers. So, you call up your assistant who listens to you, reasons with you, and helps you come up with a solution for the customer problem. Once you approve the solution, your assistant drafts an email, and communicates with your customer to set up a follow-up discussion. Your assistant has been more valuable than anyone you have ever employed. What if this learning, thinking, synthesizing, strategizing assistant is not a human being at all, but an AI system that learns and grows smarter over time? It doesn’t quite “feel” or isn’t “self-aware” as a human being would be, but can do most tasks involving intelligence that a human can. This is the Artificial General Intelligence (AGI) future first predicted in the 1950s and written about by many scholars and science fiction authors (e.g. HAL 9000 in Arthur C. Clarke’s Space Odyssey).


What is AGI?

How do we know whether an AI system is a form of AGI? Over the past many decades, researchers have come up with many different tests or metrics with this in mind such as the Turing test (ability to act intelligently in a way that is indistinguishable from a human), the coffee test (the ability to make coffee in a home by just observing objects and actions) or the ability to transfer learning across domains.


Of all the above tests, we believe that it is definitely possible for an AI system to work across adjacent business domains in a way human beings cannot do. Humans would have to spend a substantial and prohibitive amount of time learning the “tricks of the trade” of any new business domain different from what they were originally trained on. For example, consider a human expert in an operational role in the field of logistics. Such an expert has a deep understanding of the rules, practices and decision making criteria in logistics. Asking this expert to work in hospitality would require significant motivation and extensive training in this sector. At the same time, both these sectors have underlying similarities in work patterns often obfuscated by terminology and entrenched practice. An AI system can tease out these similarities and achieve the ability to attain cross-domain understanding. Doing so requires the AI system to perform thought experiments and to apply reasoning to the outcomes of such experiments. However, the view of many AI experts (and also our view) is that such reasoning ability is beyond the grasp of current LLMs, even when they seem intelligent.


LLMs are not enough

The press has often sensationalized the reasoning capabilities of LLMs. Indeed, safety concerns over the hyper capabilities of GPT-5 are believed to be a major factor in the ousting of Sam Altman from Open AI. But the reasoning exhibited in such examples typically use concepts such as Chain of Thought Reasoning (CoT) and Zero-shot-CoT that cleverly deploy advanced prompt-engineering to elicit the well-known pattern matching capabilities that LLMs possess. But in all cases, upon careful observation, one realizes that much of the heavy-lifting is done in the prompting phase; where the LLMs are “prompted” with terms or concepts that appear frequently in the training data. Multiple studies corroborate this observation: LLMs excel at solving problems featuring recurring patterns from their training data. This affirms the hypothesis that has come to be widely accepted, LLMs cannot generate robust abstract reasoning but rather identify patterns in their training data related to the given prompts, influencing their problem-solving approach. Further, this inability to reason outside of the core training data often leads to inaccurate and potentially fabricated responses which are now called hallucinations. The possibility of hallucinations poses a fundamental “trust deficit” that will inhibit the adoption of these models by enterprises.


Responsible AGI for the Enterprise

The ability of an AI system to reason is a potential antidote to the hallucination problem and a pathway to building a trustworthy, responsible enterprise AGI implementation. However, true reasoning about business problems requires the AI system to not only extract patterns from training data, but also discover new approaches to solving problems by searching for solutions in a dynamic and complex landscape. Indeed, scholars in the field of “strategic management” have been calling out to this view of the world for a long time based on the original seminal work of Herbert Simon on bounded rationality in organizations. This process of search and discovery requires an evolutionary approach to the set of algorithms that achieve the goals of the business user of the AI system much like those found in simulated annealing, hill climbing, genetic algorithms, and particle swarm optimization.


At Samvid, we are building these autonomous intelligent systems using a combination of domain expertise embedded in Chain of Thought Reasoning (Sootras) and evolving problem solving algorithms (Neeyums) for customers who have large amounts of unstructured and fragmented data. This enables our customers to make decisions faster while consuming and analyzing oodles of structured and unstructured data. That customer problem which would have taken multiple teams several days to solve and confidently arrive at a decision, will now take fewer days and fewer people.


We have seen how AI has evolved in 2 short decades, from identifying SPAM email, beating a chess grandmaster, to self-driving cars, and now speaking to us in natural language via LLMs. While talk of creating Terminators, Cylons and preventing extinction level risks have captured popular imagination, a lot of work is still needed beyond the current generation of LLMs to create AGI. We believe that given the momentum GenAI has today, the technology will inevitably progress towards AGI over time; and before it gets there, technologists, regulators and business leaders have a responsibility to work together and create common-sense guardrails for Enterprise adoption, so that the focus remains on productivity, accuracy and enhancing human intelligence. At Samvid, we are creating one such platform and tools for businesses of all sizes.




144 views0 comments

Comments


bottom of page