watsonx.ai

These days everybody in the AI world is talking about RAG: retrieval-augmented generation.

This technique minimizes the risk of hallucination by cross-referencing information with ground truth. It's a great way to get the most out of a large language model by carefully curating the prompt.

In the field of customer support a RAG system may first fetch relevant documents from the database to understand the context of a new support email, and then use a language model to create a draft response combining this information with its pre-trained knowledge. The risk of hallucination is low.

The Problem with RAG

There is a fundamental flaw with this approach: collecting enough reliable ground truth.  

The RAG approach struggles to provide reliable business insights because, in many cases, the ground truth is buried in database tables, spreadsheets, emails and share drives. Worse still, some ground truth may not exist in a digital form at all.  

Building Ground Truth with PEG

Knowledge Orchestrator overcomes this limitation by using an approach we call PEG: pre-enhanced generation.

PEG is all about building high volumes of ground truth from existing data sources. In other words, make it extremely easy for the technology to get it right in the first place, rather than spend time and resources trying to disprove it.

It's like the difference between studying for an exam over a semester, or trying to cram everything in the night before.

PEG is slow and meticulous, RAG is a flurry of activity.

Needless to say the back-end algorithms required to do this are extremely complex, but this approach makes inferencing (the expensive bit) simple, cheap and reliable, which we believe will make the technology available to a wider set of customers.

Here's a short video to illustrate the concept.  Note we still have a long way to go, but early results are extremely encouraging!

Summary

Don't get me wrong: RAG is good, but it has some big limitations. We believe PEG represents a new development path for enterprise AI.

While the technology could probably work anywhere, we chose to run on the IBM Cloud with watsonx technology because enterprise AI requires trust, security and transparency.

I'd love to hear your thoughts in the comments, or send me an email at errol.brandt@knowledge-orchestrator.com if you'd like to find out more.

#watsonx.ai