(Note: This blog post originally published on Medium, on the author's feed on May 4, 2023) 

In the past month or so I have been fighting a vague sense of unease as I watch the “hype cycle” get fully revved up for ChatGPT, and all of it’s related Large Language Model (LLM) competitors. As I read more, and watch people work themselves into a fever pitch about AI in general, I am reminded of all of the hype that followed the introduction of the Internet. It’s powered by a kernel of truth, but the ultimate payoff is further down the road, and is much more difficult, than people want to believe.

Why are people excited about LLMs?

I can’t go a day without reading about how someone has ChatGPT writing marketing pitches, emails, short stories, and advertising copy. It’s a bit overwhelming, with everyone expressing some variation of, “it will completely change our industry”.

When you look at things at a surface level, it is quite exciting. An intelligent agent that can do a lot of the “boring” or repetitive tasks for you — something that will do all of the things that eat into your precious time during the work day. An agent that understands language and can communicate effectively with people, at scale. It represents a quantum leap in automation — enabling you to achieve higher levels of productivity. I get it — and the promise of the technology is certainly very attractive.

But…. there are issues here.

Accuracy is Important

The quality of the LLMs that are available out there now is somewhat questionable. Sometimes these models will fabricate answers, fabricate supporting materials, and just provide information that is wrong. AI experts will often use terms like “hallucination” or “misrepresenting” to describe these situations where the model will seemingly make up facts for itself to confidently respond back with.

Keep in mind that these systems are built by training them on broad internet sources of information. I would guess that half of the things that I read on the internet are wrong or inaccurate. So if we’re training LLMs with data that is only 50% (more or less) factual, how can we expect the results to be any more accurate than that? It’s the old “garbage in, garbage out” problem, and when you use large training sets, you cannot guarantee the quality of the training data.

Accuracy and “being right” is important here. These LLMs are being marketed as being able to provide accurate and scalable communications for a particular knowledge domain. For some reason, humans have a bit of a blind spot when it comes to computers — we always seem to default to thinking of them as being “right” or “infallible”. This fuels the common story arc in many futuristic and dystopian stories about machine intelligence taking over the world — as humans trust their computers a little bit too much, and end up being enslaved by them.

Because of the way that people interact with these systems, often just blindly trusting their accuracy, the resulting systems need to be close to 100% accurate. That’s a VERY tough goal to reach. We are asking these systems to be more accurate than our best subject matter experts.

Who Do I Trust?

A lot of the things that I read about ChatGPT (and other similar large language model technologies), talk about how it can write effective marketing materials, answer customer emails, and communicate with your customers at scale. Making all of your customers feel like they are receiving the personal touch. Except….. they really won’t.

Think about it. Decades ago, if I received a typed letter with my name in it, I thought that someone had taken the time to type up a personal letter to me. The thought that someone spent time communicating with me, made me feel special, and gave value to the interaction. Today, I get about 5 pieces of junk mail, all with my name in them, on printed materials selling me everything from life insurance to a new roof. It’s not personal — I immediately know that these are mass mailings, and they go (in true mass fashion) into the garbage. No time or effort was spent on sending them to me, so I invest no time in reading or appreciating them.

The “bar” for personal interaction has moved. I got a hand written letter from someone last week, and I almost fainted. I showed it to my wife, and we tried to remember the last time that we received anything handwritten from anyone. It was effective — because it was personal and you KNEW that the sender felt that you were important enough to spend time writing up a letter.

This use case doesn’t work — you spend money to send out communications that are even less sincere than what you already send to your customers. This doesn’t help you at all.

How Much Does This All Cost?

The other issue with things like ChatGPT are the costs of the service. The costs for ChatGPT are directly related to the costs for supplying the service. Building, training and running the ChatGPT model can run anywhere from $100k to $700k a day — depending on whom you believe. All of that cost is going to either get passed along to the consumer, or it will be absorbed as developmental cost, which makes the service less likely to survive.

Let’s be clear here — these are the economic realities of ANY software service, not just ChatGPT. However, what makes things worse for ChatGPT is the fact that it relies on a massive number of GPUs to do it’s computing (some reports put it as high as 10,000 )— that is the piece that begins to get expensive. At some point this cost will get passed along to the application that is using ChatGPT — and that cost is going to be greater than the benefit derived from using the service.

Grabbing the Easy Value

When I think about applying technology to a business, I often look for value. What technologies are proven? Which ones have been around long enough for the price to come down? What technologies have some proven and documented business models that show their overall value?

I also like to think about risk. Most of the risk in my business should be in areas where I differentiate myself, where I am a market leader. This is because these areas are where I specialize. These are the knowledge domains where I am smarter than most of the other organizations out there, and I can make more intelligent choices. If I am a bakery, I may experiment with new cakes and cookies — because that is where I specialize, and I KNOW this stuff better than my competitors. My credit card servicing company doesn’t suggest new recipes for me, and I don’t tell them how to process credit card transactions. I don’t want to experiment with a chatbot — it’s not core to my business. If chatbot technology isn’t key to your business, why make it a high risk item?

Use Simple FAQ Chatbots Where it Makes Sense

Let’s assume that you think having some AI skills is essential for your business. So you decide that you want to dive into AI. FAQ chatbots have been around for a while now, and the various different vendors have all worked out most of the bugs. They are simple, many are low-code/no-code, they do the FAQ job quite well, and they do this at a reasonable price. Many of the use cases addressed by simple chatbots are relatively static — the information doesn’t change much, and the guidance doesn’t change often. That means that maintenance is pretty simple as well. GRAB THE EASY VALUE — use this technology — it helps you and it’s cost effective.

Use What The Leaders Use — Call Center Bots

Many leading organizations have been using chatbots to replace and augment call centers. This technology is fairly well proven, and the business case behind this technology (usually less than $0.50 per call for a voice activated chatbot vs. $5 or more per call center interaction) allows you to quickly determine deflection rates, and how much money you are spending and saving. These types of chatbots can also allow your customers to self-serve solutions to their product issues, questions, and usage concerns. GRAB THE EASY VALUE — use call center chatbots to help deflect calls from your more expensive call centers.

In addition, this has the strategic value of providing a foundation for you to begin to think about the entire customer experience. Many organizations are experimenting with providing call centers with real-time customer data, and providing a customer care representative with a full view of the customer. These systems are being deployed today by some early adopters, and as this technology and the integration of customer data matures, you will be well positioned to take advantage.

Let The Professionals Handle It

A lot of the organizations that I see in the market today are outsourcing their plunge into AI. Negotiate with a vendor to develop and then maintain and operate your chatbot/assistants, letting those organizations that specialize in this sort of thing, assume all of the risk inherent in a new technology. GRAB THE EASY VALUE — and make sure that the implementation is tied to positive business results for you. That makes it a win/win proposition for both you and your technology provider.

Conclusions

ChatGPT is a great leap forward in natural language processing capability. What is being accomplished by ChatGPT, and many of the other Large Language Models (LLMs) is impressive, and should be watched with interest. But the issue is that the hype has set up expectations that the current technology is not able to meet in a cost effective manner. Accuracy matters. I suggest using it where you can show that it makes a difference that has an economic impact. Don’t fall into the trap of using ChatGPT just because, “all of the cool kids are using it”.

I like being able to say, “I KNEW that was going to be big, that’s why I was an early adopter”. However, it doesn’t help my business at all (we don’t call it “the bleeding edge” for no reason). The risks and costs associated with the “accuracy gap” in the current LLM implementations, as well as the costs of implementation, mean that this kind of technology has limited use for a modern corporation.

Instead, focus on creating and deploying AI that has been proven over the past ten years. Use AI that supports what you want to do, that meets your quality criteria, and is reasonably priced. Have a business model in mind — define how AI is going to have an economic impact on your business — and prove it out. AI is no different from any other technology that you encounter in your business. They all need to provide some tangible benefit.