Automating Your Business

 View Only

Poor Multi-Lingual Support Still Limits the Spread of Chatbots

By Andrej Kovacevic posted Mon May 03, 2021 04:22 AM


A few years back, experts predicted that chatbots would become a widespread all-purpose customer service tool in every industry. They have since become widespread. But all-purpose? Not so much.

As it turned out, the AI and NLP technology that powered early chatbots weren't as mature as developers claimed. And that led to countless deployment problems and usage limitations that stifled adoption. In the end, many of the companies that led the adoption wave ended up relegating their chatbot systems to customer service triage duty and other low-priority tasks.

And that's where things stood early in 2020 when the coronavirus pandemic changed everything. All at once, businesses started to reevaluate some of their idled automation initiatives. And chatbots took on new importance helping to handle increased customer inquiries amid staffing shortages.

It was a sea change that's giving AI-powered chatbot technology a new burst of enthusiasm – a second chance – to fulfill some of its early promise. Today, there are signs that businesses aren't planning to retire their chatbots as things return to normal. And customers have started to accept the idea that talking to an AI isn't as bad as they once believed.

Now, you might converse with a real estate chatbot when you want to inquire about a listed property. You might talk to IBM's Watson assistant when you go looking for tech support for your work from home setup. And you'll even routinely return products to Amazon without ever talking to a person.

The point is, it looks like chatbot revolution 2.0 is now underway. But for it to take hold for good this time, there's one more major hurdle for developers to overcome. It's a lack of multilingual support that limits chatbot use almost exclusively to English-speaking populations. Here are the three challenges associated with including multilingual support in current-generation chatbots.

Breaking Free of Preprogrammed Responses

Right now, most companies that seek to add multilingual support to their chatbots do so in one of two ways. They either link their systems to a translation API like those available from Amazon or Google, or they build their response trees and rule systems in English and turn them over to translation services to create equivalent text.

The problem with the first approach is that machine translations still don't work very well – particularly when dealing with Asian languages. And even when they do work, they often produce garbled text that would render a chatbot close to useless. And the second approach means creating a system that's limited to a very specific set of user queries and possible responses. Both approaches are inflexible and inadequate.

Dealing with Input Errors

Another challenge that limits multilingual support in chatbot deployments isn't about the chatbots themselves – it's about the people using them. People, unlike machines, make plenty of mistakes. In the context of using a chatbot, mistakes like misspellings and inappropriate word usage can grind automated translation solutions to a halt. And even when they don't, they exacerbate the mistranslation issues mentioned above.

The solution to the problem lies in text normalization, but the variety of language mistakes humans tend to make defies easy solutions. Startups like Language I/O are working on this very problem, but even their efforts will require several more years of development to bear significant fruit.

Understanding Intent

Of all of the issues preventing multilingual chatbot support, the most difficult to overcome is finding a way to comprehend user intent. Today's NLP engines require developers to define twelve or more intents for the phrases they'd like their chatbot to understand. But in truth, those requirements are barely even adequate for English-speaking chatbots.

In some languages, the level of nuance and the potential combinations of words that may express the same idea are innumerable. That makes preprogrammed approaches all but impossible when building multilingual chatbots. And frustratingly, the opposite is also true. For example, quite a few languages feature fewer words for things like colors and don't use them in ways that are analogous to English. That makes even simple conversations a tall order to pre-program in current-generation chatbots.

The Bottom Line

At the end of the day, we're still a long way from chatbot technology becoming commonplace, especially outside the English-speaking world. And major brands already understand the damage they can do by providing sub-optimal experiences in the global marketplace. But if this latest surge in chatbot adoption can highlight how useful the current technology is where it does work – there should be far more interest in finding ways to make them converse in multiple languages.

But with the hurdles that remain in the way of making that happen, we're still probably a long way off from a true, multilingual chatbot solution. And that's going to remain a limiting factor for the technology for the foreseeable future. That means this latest iteration of the chatbot revolution will likely bump up against the limits of the technology once again, stalling out just as it did the first time. In other words, we'll all have to meet again when it's time for chatbot revolution 3.0 – whenever that may be.