After enough time has elapsed for your assistant to engage in numerous conversations with real users, you have the opportunity to consistently enhance and optimize its performance. You can achieve this by analyzing user interactions and feedback. Regularly monitoring and analyzing conversations and interactions through the Analyze tab allows for the identification of areas that need improvement, thereby enhancing the effectiveness of the chatbot.
This tutorial provides a user-friendly guide on using the Analyze tab. It helps identify the most frequently used actions, the least completed actions, and the current topics that are being overlooked. This information is valuable in determining which actions require updating and which new actions need to be created.
Prerequisites
To follow this tutorial, you need:
- An IBM Cloud account
- Access to any watsonx Assistant instance on a Lite, Plus Trial, Plus, or Enterprise plan
- Access to any assistant
- An understanding of the watsonx Assistant user interface
Estimated time
It should take you approximately 10 minutes to complete the tutorial.
Steps
-
Navigate to the Analyze tab. Improve your assistant by using the Overview screen, which provides engagement metrics. This page lets you identify key insights that can enhance your assistant's performance. Start by selecting the environment and date range for analysis. The charts and cards on the page reflect data that is based on your chosen environment and date range. Whenever you change the environment or date range, the charts and cards automatically update to display the corresponding data. Use the Most frequent actions and Least frequent actions cards to learn which actions are being started the most and least often.
-
Navigate to the Action completion screen. The table below the graph allows you to learn which actions reach an ‘End this action’ step the most often. Actions with low completion rates are worthy of investigating to see whether the flow of the action can be improved. Click one of the actions to display the single action analysis screen.
-
On this screen, you can learn and filter by reasons for incompletion (or completion). Click the Incomplete or Complete tab to see the action incompletions, which appear as different colors depending on the reason. Look for patterns as to why your action is or is not working. In this case, you see that the action is being abandoned by users who seem to have a matching intent to the action. This tells you that the action might need to be improved to become more approachable, or shorter in general. Close out of the screen by clicking X in the upper right.
-
Navigate to the Conversations screen. This screen provides a history of conversations between users and your assistant, depending on the environment selected. With the Live environment selected, you can use this history to understand which intents are not matching to any actions for your deployed assistant. Look for patterns in the Unrecognized topics to learn the actions that your users are hoping to find available through your assistant. Consider building these topics into new actions.
Summary
In just a few short minutes, you were able to use the Analyze tab’s Action completion screens to identify an action with a high incompletion rate and learn why that incompletion is happening. You also are able to discover several topics that you could train the assistant on by reviewing unrecognized topics in the Conversations screen. This lets you return to the Actions and Preview tabs to create, update, and preview new actions for your deployed assistant.
Remember to return to your Analyze tab frequently. When enough conversations occur to generate clusters, the Recognitions screen offers additional action recommendations. Learn more about this topic in the product documentation.
This tutorial completes the watsonx Assistant Getting Started learning path, but stay tuned for newly published watsonx Assistant tutorials for beginners and advanced users alike.
Any questions? Reach out to Alexander Dfouni.