By this time, you have probably heard of agents in the space of Generative AI. While there are several ways to describe an agent, the most relevant definition for AI engineers is not an “application”, but an “application pattern”. A pattern is a blueprint, which means that we have to write code to implement it. In the agent pattern, Large Language Models (LLMs) act as intelligent orchestrators, steering the flow of tasks and decision-making processes. This orchestration, traditionally handled by if/else rules in classical programming, can be dynamically managed by LLMs through a flexible and context-aware approach.
Orchestration by LLMs: A Shift from Traditional Methods
In traditional programming, decision-making is straightforward yet inflexible. Developers write explicit if/else statements to dictate the program's flow based on predefined conditions. With LLMs, we can leverage their understanding and reasoning capabilities to orchestrate actions.
Let’s say our goal is to implement a relatively simple yet versatile AI Assistant designed to interact with a comprehensive knowledge base. When a user asks a question in natural language, the assistant determines the best action, for example:
- If the question relates to structured data (e.g., "What was the revenue last quarter?"), it converts the question into an SQL query and executes it against the database.
- If the question pertains to unstructured data (e.g., "Find recent articles on AI advancements"), the assistant performs a search.
- If the prompt is an action, the assistant can perform a task like send an email based on the information retrieved (e.g., "Email me the quarterly report").
The implementation of agents involves two main components: tools and prompts. Tools are standard APIs that the agent can invoke to perform specific actions, such as searching the web, executing SQL queries, or sending emails. The second component, prompts, must be written in a format known as ReAct (Reasoning and Acting), which guides the LLM in how to interpret and respond to different scenarios. The ReAct format is a few-shot prompting format that contains reasoning
ReAct prompts can be generic, as shown in a sample watsonx.ai notebook, or specific to a task or decision that we need to execute. For example, a decision-specific prompt can include the following information:
Task: Distribute Quarterly Report
Thought: I need to prepare the report for distribution.
Act: [Prepare] prepare the report.
Observation: I have the report, now I need to distribute it.
Thought: I need to identify stakeholders.
Act: [Search] search for the list of stakeholders.
Observation: To distribute the report, I need to determine the distribution method.
Thought: I need to look up approved distribution methods.
Act: [Search] for approved distribution methods.
Observation: I have everything I need to send the report.
Thought: Now, I have everything I need to send the report.
Act: [Send] send the report.
Observation: I have sent the report
Thought: I have finished the task
Action: [Finish]
In this prompt, the words in brackets [Prepare], [Search], [Send], and [Finish] correspond to tools (implemented as functions) that will be invoked.
Several open-source frameworks streamline the implementation of agents. LangChain and LlamaIndex are two notable examples that provide robust APIs to facilitate the development of these intelligent agents. They offer tools and libraries to easily integrate various data sources and interaction methods into the agent's workflow.
Watsonx.ai: Enhancing Agent Development
Watsonx.ai, IBM's AI and data platform, supports the implementation of the agent framework using LangChain and LlamaIndex. This platform enables developers to build sophisticated agents by providing the necessary infrastructure and tools, ensuring seamless integration with existing systems and data sources.
IBM provides a notebook which you can use as an example for implementing the agent pattern in watsonx.ai.
#watsonx.ai