Hi there,
We have followed this documentation when setting up AWS Bedrock, and using Claude 3.7 sonnet as our LLM:
https://developer.watson-orchestrate.ibm.com/llm/managing_llm#aws-bedrock
However, it appears it is not using Thinking mode, and therefore causing a lot of hallucination.
We have been using the same LLM model on other platforms, and Claude behave the best, however, on Watsonx Orchestrate, it appears:
1. Not really doing tool calling.
2. Not using Thinking mode.
3. Ignoring my instruction.
I have tested Default agent style and ReAct agent style, both have similar experience which is not acceptable.
We have setup another custom model using OpenRouter's Deepseek v3.1, I think this model has thinking mode turned on by default and the response is good.
Many Thanks in advance.