Welcome to our in-depth exploration of watsonx.data's monitoring capabilities, designed to help you maximize the potential of the Presto query engine and watsonx.data. This comprehensive guide covers essential built-in features and observability tools to effectively monitor query execution, performance, and more.

Section 1: Understanding Presto System Connector

We begin our exploration with "system.runtime" schema provided by Presto System Connector(1-1). This schema furnishes crucial information and metrics for queries executing on Presto clusters.
Additionally, we'll introduce the potent kill-query procedure (1-2), offering guidance on efficiently managing resource-intensive tasks.

1-1 Presto: Observation "system.runtime" schema provided by System Connector : Provide insight into the internal operations and performance of a Presto cluster.

1-2 Presto: kill Query : Provide steps to terminate a currently running query within the system.

Section 2: Managing with Presto Console

Next, we introduce the Presto Console, a user-friendly interface that simplifies query execution and management. Learn how to leverage this tool to monitor query performance, manage sessions, and troubleshoot issues.

2 Presto Console : Steps to get Presto Console URL in watsonx.data and view details of Presto Console.

Section 3: Query History with watsonx.data

Transitioning to watsonx.data, we explore its Query History feature which allows users to review running and executed queries, their statuses on multiple Presto engines in watsonx.data cluster.

3 Query History : Overview and Usage

Section 4: Advanced Analytics Presto Performance with QHMM

The Presto query status tool we have introduced so far has been for currently and recently running queries. Next is an introduction to tools that store this information for a longer period.
To delve deeper into query performance analysis, we introduce Query History Monitoring and Management (QHMM). QHMM stores and manages diagnostic data such as query history and event-related information from Presto (Java) and Presto (C++) engines in a storage bucket. This section provides an overview of QHMM, introduces valuable tables and views for analysis, debugging, and monitoring, and explains how to configure QHMM.

4-1 watsonx.data QHMM : Overview, list of tables and views, and Configuration change

4-2 watsonx.data QHMM query_completed_event_view : View query_completed_event_view structure and data examples

Section 5: watsonx.data OpenTelemetry

Finally, we dive into the world of observability with watsonx.data OpenTelemetry. Discover how OpenTelemetry empowers you to trace, visualize, and analyze in watsonx.data. From setting up watsonx.data OpenTelemetry to gain the insights, this section equips you with the skills to monitor and debug watsonx.data performance remotely.

5-1 watsonx.data OpenTelemetry : Overview and Configuration

5-2 watsonx.data OpenTelemetry : See metrics in Instana

5-3 watsonx.data OpenTelemetry : View Presto trace in Instana

Acknowlegment

I extend our gratitude to Thuan Bui, from watsonx.data QA, for his invaluable contributions to this blog series. His technical expertise and meticulous reviews have greatly enhanced the quality and accuracy of this content. Thank you Thuan, for your dedication and support.