[Originally posted on Turbonomic.com. Posted by Steven Haines]
Azure Monitor Logs is a feature of Azure Monitor that collects and organizes log and performance data from monitored resources. Azure Log Analytics is a tool in the Azure portal used to edit and run log queries with data in Azure Monitor Logs. Many Microsoft Azure customers use Azure Log Analytics to capture performance metrics, such as CPU usage and Memory usage.
Today, we’re happy to share that Turbonomic has added support to retrieve virtual machine memory metrics from Azure Log Analytics. The support for Azure Log Analytics expands Turbonomic’s support for Azure memory metrics retrieval, which until now was leveraging Azure Monitor Diagnostics agent.
Why Azure Log Analytics?
Azure supports capturing memory metrics using a diagnostic extension (agent), but can be difficult to manage and has security challenges. Azure Log Analytics, on the other hand, provides an advantage over using a diagnostic extension for both ease of use and management and security. Diagnostics must be configured on a per virtual machine basis, but Azure Log Analytics centralizes the configuration in an Azure Log Analytics Workspace to which you can add virtual machines. The result is that you can set up performance monitoring more quickly and maintain better control over its configuration across your entire virtual machine estate. For enterprise organizations with thousands of workloads, managing VM diagnostics extensions is time- and labor-intensive.
The other primary benefit of using Azure Log Analytics is security. Virtual Machine diagnostics writes its performance metrics to a Storage Account and tools that pull those performance metrics must be granted permission to access your Storage Accounts. The challenge with this is that you may be storing sensitive information in your Storage Accounts and, while the performance management tool is only reading memory metrics, you must give it access to potentially read your sensitive information. Azure Log Analytics solves this because it only stores logs and performance metrics in a Log Analytics Workspace. Thus, reading from a Log Analytics Workspace ensures that your sensitive data remains secure.
Using Azure Log Analytics
Setting up Azure Log Analytics is a three-step process:
- Create an Azure Log Analytics Workspace
- Configure the metrics to be captured in the workspace
- Connect Virtual Machines to the Workspace
First, navigate to “Log Analytics Workspaces” in the Azure portal and choose “Add”:
Choose or create a new resource group, give your workspace a name, and choose “Next: Pricing Tier”:
Choose your pricing tier and add tags, if applicable. Finally, review your workspace and choose “Create”:
Navigate to your newly created workspace, select “Workspace Data Sources” “Virtual Machines,” and choose the virtual machine you want to connect to the workspace:
Click on “Connect” to connect the virtual machine to your Workspace:
Now configure the metrics that you want to capture. From the Workspace, choose “Agents configuration,” select “Windows performance counters,” and choose the metrics you want to capture. Turbonomic queries for “Memory(*)\Available MBytes” on Windows and “Available MBytes Memory” on Linux.
The following screenshot shows the configuration for Windows:
And the following two screenshots show the configuration steps for Linux:
Finally, once the Log Analytics agent has had time to capture metrics, you can review those metrics in the Azure Portal. Navigate to a virtual machine that you added to the Workspace and choose “Logs” from the Monitoring section:
The Logs section provides a collection of preconfigured queries. For memory metrics, the “Virtual Machine available memory” query returns the available Memory, in megabytes, for either a Windows or Linux virtual machine:
These metrics are exactly what Turbonomic is querying for, so if you have values populated in this chart, then Turbonomic can discover your memory metrics.
Azure Log Analytics in Turbonomic
Turbonomic supports Azure Log Analytics in two ways:
- Automatic Discovery
- Explicitly defines workspace IDs
Out-of-the-box, Turbonomic will discover all Log Analytics Workspaces in each subscription, query for memory metrics, and assign them to the correct virtual machines. If your Azure Log Analytics Workspaces are in the same subscription as your virtual machines, then it will work without any modifications. For example, the following screenshot shows the memory usage, from Log Analytics, for the virtual machine configured above:
There are cases, however, in which configuring the workspaces explicitly is required:
- Your virtual machines run in a different subscription from your workspaces
- You do not want Turbonomic to scan all of your workspaces
Many enterprises choose to standardize their Log Analytics workspaces across their whole cloud ecosystem. There are many best practices to further decrease management overhead, and if your organization has centralized your workspaces, Turbonomic can be configured to properly acquire the telemetry from these centralized workspaces. If either of these cases affects you, reach out and we’ll walk you through it.
Azure Monitor collects and organizes log and performance data for your monitored resources, such as Virtual Machines, and Azure Log Analytics is used to run log queries against the data in Azure Monitor Logs. It is a robust, easy-to-configure, and secure solution to capture and analyze your performance metrics. Turbonomic recently added support to automatically retrieve virtual machine memory metrics from Azure Log Analytics, which helps us make better decisions when recommending scaling up or down your virtual machines.
Turbonomic recommends to all its cloud customers to enable memory metrics on Azure and AWS to unlock the full potential of workload scaling. When memory metrics are not enabled, Turbonomic will never reduce the memory allocation and only scale to instance types with the same or higher memory capacity.
Turbonomic will still provide excellent workload optimization, even when Memory is not enabled, but its scaling actions will be more conservative to avoid even the slightest risk of impacting a workload performance.
After all, Turbonomic’s goal is first and foremost to assure workload performance by allocating the resources the workload needs to perform, and if Turbonomic can do it cost-effectively, it will, but never the other way around where cost-effectiveness might introduce performance risk.