In the ever-evolving landscape of cloud computing, graphics processing units (GPUs) have emerged as key to developing technologies such as machine learning, AI, and graphics intense applications and video. The usage of GPU enables virtual machines to be more effective when they are placed in the appropriate application stack. The challenge comes when developers find it difficult to decide which GPU cloud instances would serve them best and, in most cases, they end up over-provisioning. With GPU instance costs close $100 an hour, this can result in a steep increase in public cloud bills.
With Turbonomic’s focus on cloud resource optimization, one of its recommendations is to broadly cover the utilization metrics on every possible commodity (like GPU) and recommend right scaling of the cloud instances.
Turbonomic now discovers NVIDIA GPU metrics for supported AWS EC2 instance types and uses these metrics to generate VM scale actions. Currently, Turbonomic supports P2, P3, P3dn, G3, G4dn, G5, and G5g instance types with Linux AMIs.
Metrics include the number of utilized GPU cards and the amount of GPU memory in use. To optimize performance and costs, Turbonomic can recommend actions that scale down the number of GPU cards, or scale GPU memory up or down.
Turbonomic applies intelligent analytics dynamically to optimize CPU, memory, network, storage, and GPU usage. This capability optimizes the tradeoffs of GPU resources needed to assure application performance for graphic-intense workloads while identifying cost savings.
Example
The new metrics can be seen in the Capacity and Usage and Multiple Resource charts resulting on the following:
A real time example is exposed below where Turbonomic recommends scaling down from p3dn.24xlarge to p3.8xlarge based on the Nvidia’s GPU metrics. Furthermore, under the resource impact tab, it included the impact of taking this action on the GPU metrics.
With this new GPU optimization feature, Turbonomic adds additional value to help customers effectively utilize their GPU workloads for ideal performance and cost.
Customers that are currently on version 8.12.0 or higher can utilize this feature today. For additional details, review the documentation.
New to Turbonomic? You can now try IBM Turbonomic for free for 30-days with no credit card required. For more information, visit our trial sign up page.