Happy to share that IBM Storage Insights through its 3Q2023 update, is now offering Capacity Forecasting and Planning for IBM Flash Storage !!! This is first in a series of AI centric features in IBM Storage Insights' AIOps Roadmap.
This blog aims to provide a brief overview of the technical details of the feature and how users can navigate and experience it. So let's start.
Technology Innovation with AI
IBM Storage Insights in collaboration with IBM Research designed an AI forecasting framework to extend, configure and leverage AI algorithms for training, validating and forecasting capacity growth with high confidence.
Through the discovery, analysis and proof-of-concept phases, the team analysed thousands of storage devices and their capacity growth patterns from over a few days to multiple years. This helped understand various factors influencing capacity growth and design a forecasting framework tailored for Storage.
The resultant solution while forecasting, will dynamically evaluate
- the models and their parameters for the chosen storage system/s, their composition (when aggregates, pools are chosen)
- depth of historical trends
and surface the best possible forecasting for the user selection.
Further, the framework will allow collection of run time information about the models, their parameters and forecasting accuracy. This in turn will be leveraged for re-calibrating and fine tuning the forecasting model to evolve, adapt and improve its accuracy.
Business Use Case
The forecasting feature will allow forecasting on available Block Systems, File Systems, their aggregation as a STaaS (Storage as a Service) as well as their constituent parts (pools which adds up to the device)
This AI-infused capacity forecasting feature is being offered to Storage Administrators for following use cases
- Trigger forecasting from current date up to two years in future. This will help to understand how far (in time) the storage device will last in terms of capacity. This in turn helps plan
- Timely procurement of additional storage
- Implement storage optimisations (through data purging/archival, workload migration etc.)
- Perform Scenario Modelling (What-If Analysis) with user supplied capacity growth rate and compression ratio (ratio of written capacity to used capacity). This helps in two ways
- User supplied growth rate is better, when user (as a planner, decision maker) is aware of a un-planned change/ addition/deletion in storage workload which the AI model is unaware of.
- Understand the influence of enabling and applying Compression Ratio to improve / extend the date to exhaust capacity (capacity limit).
User Journey and Experience
Now with an understanding of the technical design and its manifestation in use cases, let's see how these features can be put to use.
Steps
1. Login in IBM Storage Insight tenent and navigate to Capacity Planning through IBM Storage Insights>>Insights Menu>>Capacity Planning
Diagram 1. (Navigating to Capacity Planning)
2. Select a Resource Type from among Block Storage, File Storage, STaaS or Pools and hence use the check boxes to select one or more of the storage devices. (Selection of multiple devices is treated as a logical aggregation for the purpose of capacity forecasting)
Diagram 2. (Device Selection for Capacity Planning)
3. Trigger Capacity Forecasting for the chosen device (or group of devices) to receive
a. Forecast Summary - which is a tabular representation of current (factual) and future (forecasted) growth parameters
b. Growth Projection - a graph representation which compares historical and forecasted growth with capacity limit with reference to a time scale
Diagram 3. (Trigger and Consume Capacity Forecasting)
4. Further, toggle to switch from "Growth Forecast" to "Expected Growth Forecast" to perform growth scenario modelling with following features
a. Switch to Written Capacity (from used capacity)to model the capacity that the application workload writes to the storage device
b. Key in "Expected Growth Rate %" to override AI forecasted growth rate with User Supplied growth rate
c. Key in "Expected Compression Ratio" to override override the observed compression ratio
Diagram 4. (User driven Scenario Modelling)
and lastly,
5. a sensible user experience !!!. It does not require the user to wait while the AI framework trains, validates, forecasts and renders the results. The feature offers user the choice to navigate away to other dashboards in Storage Insights and be notified when the forecast is ready to be consumed.
Diagram 5. (Background execution and user notification)
Behind the scene : The development journey
I am fortunate to have a team comprising of expert data scientists, data engineers, designers, front end & back end developers, testers, SREs in IBM Systems Development Lab. (IBM ISDL).
The features that I blogged here is a culmination of effort of my team working in collaboration with IBM Almaden Research Center (IBM ARC)
A big shout out to my team and my colleagues in ARC !!!
And with this, let me wrap up this blog. I will urge our esteemed customers to go ahead and try out the new features.
For comprehensive details of features and enhancements in IBM Storage Insights Q3, 2023 update, please refer https://www.ibm.com/docs/en/storage-insights?topic=whats-new
IBM Flash Customers, who are curious to know and adopt Storage Insights may please refer IBM Storage Insights page for details and also get a quick hands-on with Storage Insights Demo URL
Thank you.
Binayak Dutta
AI Architect - IBM Storage Insights
IBM Systems Development Labs., India.
https://www.linkedin.com/in/binayakdutta/