Primary Storage

 View Only

IBM Storage Insights: Are there best practices for deploying data collectors?

By Chris King posted Tue December 15, 2020 01:00 AM

  

Yes there are! No matter how many -- or few -- storage devices that you plan to monitor with IBM Storage Insights, there are some rules of thumb that can help you when deploying data collectors. 

Before getting into the recommendations however, here's a quick refresher about what a data collector is


A data collector is a lightweight application that you deploy on a server or virtual machine in your data center.

It collects capacity, configuration, and performance metadata and sends that metadata for analysis over HTTPS connections to your IBM Storage Insights dashboard.

In 60 seconds or less, you can learn more about data collectors, the metadata that they collect,  and the security around them here: https://ibm.biz/insightssecuritypage
IBM Storage Insights Security Sheet

Now, let's get into some best practice recommendations for deploying data collectors in your environment.

Redundancy x2

To make your data collection services more robust, deploy two or more data collectors on separate servers or virtual machines in your data center.

IBM Storage Insights data collectorWhen you add storage devices, the data collectors that you deployed are tested to see whether they can communicate with those devices. If multiple data collectors can communicate with a device, then the data collector with the best response time collects the metadata. If the collection of metadata is interrupted, the data collectors are tested again and the data collectors with the best response times take over.

Monitoring devices in multiple data centers

To avoid high network latency and interruptions in the collection of metadata when you monitor storage devices in data centers in different locations, deploy two or more data collectors on separate servers in each data center.


IBM Storage Insights data collector witl failover
Let's say that you install data collectors in your Washington and Chicago data centers and both data centers are connected over the network. If the data collectors in your Washington data center go offline, then the data collectors in your Chicago data center will take over the collection of your metadata for both data centers

Large environments

Storage environments comes in all shapes and sizes. Some are distributed across the world in multiple data centers, while others are more consolidated into one or two sites. Different storage architectures and tiers can also be in the mix and working together, such as object, block, and file storage systems, and Flash and HDD-based arrays.

So when using IBM Storage Insights and data collectors to monitor the health and performance of your storage, what's considered large? Here's the skinny: If your organization uses more than 25 storage devices, or your devices have more than 50,000 volumes, then your environment is considered "large". 

The best practice is to deploy one data collector for every 25 storage devices that you want to monitor. 

The number of volumes that your storage devices manage also determines the number of data collectors that you need to deploy. Typically, one data collector can monitor up to 50,000 volumes. If you have more than 50,000 volumes, it's recommended that you deploy another data collector to maintain the speed and efficiency of metadata collection.  For example, if you monitor 10 storage devices that manage 70,000 volumes, deploy two or more data collectors.

Hardware requirements: If your environment qualifies as "large", it's recommended that you provide at least 4 GB of RAM and 4 GB of disk space on the server or virtual machine where a data collector is deployed.

Reader's Digest version

  • Data collectors are lightweight and require just 1 GB of RAM and 3 GB of disk space in most environments, and 4 GB of RAM and 4 GB of disk space in large environments
  • The devices that you want to monitor must be visible / detectable to the data collectors that you deploy
  • Deploy two or more data collectors on separate servers or virtual machines in each of your data centers
  • Deploy one data collector for every 25 storage devices or 50,000 volumes, whichever comes first
  • Assign data collectors to devices if you want to further optimize metadata collection in your environment
For more information about deploying data collectors, check out this page in the IBM Storage Insights Knowledge Center: https://www.ibm.com/support/knowledgecenter/en/SSQRB8/com.ibm.spectrum.si.doc/tpch_saas_t_data_collector.html

Joining the conversation

Be social! Join the conversation about IBM Storage Insights at #IBMStorage, #IBMCloud, #ibmsystems, and #softwaredefinedstorage. You can also communicate with product experts and other users right here in our technical community.

For continuing news and documentation updates for IBM Spectrum Control and IBM Storage Insights, follow me on Twitter @Chris_TKing.

 






#IBMStorageInsights
#PrimaryStorage
#IBMstorage
#StorageManagementandReporting
#Storage
0 comments
326 views

Permalink