AIOps

AIOps

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.


#ITAutomation
#AIOps
#CloudPakforAIOps
#AIOps

 View Only

Openshift Data Foundation (ODF) Troubleshooting through CP4AIOps MustGather

By Daniel Yeap posted Thu October 24, 2024 03:20 AM

  

Openshift Data Foundation (ODF) is one of the recommended storages for IBM Cloud Pak for AIOps.

The CP4AIOps MustGather offers an easy way to interact with and troubleshoot ODF.

[TEST ENVIRONMENT]

OCP Server = 4.16.11
OC Client = 4.16.11
ODF Version = 4.16.2
CP4AIOps MustGather = 1.23.13
CP4AIOps = 4.7.0

To download CP4AIOps MustGather:
https://www.ibm.com/support/pages/node/7022441

To upgrade CP4AIOps MustGather to the latest:
./waiops-mustgather.sh -UD

(1) To list all pre-configured tasks:

./waiops-mustgather.sh -RMm .listmc


Eg.
====================================================================================================================================================================================
TAG                       DESCRIPTION                                                                                EXAMPLE                                                 
====================================================================================================================================================================================
...                                  
odf-clicmd                To provide cli command execution from rook-ceph-operator in openshift-storage namespace    -m odf-clicmd:mc.cfg -n 'CLICMD="rados lspools"'        
odf-healthcheck           To collect health/storage data from rook-ceph-operator in openshift-storage namespace      -m odf-healthcheck:mc.cfg                               
odf-healthreport          To collect 'ceph report' output from rook-ceph-operator in openshift-storage namespace     -m odf-healthcheck:mc.cfg                               
odf-osddump               To collect 'ceph osd dump' output from rook-ceph-operator in openshift-storage namespace   -m odf-healthcheck:mc.cfg                               
odf-rados-bench           To collect 'rados bench -p ocs-storagecluster-cephblockpool 10 write' output               -m odf-rados-bench:mc.cfg                               


(2) To obtain a comprehensive healthcheck report of ODF:
./waiops-mustgather.sh -RMm odf-healthcheck:mc.cfg


The ODF healthcheck involves the following commands:

ceph status
ceph df
ceph health detail
ceph healthcheck history ls
ceph osd tree
ceph osd df tree

(3) To obtain output of "ceph report" command:
./waiops-mustgather.sh -RMm odf-healthreport:mc.cfg


(4)  To obtain output of "ceph osd dump" command:
./waiops-mustgather.sh -RMm odf-osddump:mc.cfg


(5) To test write (I/O) performance of Ceph storage:
./waiops-mustgather.sh -RMm odf-rados-bench:mc.cfg

(6) All of the above please!
./waiops-mustgather.sh -RMm odf-healthcheck,odf-healthreport,odf-osddump,odf-rados-bench:mc.cfg

(7) Anything else but the above?

If you have a need to execute command(s) not covered by the above, you can also do this:
./waiops-mustgather.sh -DRMm odf-clicmd:mc.cfg -n 'CLICMD="<CMD>"'


Eg.
./waiops-mustgather.sh -RMm odf-clicmd:mc.cfg -n 'CLICMD="rados lspools"'

===================================================
[MANUALCOLLECT] OUTPUT OF EXECCMD (PROD_GROUP = cp4aiops@ibm-cp-aiops / PROD_VER = 4.7.0)
===================================================
[1] [OUTPUT]
.mgr
ocs-storagecluster-cephblockpool
ocs-storagecluster-cephobjectstore.rgw.control
.rgw.root
ocs-storagecluster-cephobjectstore.rgw.buckets.index
ocs-storagecluster-cephobjectstore.rgw.otp
ocs-storagecluster-cephobjectstore.rgw.meta
ocs-storagecluster-cephobjectstore.rgw.buckets.non-ec
ocs-storagecluster-cephobjectstore.rgw.log
ocs-storagecluster-cephfilesystem-metadata
ocs-storagecluster-cephobjectstore.rgw.buckets.data
ocs-storagecluster-cephfilesystem-data0


Hope that helps!

0 comments
19 views

Permalink