Cloud Pak for Data

 View Only
Expand all | Collapse all

Questions for AMA: Setting up Cloud Pak for Data

  • 1.  Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu October 21, 2021 04:41 PM
    Edited by System Test Fri January 20, 2023 04:38 PM
    We'd like to answer your questions about setting up Cloud Pak for Data. 
    We've arranged for experts from across IBM to answer your questions right here in this forum thread on November 4th at 2pm Eastern/11am Pacific for a whole hour of AMA (Ask Me Anything).  Our topic is Setting up Cloud Pak for Data, so if you have questions, please start posting them as a response to this post.  Here are some ideas for topics:
    • Planning and preparing, including storage support, planning for high availability and disaster recovery, regulatory compliance, multi-cloud (Azure, AWS, IBM Cloud, etc.)
    • Installing the platform, including setting up a private container registry and performing installations in an air-gapped environment, installing on a shared cluster with dedicated nodes, and installing on a shared cluster with a namespace quota.
    • Installing the services, including recommended service combinations and best practices for installing services.
    • Setting up the platform, including certificates, using shared credentials, and creating a custom route.
    • Securing the platform, including user management best practices around SSO and LDAP.
     
    Our experts will hop on the Cloud Pak for Data Community discussion forum on November 4th at 2pm Eastern/11am Pacific and start answering your questions right here in this thread. 
    To learn more, or to get this AMA on your calendar, go to the event page AMA: Setting up Cloud Pak for Data. This event will take place entirely in the discussion forum, so there is no meeting to join.  If you can't be online during the hour, don't worry; you can post your questions in advance and read the responses later.  


    ------------------------------
    Shannon Rouiller
    ------------------------------
    #CloudPakforDataGroup


  • 2.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu October 28, 2021 05:29 PM
    What are some examples of storage options that we can use?

    ------------------------------
    Kelley Tai
    ------------------------------



  • 3.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:12 PM
    OpenShift Container Storage , Portworx,  ibmc-file-gold NFS etc.

    more details here:  https://www.ibm.com/docs/en/cloud-paks/cp-data/4.0?topic=planning-storage-considerations

    ------------------------------
    Sriram Srinivasan
    ------------------------------



  • 4.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Fri October 29, 2021 03:52 PM
    How is data on different storage types being backed-up?

    ------------------------------
    Kelley Tai
    ------------------------------



  • 5.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:37 PM

    It largely depends on the options provided by the Storage vendor.  Most vendors now support the concept of snapshots that can be used to take point-in-time backups.  
    Note - NFS as a protocol does not natively enable such snapshots - however the expectation is that the storage backend that exposes such NFS volumes would be able to support snapshots. 

    In addition,  we have the cpd-cli backup-restore volume-backup mechanism to copy file content  from persistent volumes to a remote store.

    Also Cloud Pak for Data v4 recently introduced an enhancement to the cpd-cli backup-restore command to leverage the `OpenShift® APIs for Data Protection (OADP)' feature.

    More details here: https://www.ibm.com/docs/en/cloud-paks/cp-data/4.0?topic=cluster-backing-up-restoring-your-deployment



    ------------------------------
    Sriram Srinivasan
    ------------------------------



  • 6.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Mon November 01, 2021 12:17 PM
    Hello! Thank you for this AMA. 

    I am new to CP4D and was wondering what is the best recommended way to get started installing a cluster and cartridge (official docs, TechZone, Your Learning, etc.) ? I would prefer a hands-on tutorial that would allow me to practice provisioning an environment. Thank you!

    ------------------------------
    Brian Bui
    Cloud Engineer
    IBM
    ------------------------------



  • 7.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:04 PM
    At this time - the best way would be to follow the instructions here: 
    However, I realize that this describes multiple options and may  be a bit complex as a quick start.  We will take a look at publishing a short tutorial for the simplest (perhaps not for production) flow. 


    ------------------------------
    Sriram Srinivasan
    ------------------------------



  • 8.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:05 PM
    Installation and other CPD topics are covered really well by this learning path: https://www.ibm.com/training/path/administrator%3Aibmcloudpakfordatav4.x

    ------------------------------
    Yalon Gordon
    ------------------------------



  • 9.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Tue November 02, 2021 09:15 AM
    Hi! Another question on storage: How does licensing work for storage? How would we pay for it?

    ------------------------------
    Kelley Tai
    ------------------------------



  • 10.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:48 PM
    Cloud Pak for Data supports multiple different storage options, if you choose to use NFS then you do not need any additional entitlements.

    If however, you choose to use OCS/ODF or Portworx, there is a freemium tier. Once the consumption exceeds the freemium limit, additional entitlements can be purchased based on VPC metrics.

    ------------------------------
    Deepak Rangarao
    ------------------------------



  • 11.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Tue November 02, 2021 02:30 PM
    Edited by System Test Fri January 20, 2023 04:49 PM
    This is a great AMA thread. Here's my question. The install guide talks about Foundational Services and Operator Subscriptions, what are they and how are they related to Cloud Pak for Data?


    ------------------------------
    Trish Smith
    ------------------------------



  • 12.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:07 PM
    Edited by System Test Fri January 20, 2023 04:22 PM
    1).  IBM Cloud Pak Foundational Services is a set of capabilities that are used by multiple Cloud Paks.  This includes capabilities like service operand dependency management, security (namespace scoping, certificates) and key integration components such as Identity and Access Manager that helps customers set up single-signon, even across multiple Paks in the cluster.  The foundation services are deployed only once in a cluster and shared by all Cloud Pak's deployed in that cluster.
    2) An Operator is essentially software that runs in a Kubernetes environment and manages the deployment and configuration of an application in that Kubernetes environment.  This is a good reference in the Operator Lifecycle Manager project - https://olm.operatorframework.io/docs/concepts/olm-architecture/  to describe some of the core concepts
    a short summary:
    CustomResourceDefinition (CRD): Custom resources are extensions of the Kubernetes API.  Defining a CRD object declares a new Custom Resource with a unique name and schema & represents the new API being introduced in that cluster.
    ClusterServiceVersion (CSV):  metadata about the Service/Application managed by the Operator [name, icon, description etc.]. Identifies pre-reqs on which other Operators are needed, describes RBAC needed for running the Operator,  and Custom Resource(s) it manages
    CatalogSource :  a collection of Operators - CSVs, CRDs etc.  A store that you can query to discover and install operators.
    Subscription:  essentially indicates which Operator to stand up  from a specific CatalogSource
     
    --- 
    Cloud Pak for Data Service Operators are "listed" in Catalog Sources,  

    Operators are deployed via a Subscription

    A Service Operand is represented by a "Custom Resource" (CR) that cause the  deployments of that selected service in a CPD "instance" namespace 

    Based on the CR directive, Operators deploy & manage Operands as well as upgrade them when needed.  As end users, you primarily interact with these operands
     
    A IBM Cloud Pak for Data deployment instance includes several integrated and dependent applications, consequently the deployment and configuration of those applications requires the corresponding set of Operators to be deployed.  
    The installation guide details an initial set of Operators to be deployed.  Deployment of those Operators will automatically deploy many additional dependent Operators including IBM Cloud Pak foundation services.


    ------------------------------
    Sriram Srinivasan
    ------------------------------



  • 13.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Tue November 02, 2021 03:16 PM
    Edited by System Test Fri January 20, 2023 04:23 PM
    Looking forward to the AMA!

    Here's my question: What types of things have you found that teams forget to plan for before setting up Cloud Pak for Data?

    ------------------------------
    SHARYN RICHARD
    ------------------------------



  • 14.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:48 PM
    Cloud Pak for Data 4.0x has few new requirements

    - OpenShift version should be 4.6.30+
    - Private Registry, compliant with the Docker Image Manifest Version 2, Schema 2
    - Planning the network access for IBM entitlement registry and github
                         cp.icr.io/cp
                         icr.io/cpopen
                         quay.io/opencloudio
                         docker.io
                         github.com
    - OpenShift admin permissions
    - Cluster each nodes should meet Cloud Pak for Data minimum requirements
        16 vpu, 64GB RAM and 300GB storage
    -
    Please refer this knowledge center.
    https://www.ibm.com/docs/en/cloud-paks/cp-data/4.0?topic=planning



    ------------------------------
    Lakshmana Ekambaram
    AI Architect
    IBM
    San Jose CA
    ------------------------------



  • 15.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:58 PM

    Storage is also critical to plan out for. -  some background:
    https://www.ibm.com/docs/en/cloud-paks/cp-data/4.0?topic=tasks-setting-up-shared-persistent-storage

    You should also decide what use cases to start with and then deploy the appropriate services that you need - and size the cluster (worker node compute) for that use case correctly.
    For example,  if you need to start with a Data Science use case, install Watson Studio, Watson Machine Learning to begin with - and then later if you need Data Governance, introduce that Service, and expand the OpenShift cluster at that time.



    ------------------------------
    Sriram Srinivasan
    ------------------------------



  • 16.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Tue November 02, 2021 04:11 PM
    Hello again! My enterprise does not grant cluster admin access to everybody on a shared cluster.  My question is, what does a cluster admin need to do first before authorizing the project administrator to install Cloud Pak for Data? 

    ------------------------------
    Kelley Tai
    ------------------------------



  • 17.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:08 PM
    The Cluster Admin would need to make the Operators available and to setup the Namespaces where Cloud Pak for Data would reside in.  The Cluster Admin would also grant authorization to the Kubernetes Namespace (OpenShift Project) users to trigger the installation of Cloud Pak for Data servicces.
    For more details:


    ------------------------------
    Sriram Srinivasan
    ------------------------------



  • 18.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Tue November 02, 2021 04:11 PM
    Edited by System Test Fri January 20, 2023 04:43 PM

    Here's a great question for the AMA. Does Cloud Pak for Data support Single Sign-on as a standard?

    Our security requires that all our users need to be authenticated by our enterprise SSO.

    How is it configured? Are there any prerequisites?



    ------------------------------
    Trish Smith

    ------------------------------



  • 19.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:14 PM
    Yes - via the  SAML  WebSSO standard 
    The Cloud Pak Foundational Services Identity and Access Manager (IAM) is another option that enables SSO - it can also help support SSO across multiple Paks on the same cluster


    ------------------------------
    Sriram Srinivasan
    ------------------------------



  • 20.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Tue November 02, 2021 04:15 PM
    My enterprise does not grant end-users Kubernetes access for security reasons and some Cloud Paks expect that. My other question is, does Cloud Pak for Data need such access too?

    ------------------------------
    Kelley Tai
    ------------------------------



  • 21.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:11 PM
    No.  Cloud Pak for Data end users are not expected to have access to Kubernetes and the recommendation is not to do so. Even those users who are granted the "Administrator" Role in Cloud Pak for Data do not require Kubernetes access
    Only OpenShift cluster Admins or users performing installation or upgrade activities should need any access to Kubernetes.


    ------------------------------
    Sriram Srinivasan
    ------------------------------



  • 22.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Tue November 02, 2021 04:26 PM
    Edited by System Test Fri January 20, 2023 04:22 PM
    Anyone have some answers for this question? Does Cloud Pak for Data support Multi-factor authentication (MFA)? Are there any steps that I can follow?

    ------------------------------
    Trish Smith
    ------------------------------



  • 23.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:14 PM
    Cloud Pak for Data would rely on the configured Enterprise Identity Provider to support MFA.   A typical approach is to configure SAML and Cloud Pak for Data will delegate authentication to that Identity Provider.  Such Identity Providers would then be able to validate user access or provide MFA steps for them to authenticate themselves

    ------------------------------
    Sriram Srinivasan
    ------------------------------



  • 24.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Tue November 02, 2021 04:28 PM
    Hi, looking forward to this AMA. Here's my question:  What facilities exist for security auditing of Cloud Pak for Data clusters?

    ------------------------------
    Karin Moore
    ------------------------------



  • 25.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:16 PM
    There are multiple levels of auditing - one at the OpenShift level -
    RHEL Auditing: 
    apart from auditing Cloud Pak for Data itself
    CPD provides for forwarding to different SIEM providers as well as works with IBM Guardium for sensitive data access auditing


    ------------------------------
    Sriram Srinivasan
    ------------------------------



  • 26.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Tue November 02, 2021 06:05 PM
    Our Red Hat OpenShift clusters are not open to pull from the public internet registries because the outbound network is blocked from a security perspective. How would we get started with Cloud Pak for Data in such a case?

    ------------------------------
    SHARYN RICHARD
    ------------------------------



  • 27.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:15 PM
    The approach would be to use your own Private Registry. 
    Such a registry needs to be Docker schema v2 manifest compliant & should support hierarchical structure (paths)  [ the internal out-of-the-box Openshift registry  cannot be used]
    You will then mirror all images into that private registry


    ------------------------------
    Sriram Srinivasan
    ------------------------------



  • 28.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:23 PM
    Customer should follow the AIRGAP installation method by setting up a private registry and mirror images from IBM entitlement registry to private registry.  Then configure openshift to pull images from this private registry within their network.

    https://www.ibm.com/docs/en/cloud-paks/cp-data/4.0?topic=tasks-mirroring-images-your-private-container-registry

    ------------------------------
    Lakshmana Ekambaram
    AI Architect
    IBM
    San Jose CA
    ------------------------------



  • 29.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Tue November 02, 2021 07:39 PM
    Security-related question: Can we run security scans across the Cloud Pak for Data Docker images before we pull them? What is the procedure to report scan failures?

    ------------------------------
    SHARYN RICHARD
    ------------------------------



  • 30.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:17 PM
    Yes - you can scan all the images upfront - ideally by mirroring them to your private registry first
    It would be best to open up a Support Case ( https://www.ibm.com/mysupport/s/topic/0TO50000000IYkUGAW/cloud-pak-for-data?language=en_US ) - as part of that case, you can also get information about newer versions of these images that may have remediated the problem you reported. 
    The CPD team also scans all images - but new CVEs get reported frequently in the Linux world. Our  policy is to release new versions of these images approximately every 30 days or so - to pick up any fixes for the Linux CVEs.


    ------------------------------
    Sriram Srinivasan
    ------------------------------



  • 31.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Wed November 03, 2021 04:34 AM
    Hi Experts,

    I am happy to see that the topic for Setting up Cloud Pak for Data is gathering all these valuable questions in one thread. 
    My question to you is: Does Cloud Pak for Data support encryption at rest and in motion?
    Thank you!

    ------------------------------
    Polya Markova
    ------------------------------



  • 32.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:22 PM
    ou need to consider at rest encryption  at the OpenShift level with  full disk encryption using Linux Unified Key Setup (LUKS) 
    Cloud Pak for Data itself relies quite a bit on the storage layer to support encryption at rest - for example -  OCS, Portworx and others provide for that
    Inter-micro-service pods use TLS for encryption in motion  - (with very few exceptions)
    References: 


    ------------------------------
    Sriram Srinivasan
    ------------------------------



  • 33.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Wed November 03, 2021 04:42 AM
    And another security-related question from my side:

    We expect to run multiple installations of Cloud Pak for Data in the same shared cluster for different tenants. How can we ensure security and isolation between tenants?

    Thanks once again for the discussion, I will be looking forward to your reply.

    ------------------------------
    Polya Markova
    ------------------------------



  • 34.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:27 PM
    The  recommended approach here would be to install multiple instances of Cloud Pak for Data into different OpenShift Projects (Kubernetes namespaces)

    You would then be able to grant authorization to different instances of Cloud Pak for Data to different "tenants", identify individual tenant admins to delegate to  and even configure Auditing for each instance independently

    Note: The set of Operators (and the scheduler) manage and control these multiple instance deployments from a central namespace - and can even upgrade individual instances of CPD to different version levels while retaining other instances at a different release level.

    Some background here: https://www.ibm.com/docs/en/cloud-paks/cp-data/4.0?topic=planning-multitenancy-support

    ------------------------------
    Sriram Srinivasan
    ------------------------------



  • 35.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Wed November 03, 2021 03:37 PM
    Hello again! I have another question for the AMA:  Our security standards require that containers cannot access the host file systems or networks. Would Cloud Pak for Data be able to comply?

    ------------------------------
    Karin Moore
    ------------------------------



  • 36.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:34 PM
    Redhat Openshift platform security standards are enforced through Security Context Constraints (SCC), the restricted SCC prevents access to host file system etc thereby complying with the necessary security standards. Most of Cloud pak for data services use the restricted SCC, though we have a few services that still use custom scc. You will see even these services move towards restricted SCC in the future but in the meantime the cluster administrators still have the ability to manage what custom SCCs can do.

    ------------------------------
    Deepak Rangarao
    ------------------------------



  • 37.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 09:22 AM
    Edited by System Test Fri January 20, 2023 04:36 PM
    • I want to control the version of services deployed on my environment that the AirGap mode provides but I do not want to maintain a Private Container Registry. Do I have a choice?


    ------------------------------
    Trish Smith
    ------------------------------



  • 38.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:07 PM
    Edited by System Test Fri January 20, 2023 04:29 PM
    You can configure cluster  firewall/proxy  to allow access to IBM image registry.  Then, you don't need a private registry.

    After the install, ensure you add version info to the service CRs. to  prevent accidental upgrade of the service to latest version.

    ------------------------------
    Lakshmana Ekambaram
    Architect
    IBM
    San Jose CA
    ------------------------------



  • 39.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:50 PM
    Yes, you have an option that combines the control of service versions that comes with AirGap mode with the simplicity of Online deployment via the IBM Entitled Registry.  You create mini-catalogs in the same way as you would for AirGap mode using downloaded CASE packages except defaulting the registry to the IBM Entitled Registry instead of the Private Container Registry. You do not need to create Image Content Source Policy and your pull secrets are for the IBM Entitled Registry only. Then create subscriptions that reference these mini catalogs. While you gain simplicity of deployment avoiding Private Container Registry you lose the ability to scan the images for security issues prior to them being deployed in your environment.

    ------------------------------
    Amitabha Das
    ------------------------------



  • 40.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 09:41 AM
    Looking forward to the AMA later today! Can you give some tips on how I can increase my probability to deploy successfully?

    ------------------------------
    Kelley Tai
    ------------------------------



  • 41.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:27 PM
    Preparation is key, make sure you have all the information you need for the install including 
    - Storage to be used
    - T-shirt sizes for the individual service instances, making sure you have enough resources available.
    - Specs copy/pasted to Vi or some editor that does not add unnecessary characters

    BEFORE you start your install process. The next key to success is to ensure each step completes successfully before proceeding to the next step.

    ------------------------------
    Deepak Rangarao
    ------------------------------



  • 42.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 03:20 PM
    Following these steps can help
    1. Ensure the Disks meet the minimum required performance
    2. Configure the required Node Settings for the cluster
    3. Ensure the Global Image Pull Secrets has been configured
    4. For AirGap, ensure Image Content Source Policy has been configured
    5. For AirGap, ensure download speed from Private Container Registry is reliable and good
    6. All the requisite Catalog Sources have been configured
    And then for each service
    1. Configure any service specific node settings requirements have been configured or any specific dependent services needed have been deployed
    2. Create the Service Subscription via YAML and ensure the subsequent Operator pod(s) have been created
    3. Create the Service CR via YAML and wait for it to complete. Verify its status


    ------------------------------
    Amitabha Das
    ------------------------------



  • 43.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 09:42 AM
    Some of the configurations I perform result in rolling updates of the nodes on the cluster. Can I optimize?

    ------------------------------
    Kelley Tai
    ------------------------------



  • 44.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:23 PM
    Yes, you can get/execute all commands that result in a rolling update together. Some examples include
    - Updating the global pull secret.
    - Updating CRIO settings
    - Updating Kernel Parameters.

    Executing the above together will result in a single rolling update.

    ------------------------------
    Deepak Rangarao
    ------------------------------



  • 45.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:40 PM
    Also - Red Hat has made some serious improvements with OCP v4.8 to avoid needing worker node reboots - https://docs.openshift.com/container-platform/4.8/release_notes/ocp-4-8-release-notes.html#ocp-4-8-machine-config-operator-image-content-source-object-enhancement

    ------------------------------
    Sriram Srinivasan
    ------------------------------



  • 46.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 03:06 PM
    To perform rolling update on a node, it must drain all the pods from that node to other nodes before the update can be applied so it is better and faster to do so when there are fewer pods. Also create and execute a script file with all these configuration changes, so all of them can be configured using one round of rolling update. Following configurations initiate rolling update of the cluster
    1. Enabling Unsafe Sysctl needed by the Db2U Operator
    2. Updates to global Pull Secrets
    3. Updates to Image Content Source Policy
    4. Updates to Cri-o Configurations


    ------------------------------
    Amitabha Das
    ------------------------------



  • 47.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 09:46 AM
    Hi experts, one more question from my side:

    While Ansible, Operators, ODLM, etc make the complex deployment simple, it is very hard for me to follow the progress.
    How do I monitor the progress of service deployment especially those using ODLM and CASE dependencies when using command line tools?

    ------------------------------
    Polya Markova
    ------------------------------



  • 48.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:21 PM
    The overall installation process is, check status in the sequence below
    1) Install catalog source
    2) Create subscription 
    3) Create service instance

    Checking catalog source

    Catalog source should be created/visible when you run this command
    oc get catsrc -n openshift-marketplace
    All pods should be running
    oc get po -n openshift-marketplace

    Checking subscription
    Running below command should return CSV name (to be used in next command)
    oc get sub -n <operator-namespace> <subscription-name> -o jsonpath='{.status.installedCSV}'
    Running below command should say succeeded
    oc get csv -n <operator-namespace> <csv-name> -o jsonpath="{ .status.phase } : { .status.message} {"\n"}"
    Running the below command should return >=1
    oc get deploy -n <operator-namespace> -l olm.owner="<csv-name>" -o jsonpath="{.items[0].status.availableReplicas} {'\n'}"

    Checking service instance
    When you create the service instance, you will get the custom resource name. Check the status of the custom resource using command below
    oc get <custom-resource-name> -o yaml

    ------------------------------
    Deepak Rangarao
    ------------------------------



  • 49.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:57 PM
    watch is a great utility that runs the command specified at regular interval (2 seconds). tac is another utility reverses the order of display; last line shows as 1st line, etc. tac is reverse of cat in Unix/Linux. You can add --sort-by=.status.startTime to certain oc get commands to order the objects as they were created. With this knowledge, you can execute following commands to monitor progressAssuming you have created all the needed mini-catalog sourcesUse below command in one terminal window to monitor pods as they instantiate (with latest pod created being at the top) in the CPD project namespace. Below in the example we assume zen,
    watch "oc get po -n zen --sort-by=.status.startTime | tac"
    Here is a variation to only monitor pods that are not complete or not ready yet
    watch "oc get pods -n zen --sort-by=.status.startTime | grep -Ev '1/1|2/2|3/3|4/4|Completed'"
    If it is a complex service like Watson Knowledge Catalog that uses ODLM / CASE Dependency, use below command to additionally monitor the Operators that are created as part of the above process using
    watch "oc get po -n ibm-common-services --sort-by=.status.startTime | tac"
    If CPD Operators are in a different namespace than Bedrock (Specialized install), specify that for namespaceAnd you can go a step further by monitoring the logs of the active operator pod from above list
    oc logs <Operator Pod> -n ibm-common-services -f
    And don't forget to confirm the CR is showing Completed status when done.


    ------------------------------
    Amitabha Das
    ------------------------------



  • 50.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 10:31 AM
    Question about deployment: I like the simplicity the IBM Operator Catalog brings in deploying Cloud Pak for Data. Can I recommend this to our Enterprise customers. If not, where does it make sense to use it?

    ------------------------------
    SHARYN RICHARD
    ------------------------------



  • 51.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:07 PM
    Yes, using the operators/operator catalog is the recommended approach to installation. Operators bring simplicity and consistency to the installation process.

    ------------------------------
    Deepak Rangarao
    ------------------------------



  • 52.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:48 PM

    The IBM Operator Catalog is simple to get access to all Operators from all Paks - so is very convenient. 

    However, some care is also needed (apart from familiarity with OLM). This is because any changes to the IBM Operator Catalog also typically upgrades all Operators automatically with any new versions and any un-pinned Operands as well. 

    So - if you have a requirement to freeze (pin) versions to a specific level, you would be better off using the cloudctl mechanism to  introduce specific versions  of the CPD Services' operator catalog sources.  There are other alternatives - including switching all install plans to manual.



    ------------------------------
    Sriram Srinivasan
    ------------------------------



  • 53.  RE: Questions for AMA: Setting up Cloud Pak for Data

    Posted Thu November 04, 2021 02:43 PM
    Enterprise customers need all their environments to be at the same service versions as they progress through their validation sequence before finally deploying the same service versions on their production environment. They normally do their initial functional validation on a small development cluster. Once satisfied, they would want to deploy the same service versions in their pre-production/QA environment before deploying on their production environment. Cloud Pak for Data has been and is planned to release updates on a regular monthly cadence to address customer needs and provide latest security fixes. This breaks the need for customers to have the ability to deploy the same version of the services over time as IBM Catalog Operator will always point to the latest versions of the individual services.Another factor to consider is enterprise customers would want to only deploy service images on their environment that have passed their security scans. With IBM Operator Catalog they have no control of which images they have scanned and validated for deployment in their environments.So where can I use IBM Operator Catalog?
    1. For POCs. These are short lived environments where you can demonstrate the capability of the services. And where customers can play and learn about the innovative capabilities we provide
    2. For customers who do not have enterprise customer like needs for ensuring all their environments are at the same service levels. They want the simplicity of auto-updates to keep each of their environments at the latest level. They also do not have a need to scan the images for security issues before they are deployed on their environments.


    ------------------------------
    Amitabha Das
    ------------------------------