Cloud Pak for Data

 View Only
Expand all | Collapse all

Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

  • 1.  Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Tue August 09, 2022 05:17 PM
    Edited by System Fri January 20, 2023 04:11 PM

    We'd like to answer your questions about the operational improvements we made in Cloud Pak for Data 4.5 in the areas of installation, upgrade, backup, and restore. We've arranged for experts from across IBM to answer your questions right here in this forum thread.

    If you have questions, you can start posting them now as a response to this post. Our experts will hop on the Cloud Pak for Data Community discussion forum on August 25 at 2 pm Eastern/11 am Pacific and start answering your questions right here in this same thread.

    To learn more, or to add a reminder about this AMA to your calendar, go to the  Day 1 and Day 2 operational Improvements in 4.5  event page . This event will take place entirely in the discussion forum, so there is no meeting to join. If you can't be online during the hour, don't worry; you can post your questions in advance and read the responses later.



    ------------------------------
    Shannon Rouiller
    Content strategist, Cloud Pak for Data
    ------------------------------


    #CloudPakforDataGroup


  • 2.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Wed August 10, 2022 05:18 PM
    Edited by System Fri January 20, 2023 04:11 PM

    Hi team,

    Based on the release notes, backups/restores was a very much needed ability. Glad it's here. I have a few general questions about the release:

    In v4.5 have all facets of migration been covered? What I mean is it looks like in place upgrades are covered, but how about going from a legacy version to a clean install? Is migration a complete feature for moving data artifacts and other data? I know in the past we had to script solutions to import/export.

    How have upgrades been improved? Are there notifications if older configurations or setups are not compatible with the new product version? One of the biggest challenges is to identify how close or far your configuration is to the out of the box setups. The reason this is important is you may want to know if an old configuration is disabling or modifying new functionality you may want to use, and also because your platform may not align to the documentation.

    I know that was many questions in one, but look forward to learning more. Thanks!



    ------------------------------
    Shawn Lunny
    ------------------------------



  • 3.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Thu August 25, 2022 02:38 PM
    Hi Shawn,

    Migration from legacy software to IBM Cloud Pak for Data services is not broadly available. 
    A few services have information on how to migrate from the legacy software to the IBM Cloud Pak for Data offering. For example, Watson Knowledge Catalog offers guidance on Migrating data from IBM InfoSphere to IBM Cloud Pak for Data

    Are you looking for information on migrating from a specific product?


    ------------------------------
    JULIA Montarbo
    ------------------------------



  • 4.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Thu August 25, 2022 02:42 PM
    Regarding the second half of your questions, there is no incompatibility reported in this version of CPD. There are some migration process but they are automated into the upgrade flow. So no "notification" is needed or implemented on this front.

    ------------------------------
    Kaihua Zhou
    ------------------------------



  • 5.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Thu August 11, 2022 10:45 AM
    What are the operational improvements we made in Cloud Pak for Data 4.5 in the areas of installation, upgrade, backup, and restore ?

    ------------------------------
    Tom Kochie
    ------------------------------



  • 6.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Thu August 25, 2022 02:09 PM
    Hi Tom,

    For installation and upgrade
    , the improvements are focused on:
    • Reducing the number of steps that it takes to install or upgrade by. The cpd-cli allows you to batch tasks. For example, you can run one command to create the custom resources for all of the components that you plan to install.
    • Automating multi-step, error-prone tasks, such as creating operators. The cpd-cli runs an Ansible playbook to ensure that each step is complete before it moves to the next step. For example, the playbook ensures that the catalog source exists before it creates the operator subscriptions.
    • Simplifying the process of mirroring images to a private container registry. The new process includes verification steps so that you don't encounter unexpected failures because you don't have access to an image or because an image wasn't mirrored as expected.

    I hope this helps!

    Best,
    Julia

    ------------------------------
    JULIA Montarbo
    ------------------------------



  • 7.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Thu August 25, 2022 02:22 PM
    Edited by System Fri January 20, 2023 04:37 PM
    Thanks for the question Tom: 

    Backup and Restore improvements are focused on: 
    • Supporting CPD Backup's without quiescing or interrupting deployed Services.
    • CPD Online Backup & Restore leverages Container Storage Interface (CSI) Snapshots to capture Volume Data while Services and Service Instances remain functional/responsive which eliminates down time for critical deployments.
    • The IBM Knowledge Center for CPD 4.5 provides detailed break down of the various Backup & Restore approaches and Service support.

    ------------------------------
    Andy Streit
    ------------------------------



  • 8.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Thu August 25, 2022 02:42 PM
    See Backing up and restoring Cloud Pak for Data in IBM Documentation for the breakdown of approaches.

    ------------------------------
    JULIA Montarbo
    ------------------------------



  • 9.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Tue August 16, 2022 09:41 AM
    Are there any plans to run these at EMEA friendly times? Currently the session is due to start 14:00 EST, which is 19:00 UK time and probably later for rest of Europe

    ------------------------------
    Raj Kerai
    ------------------------------



  • 10.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Tue August 16, 2022 12:10 PM
    Good question, Raj. The timing of the AMA should not be a problem because there is no meeting to join (the whole thing takes place in this forum post). We set up the AMA so that you can ask your questions in advance.  The time that is set for the AMA is the time that we start answering the questions, but the answers will be available whenever you want to view them.  I hope this helps!

    ------------------------------
    Shannon Rouiller
    Content strategist, Cloud Pak for Data
    ------------------------------



  • 11.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Wed August 17, 2022 09:34 AM
    Hi experts,
    Is it possible to backup/restore just parts of CP4D?
    For example, backup/restore particular Projects, Catalogs and its assets.
    If possible, I would like to know how.
    If not possible, I would like to know if there is any plan for such features.

    ------------------------------
    NAOKO TAKAYA
    ------------------------------



  • 12.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Thu August 18, 2022 12:56 PM
    Have you seen this cpd-cli/cpdtool at master · IBM/cpd-cli ?
    This is suppose to be provide a unified export import capability across services. It however needs work from various teams and may not cover your usecase. Suggest you to open an enhancement request

    ------------------------------
    SACHIN PRASAD
    ------------------------------



  • 13.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Thu August 25, 2022 02:50 PM
    Edited by System Fri January 20, 2023 04:46 PM
    1) Is it possible to backup/restore just parts of CP4D? For example, backup/restore particular Projects, Catalogs and its assets.
    No, this is not supported.

    2) If not possible, I would like to know if there is any plan for such features.
    No plans using backup/restore.  This could be supported using export/import, but not many CPD services support it.

    There is common export/import framework using cpd-cli, but few CPD services support it:
    Services that support the metadata export and import utility

    Individual CPD services may also document service-specific export steps, such as:

    ------------------------------
    Henry Chiu
    ------------------------------



  • 14.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Fri August 19, 2022 02:14 PM

    Hi team,
    Thanks for doing this AMA.

    I like the idea of a simplified installation experience, but I also want to know what changes are being made to my cluster. How can I have more insight and control over the changes to my cluster?

    Thank you!



    ------------------------------
    Karin Moore
    ------------------------------



  • 15.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Thu August 25, 2022 02:15 PM
    Hi Karin,

    Thanks for your question.

    Some of the cpd-cli manage commands include a --preview option that you can use to understand the oc commands the cpd-cli will run on your behalf. 
    The oc commands are saved to the preview.sh file in the cpd-cli-workspace/olm-utils-workspace/work directory. 
    You can optionally copy the oc commands from the output and run them yourself. However, this method is not recommended. When you run the commands manually, you do not have access to the additional helper scripts that are included in the underlying Ansible playbook.
    For more information about which cpd-cli manage commands support the --preview option, see the cpd-cli manage command reference.

    Best,
    Julia


    ------------------------------
    JULIA Montarbo
    ------------------------------



  • 16.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Fri August 19, 2022 02:24 PM

    Hi folks,

    I'm interested in knowing more about the main improvements for the air gapped install and upgrade experience?

    Thanks



    ------------------------------
    [Trish] [Smith] [MBA, BMath, Mom]
    [Content Developer]
    [IBM]
    [Ottawa] [ON]
    [613-356-5435]
    ------------------------------



  • 17.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Thu August 25, 2022 02:30 PM
    1. The new cpd-cli manage provide commands to set up global pull secrets and image content source policy directly, without editing some yaml template.
    2. The image mirroring process for multiple components can now be grouped together, with retry options for improved resiliency


    ------------------------------
    Kaihua Zhou
    ------------------------------



  • 18.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Thu August 25, 2022 02:59 PM

    The process for mirroring images is much more straight forward in Cloud Pak for Data 4.5.

    • Previously, you had to manually download the CASE packages for each service that you wanted to install. However, in 4.5, if you use the COMPONENTS environment variable to specify a comma separated list of all of the components that you want to mirror to your private container registry, you can download the CASE packages in a single command.

    • In addition, the process can be completed in a few simple steps, including validation steps that help you avoid unexpected issues because you cannot access an image in the IBM Entitled Registry or because an image was not mirrored to your private container registry.

    • If you cannot connect to both the internet and to your private container registry at the same time, the cpd-cli automatically sets up an intermediary container registry for you. For details, see Mirroring images using an intermediary container registry in the product documentation.


    ------------------------------
    JULIA Montarbo
    ------------------------------



  • 19.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Tue August 23, 2022 02:37 PM
    Hello team,

    What versions of Cloud Pak for Data can I upgrade from? For example, if I'm running IBM Cloud Pak for Data Version 3.5, can I upgrade directly to IBM Cloud Pak for Data 4.5.x?

    Thank you.


    ------------------------------
    Jennifer Smith Gray
    ------------------------------



  • 20.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Thu August 25, 2022 02:30 PM
    Hi Jennifer,

    You can upgrade to IBM Cloud Pak for Data Version 4.5.x from:
    • Version 3.5
    • Version 4.0
    Ensure that you are running the minimum required refresh on each version. For details, see:


    ------------------------------
    JULIA Montarbo
    ------------------------------



  • 21.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Wed August 24, 2022 11:45 AM
    Hi guys.
    Very good initiative on this thread.
    I am happy to be able to talk to people responsible for the product.

    I have already used/installed version 3.0, 3.5 and 4.0. I havent used v4.5 due to the inexisting CP4D ROKS schematics. 
    Why this version is not yet available?
    I havent much to ask about v4.5 because I havent yet installed, but I have some general questions about v4.0.

    About the backup and migration procedures (using cpd-cli and cpdtool): I have already tryed to use on v4.0, but found some errors on the documentation that forced me to give up at the time. I found a little bit confused.
    By the way, documentation must be improved. Lots of errors or lacking of procedure information ...

    I appreciate the use install variable script (cpd_vars.sh). It facilitates a lot the execution of the procedures!

    I have some sugestions on the migration/backup/restore procedures, tools and documentation on:
    1 - How execute on users, groups and roles (access control)?
    2 - How to execute on a specific service (eg. WKC. DV. WS)
    3 - How to execute on specific parts of a service (eg. One Catalog/Project from WKC. All catalogs/Projects from WKC. All data classes. One data class. All Platform conections.... etc etc)
     I know that is already possible to import/export data classes, but I have some problems during import due to the Catagory, Business terms and Custom attibutes issues (it seems to be necessary to have the category created before. Also the business terms. Also the custom attributes...). So, why dont do this indicating a correct order to import? Or even improving the import tool?
    4 - How to execute on an entire namespace?
    5 - How to execute on an entire cluster?
    6 - etc etc

    I think the tools could offer options for an admin that need to backup everything in an easy procedure, or the admin that needs to execute on specific parts for migration purposes (to a remote object store or locally)


    Again, I dont know the improvements on v4.5 (maybe all my sugestions have already been applied). In this way, sorry for that ...

    One point of the product that I miss is an architecture vision that cross-references information from PODs, CRDs, etc with information from cartridges, services and functionality. This information would be useful to analyze issues and execute the problem determination.

    E.g.: If I am having trouble logging into CP4D, which PODs should I look at? If I have a problem with Data Discovery, which PODs are responsible for storing the IADB? And so on...

    I have a lot of more questions, but I will leave them for later... kkkk

    Thanks in advance!!!

    Igor Franco
    Pre Sales specialist
    MD2 Consultoria (Brazilian BP)
    https://www.linkedin.com/in/igorsilveirafranco/

    ------------------------------
    Igor Franco
    ------------------------------



  • 22.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Thu August 25, 2022 02:26 PM
    Hi Igor,
    Some of your questions are a little off-topic for this AMA, but we'll answer what we can. (We might not have the right experts here to answer all of your questions.) You can also post a question on the community at any time. We do our best to monitor the posts and respond.

    I'm not quite sure I understand what you mean when you say the "CP4D ROKS schematics". Do you mean that the IBM Cloud Catalog entry for IBM Cloud Pak for Data still points to Version 4.0.x? I believe that work is underway on this, but I don't know when it will be available.

    However, you don't need to wait for the catalog to be updated. Cloud deployment environments
    in the product documentation provides information on the different ways you can install IBM Cloud Pak for Data on your cluster. 
    Regarding errors in the 4.0 documentation, I encourage you to use the Feedback option at the top of the page when you run into problems. We are always eager to get concrete feedback about places where you ran into problems. 
    I'm very glad to hear that you are using the environment variables script. We're using it in a lot more commands in the 4.5 documentation. 


    ------------------------------
    JULIA Montarbo
    ------------------------------



  • 23.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Mon August 29, 2022 11:02 AM
    Hi Julia.
    Thanks for the answers.

    Yes, since the announcement of CP4D 4.5, it does not exist Cloud pak for Data v4.5 in IBM Cloud catalog. The most recent product version option is v4.0.8.

    Ok. I will make a full v4.0. backup and try to update to v4.5 using the procedures listed on the documentation.

    Best Regards

    ------------------------------
    Igor Franco
    ------------------------------



  • 24.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Wed August 24, 2022 03:23 PM
    Hi team! 

    Supporting Online Backup & Restore will significantly reduce/eliminate the time CPD is unavailable during Backup.  However, why isn't "Restoring to Different Cluster" currently supported with Online Backups?  

    Thanks!

    ------------------------------
    Areej Ansari
    ------------------------------



  • 25.  RE: Questions for AMA: Day 1 and Day 2 operational improvements in 4.5 (installation, upgrade, backup, and restore)

    Posted Thu August 25, 2022 02:29 PM
    Areej,

    Good question as the Online Backup & Restore currently has some additional requirements/restraints compared with Offline Backup & Restore.

    Online Backup & Restore exclusively leverages Container Storage Interface (CSI) to capture Volume Data to local storage Snapshots.  CSI Snapshots are significantly faster than Restic File backups, however the Snapshots do not natively support relocation.  Consequently, the ability to relocate the CSI Snapshots to secondary location for restoration to a different cluster is not yet available. 

    CPD is pursuing several alternatives including: data mover technology, storage replication and storage specific solutions to align with for future relocation support.

    Thanks,

    ------------------------------
    Andy Streit
    ------------------------------