We are thrilled to announce the general availability of Cloud Pak for Data version 5.0, marking a significant milestone as our 15th feature release. Since its inception in 2018, when IBM envisioned a transformative Data and AI platform, Cloud Pak for Data has experienced exponential growth. Today, it boasts thousands of customers, over 60 integrated services, and a dedicated user base that values its unified experience.
Amidst the dynamic economic landscape, our team has remained steadfast in fortifying the platform’s core, ensuring it meets enterprise-grade standards with robust Day 1 and Day 2 capabilities. These enhancements include disruption-free operations, disaster recovery solutions, enhanced serviceability and observability, compliance/security measures, and essential industry certifications.
While maintaining a strong focus on Day 1 and Day 2 operations, Cloud Pak for Data also plays a pivotal role in IBM’s broader strategy. It strives to become the preferred delivery mechanism for IBM containerized products in on-premises environments. Version 5.0 is a testament to this vision, paving the way for future innovations and reinforcing IBM’s commitment to providing resilient, scalable, and efficient solutions to meet the evolving needs of businesses worldwide.
Cloud Pak for Data version 5.0 brings two key features in that direction: Remote Data Planes and Immersive Experience, along with new functionality and feature enhancements. Remote Data Planes provide the capability to extend the platform beyond the traditional OCP cluster to multiple clusters both on-premises and in the cloud as one instance. Immersive Experience facilitates the co-existence of watsonx and Cloud Pak for Data services on the same OpenShift cluster in the same namespace streamlining IT operations, with unified and consistent deployments and Day 2 operations. Each offering has its own branding and perspective based on the available capabilities.
Remote Data Planes
This is a brand-new feature enabling customers to broaden Cloud Pak for Data capabilities by deploying lightweight yet robust data planes beyond one OpenShift Cluster.
What is a data plane?
A data plane provides a logical organization of physical locations into groups to extend the platform in one instance. Remote data planes will enable a cohesive extension of data and AI services, with the DataStage service as the first offering in v5.0, across diverse clouds, geographical regions, and on-premises environments, achieving a harmonized usage and management experience. This new framework allows workload definitions to be created on the Cloud Pak for Data control plane and then deployed to the remote data planes. The remote data plane could be in another data center or on a hyper-scaler.
It is crafted to consolidate, expand, and refine workloads throughout on-premises and multiple cloud ecosystems, prioritizing compliance, performance, and cost-effectiveness. This allows workloads to be moved closer to where the data resides. Moving the workload closer to the data source provides not only data gravity but also data sovereignty to meet compliance and regulations.
Remote Data Plane also brings a Tech Preview feature called Bring Your Own App (BYOA) which enables the deployment and integration of custom workloads on Remote Data Plane, allowing organizations in future to achieve greater compatibility and seamlessness with Cloud Pak for Data Services. This approach ensures that custom workloads can leverage the same monitoring, auditing, and management practices as other Cloud Pak for Data services.
For more information about remote data planes read here, and check out the blog.
Immersive Experiences
Facilitates watsonx and Cloud Pak for Data integration within a single OpenShift namespace, streamlining IT operations and maintenance post-installation. Immersive experience ensures distinct user experiences for each brand, despite shared namespace installations. This is achieved using perspectives that are launched based on the cross product features. Under the covers, Node Pinning is used to enhance license adherence for better compliance and chargebacks.
For more information about immersive experiences read here, and check out the blog.
Infrastructure
Hosted Control Plane
The Hosted Control Plane (HCP) is a centralized management system to govern entire Kubernetes clusters offering significant cost savings and footprint reduction. It adds to the security of a cluster by providing a single point of access for control which also helps with administrative tasks and logistics. With the introduction of HyperShift, Cloud Pak for Data v5.0 extends the HCP concept to HyperScalers, in this case to AWS. HyperShift separates Control Planes from worker nodes, replacing multiple individual clusters with a single shared control plane across all clusters. This innovation simplifies cluster management and improves scalability and efficiency.
Read here for more information about Hosted Control Planes.
Fusion HCI
IBM Fusion HCI was the industry's premier on-premises system using OpenShift that has evolved to support and enhance OpenShift deployments. This system was first introduced in 2021, and now the Fusion HCI appliance is advancing to support OpenShift's new Hosted Control Plane technology. These benefits can now be extended to Cloud Pak for Data on-premise deployments. In addition to supporting decoupled control planes, Fusion introduces Provider Mode, which centralizes storage and supports multiple hosted clusters through HCP. Clusters requiring storage are automatically provisioned with storage classes backed by a centralized storage hub, significantly reducing Day 2 activities.
For more information read here.
Google OpenShift Dedicated
Cloud Pak for Data has been certified on the following managed OpenShift offerings for IBM Cloud, AWS, and Azure. This will now expand to include a managed OpenShift offering on Google Cloud Platform: Google OpenShift Dedicated. Similar to the other managed OpenShift offerings, Red Hat OpenShift Dedicated takes advantage of the native Google Cloud services. This certification provides more options and flexibility for Cloud Pak for Data hybrid cloud deployments.
Read here for more information about Google OpenShift Dedicated.
Single Node OpenShift
The introduction of Cloud Pak for Data on Single Node OpenShift offered in previous releases, targeted for smaller, fault-tolerant deployments. This setup is perfect for managing proofs of concept (PoCs), and building MVPs in a cost-effective manner. Initially available only for Cloud Pak for Data Express offerings, this is now extended to all services, including base services and cartridges, maintaining the same value and functionality. Single Node OpenShift is particularly beneficial for short-term projects, development, and testing environments. It's also ideal for edge deployments, reducing latency, bandwidth, and costs.
Seviceability
Telemetry
Cloud Pak for Data 5.0 emphasizes the critical role of telemetry in enhancing the customer experience. Customers are encouraged to consider enabling the one-click solution to gain deeper insights into software usage and license compliance. The latest updates extend telemetry coverage to include additional technical and health indicators, aiding IBM in delivering superior customer experiences across various touch points such as support and product management. The key data points collected to better serve our customers include:
-
VPC Consumption
-
CPU & Memory Consumption
-
CPD, OCP, and Operator installed including version information.
-
General health-related metrics
The following is a sample of the IBM Cloud Pak for Data newsletter tailored with customers information
For more information about telemetry services read here, and check out the blog.
IBM AssistME
In-app assistance or AssistMe provides uniform in-app contextual assistance for an increasing number of IBM products, which now include Cloud Pak for Data v5.0. It can be invoked via the '?' icon from the top banner, and it shows a list of recommended articles that are applicable to the displayed content. Close and open the assistance panel when you move to a new page to see updated recommended articles. You can also enter search terms to find information that is scoped to the relevant release. The assistance panel also includes additional support links to access the community, review and submit enhancement requests, videos, documentation as well as links to open a Support case.
Service Functionality Checks
Cloud Pak for Data Platform v5.0 comes with new Service Functionality Checks, where assessments are conducted, in a scaled down manner to test the operation of the service. Depending on the user and service, Service Functionality Checks are user-initiated as an on demand operation. The purpose of Service Functionality Checks are to provide administrators with a variety of reliable insights for services including service usability, pre/post upgrades, and also act as a troubleshooting tool to isolate problems. The checks would only be conducted on test data in order to prevent a client’s genuine cluster assets from being impacted. The checks then return detailed reports of validation or failure of the products tested. To find out more, and for a complete list of services that are leveraging Service Functionality Checks, check here.
Usability
Management Services
Management Services is an evolution of the Cloud Pak for Data Command Line Interface to streamline and simplify the cpd-cli manage commands. This new framework provides a set of RESTful API endpoints to manage and maintain the platform and its services. Currently, these tasks are handled by the cpd-cli manage commands, which are versatile and capable, and now with management services Cloud Pak for Data administrators can use a declarative method to perform these tasks through these new APIs. In version 5.0 install and upgrade, along with service shutdown and restart will be available with more administrative tasks to follow in future releases.
Compliance and Security
The need for Security and Compliance is always paramount to protect your assets and sensitive information from getting into the wrong hands. So, here comes Cloud Pak for Data v5.0 which will continue to stand on its foundation to support various security and compliance features while enhancing your experience. Cloud Pak for Data is committed to securing your application data, eliminating any systems vulnerability, and providing seamless access to your data. Cloud Pak for Data has recently delivered CIS and FIPS compliance features to enhance our security capabilities. For more details review regulation and compliance section in the documentation..
Building upon its existing strong security and compliance foundation, Cloud Pak for Data v5.0 is introducing new features to enhance our systems. Cloud Pak for Data v5.0 now supports HIPAA compliance, AWS GovCloud, and Dual IPv4/IPv6 Stack that will make your experience more secure and reliable.
HIPAA
Cloud Pak for Data v5.0 now simplifies adherence to HIPAA compliance which is a set of standards protecting the privacy and security of electronic protected health information (ePHI) by U.S. Department of HHS applicable to healthcare organizations, and cloud-hosted company that meets the definition of a covered entity. Cloud Pak for Data v5.0 is committed to assist customers with know how to achieve HIPAA compliance by following a set of guidelines. Read more about Cloud Pak for Data considerations for HIPAA readiness.
AWS GovCloud
Cloud Pak for Data v5.0 now expands its deployment options to include highly secure AWS GovCloud which gives government customers and their partners the flexibility to architect secure cloud solutions that comply with various compliance regimes.
Dual IPv4/IPv6 Stack
Cloud Pak for Data v5.0 takes a step forward in network compatibility by supporting FISMA Dual Stack IPv4/IPv6 with ipFamilies priority set to IPv4 (which is also the default). This will enhance the network security capability of your clusters. This step has put forth a foundation for clusters with ipFamilies priority set to IPv6 for our future versions.
Connectivity
Accessing data sources is an important component of Cloud Pak for Data, which requires providing connectors and enhancements. Cloud Pak for Data supports over 100 connectors and various formats, with the addition of using generic JDBC to build custom connectors. In addition to using generic JDBC, the Connector SDK can be used to build custom connectors. The Connector SDK uses the Arrow Flight Framework, which is a general-purpose client-server framework to simplify high-performance transport of large datasets over network interfaces. In addition to these enhancements and new connectors, a new feature was introduced to aid in the management of data and connection assets: Data Source Definitions.
Data Source Definitions
Data source definitions help to properly enforce data protection solutions and improve the management of data and connection assets. This allows you to document all endpoints associated with a given data source instance, as well as any deep enforcement solution that may be configured with the data source. Once a data source definition is created, it is assigned to all connections and connected data that is associated with that data source.
Learn more about data source definitions here.
HTTP Proxy
HTTP Proxy Support in Cloud Pak for Data v5.0 facilitates remote access in a secure environment acting as a bridge between remote users and internal network resources. Proxies help safeguard sensitive information from unauthorized access and provides protection from cyber attacks. It does this by authenticating users and encrypting data transmission. For ease of use for Cloud Pak for Data 5.0 users, the cpd-cli-utility has been enhanced to support the deployment of HTTP Proxy across all supported services by executing two easy commands in Create and Enable. Deploying an HTTP Proxy server and configuring it correctly is a significant step in preventing cyberattacks and data breaches.
Learn how to configure and use HTTP Proxy here.
Product Support, Lifecycle, and Red Hat OpenShift Compatibility
Product Support and Lifecycle
Cloud Pak for Data v5.0 will adhere to Support Cycle - v2 (2 + 1 + 3). This means providing two years of base support, followed by one year of initially extended support for critical defect and security fixes, and an additional three years of ongoing support for usage and known issues.
IBM will continue releasing defect fixes and CVEs on the 4.8.x track, ensuring at least a year of overlap to give our customers ample time to explore and upgrade to v5.0. The upgrade process remains seamless and follows the established pattern. Our team is dedicated to assisting customers as they transition from 4.x to 5.x, providing comprehensive support throughout the process.
Red Hat OpenShift Container Platform Support
Cloud Pak for Data now supports all versions of OpenShift within 30-45 days of the general availability. This significant enhancement ensures that customers can continuously upgrade their underlying operating system and OpenShift Container Platform to meet their company’s security mandates. It also enables customers to deploy Cloud Pak for Data on fully managed OpenShift platforms on hyperscalers without compatibility concerns.
For customers planning to stay on v4.x longer, it is advisable to review the OpenShift Lifecycle to determine the best course of action to continue receiving support from Red Hat. Some of the releases provide “Extended Update Support Term 2”
Upgrade to version 5.0
Cloud Pak for Data v5.0 supports direct upgrades from v4.7 and v4.8, with specific steps related to OpenShift version compatibility. Please refer to the upgrade guide for detailed information. As v4.8.x will continue to have further releases, customers on v4.8.7 and beyond must upgrade to the latest GAed version of Cloud Pak for Data v5.x to ensure smooth upgrades. The team unfortunately cannot certify such upgrades retroactively on the already released versions of v5.x.
To learn more about the new features and functionality for the IBM Cloud Pak for Data platform and services check out the What's New section in the documentation.
For more information check out these blogs:
Remote Data Planes
Immersive Experience
IBM unveils IBM Cloud Pak for Data 5.0
#Featured-area-2-home
#data-spotlight
#Spotlight
#Highlights-home
#data-highlights-home