By: Sairaman K and Nandhinee Dhevi Ramachandran
Introduction:
IBM Db2 Bridge is a data movement tool designed to support a wide range of enterprise use cases, including infrastructure modernization, database workload migration, system consolidation, and cloud-native application development. It provides a unified solution for managing data transfers across diverse environments.
This post introduces IBM Db2 Bridge and explains how it facilitates a seamless and efficient migration from IBM Integrated Analytics System (IIAS) to IBM’s Power10 Private Cloud Rack (P10 PCR). It outlines how the tool supports the entire transition—from initial planning through to execution.
What is IBM Db2 Bridge?
IBM Db2 Bridge is a data movement tool designed to support a wide range of migration and data transfer scenarios encountered in enterprise environments. It can be used in any context where an ODBC connection is available. The tool is built to accommodate changes in operating systems, hardware architectures, and deployment topologies that often occur during system upgrades.
IBM Db2 Bridge employs various methods to enable users to switch between different Db2 instances with minimal effort.
Key features of IBM Db2 Bridge:
- Platform agnostic by utilizing ODBC connectivity, IBM Db2 Bridge connects directly to source and target systems and is not impacted by operating system differences such as endianness and Topological difference of the databases.
- Preserves the integrity of data by analysing dependencies. From a source system, the user can select an entire database, or a subset of objects and Db2 Bridge manages the movement of all the objects and the dependency required to move the user selected object
- Supports Db2-based deployments. IBM Db2 Bridge supports various transactional and analytical workloads and deployment types on premises or on cloud. Note: IBM Integrated Analytics System (IIAS) is supported as a source system only.
- The interface includes both graphical and command-line options that support tasks such as defining the source and target, selecting objects from the source system, running the migration job, and viewing the results.
Architecture of IBM Db2 Bridge:
IBM Db2 Bridge provides both a graphical user interface and a command-line interface for performing data movement tasks. It uses ODBC connectivity and Db2 Catalog functionality to support various migration and data transfer scenarios.
Figure 1: Db2 Bridge Graphical User Interface
Since data movement can take anywhere from a few minutes to several days for large databases, it is essential for Db2 Bridge to operate using a client-server architecture. This allows users to run data movement tasks independently, without relying on external tools to manage connections and sessions.
Once a request is sent from db2bridge_client (CLI) or Web UI to the Db2 Bridge Server, Db2 Bridge does a lot of things in the backend to make the data movement smooth for the users including, but not limited to:
- Verifying the existence of the objects provided by the users and identifying their dependencies.
- Running pre-checks to verify the configuration provided by the user can be run on the source and target databases.
- Determining the strategies supported by the Source and target database for data movement.
- Migrating objects at an individual level while keeping dependencies intact in the process.
- Verifying the migration status by running some post checks and data validations if requested by the users.
Db2 Bridge supports various strategies for data unload and data load as part of Table movement. The strategy is selected efficiently by Db2 Bridge depending on the type of table, the column types and other factors. The following three strategies exist with each strategy having its own advantage:
· External Table strategy
- This is the default strategy used by Db2 Bridge, as internal evaluations have shown it to be the most effective in typical scenarios.
- This Strategy will be used in all table types except for typed tables, or if the table has a BLOB/CLOB/XML Columns.
- This Strategy has also been optimized to avoid using disk space, enabling users to migrate large databases without requiring additional storage for temporarily unloading data to files.
· Catalog DB Strategy:
- When dealing with a typed table or a table containing BLOB/CLOB/XML columns, this strategy is automatically used by Db2 Bridge for data movement.
- This Strategy internally uses Db2 Catalog, Db2 Export and Db2 Import Utilities and all of these are hidden from the users as everything is taken care off by the tool.
- Because the data includes BLOB, CLOB, and XML types, disk usage during the migration process is kept minimal. Temporary storage used during data movement is automatically cleaned up by Db2 Bridge upon completion of the migration.
· HPU Based Strategy:
- Any Users who already have IBM Optim High Performance Unload (HPU) can choose to use HPU for unloading data directly from a backup file.
- This strategy helps with data movement using backup directories which reduce the impact on the source machine to near zero.
- When doing a bulk data movement (at schema or database level), Db2 Bridge takes care of configuring HPU and users just need to provide the HPU installation path.
Figure 2: Db2 Bridge Data Unload & Data Load Strategies
Where to Install IBM Db2 Bridge?
IBM Db2 Bridge can be installed in any server that has ODBC connectivity to source and target Database. This makes Db2 Bridge versatile in any Migration/Data movement scenario that the users might come across.
Currently Db2 Bridge can be installed on Red Hat Enterprise Linux (RHEL 8) with support for x86 and ppc64le architecture. RHEL 9 and support for more CPU architecture is something that will be available in future releases.
For migrations from IIAS to P10CR, there are three distinct methods for installing Db2 Bridge. The following section analyses the benefits and limitations of each option.
1. Db2 Bridge installed on target machine:
Installing Db2 Bridge on IBM’s P10 PCR gives a user the ability to use the latest hardware for their migration from IIAS. Installing Db2 Bridge also comes with a host of benefits like:
- Reduction in network hop compared to installing Db2 Bridge on an intermediate machine that reduces the network bandwidth to half.
- It has minimal impact on the source machine, allowing the business-critical applications to continue operating with little to no disruption.
- Newer hardware like the P10 PCR comes with fast storage that can be useful if Db2 Bridge needs to use the disk as part of data unload
Figure 3: Installing Db2 Bridge on the Target System
2. Db2 Bridge installation on the Source cluster:
This installation method is similar to the previous one, with the main difference being that Db2 Bridge is installed directly on the source IIAS machine. This configuration is particularly suitable when a backup file is used as the data source for migration. Since the backup file is already located on the IIAS machine, no additional step is required to transfer it to the Db2 Bridge installation directory.
This option also offers the same benefit of reduced network hops compared to installing Db2 bridge on an intermediate machine. However, it may impose a significant load on the CPU, potentially leading to performance bottlenecks on the source IIAS machine.
Db2 Bridge can be installed on the dash DB container which reduces the network utilization and provides access to backup files without the need for any additional configuration updates.
Figure 4: Installing Db2 Bridge on the Source System
3. Db2 Bridge installation on an intermediate machine:
Whenever it is not possible to install Db2 Bridge on either the P10 PCR or IIAS, Db2 Bridge can be installed and configured on a intermediate server. The only requirement for this kind of installation is to verify whether there is ODBC connectivity between the servers.
This installation allows the users to run migrations with very little CPU overhead on the source(IIAS) and target(P10CR) machines . If the backup file resides on this machine, data movement for tables can be performed without requiring a connection to the source system. Note: A connection to the source system is required to gather other information required for migration.
The only disadvantage with this approach is that the available network bandwidth for the data migration process is split in half due to the network connection between the source and target.
Figure 5: Installing Db2 Bridge on an Intermediate Machine
Migration Planning:
Db2 Bridge supports migration of objects at Database level or at a schema level even down to an individual object. This allows the users to plan their migration based on the system availability and usage such that migration can take place without much impact to the source infrastructure.
Users can migrate their whole database using Db2 Bridge. In that scenario the user must mention the source and target environrment information and just add --whole-db as argument or select all the objects with a click of a button in the UI. The product migrates all the supported objects from the source to target and displays the migration status in an intutive manner.
Similar to whole Database migration, the users can migrate at a schema level using the --whole-schema flag and Db2 bridge would extracts the supported objects within the schema along with its dependencies and referential integrity for seamless migration.
Db2 Bridge supports migrations of the following objects: Tables, Views, Indexes, Triggers, Comments, Sequences, Procedures, Functions, Aliases, Types, Constraints, Global Variables, Modules which are schema specific objects and Security Policy Labels, Security Labels, Bufferpools, Tablespaces, Histogram templates, Service classes, Audit policies, Roles which are schema independent objects.
Configurations:
Db2 Bridge provides options to configure the parameters so that a users can tweak the migration performance based on their need. It includes:
- number of parallel data streams for the migration,
- whether the data needs to be transferred in binary format,
- The compression (LZ4, GZIP) to be applied for efficient data migration,
- ability to filter data within the table for migration,
- hpu related configurations.
- An option to specify whether the target table should be truncated before loading data into it.
Validations:
Db2 Bridge provides two types of validation mechanisms to ensure data integrity during migration from the source to the target system:
1. Row Count Validation
Compares the number of rows in the source and target tables after migration. If a mismatch is detected, an error is returned.
2. Checksum Validation
Performs a checksum comparison between the source and target tables. Users can specify the level of validation using either COUNT or FULL checksum modes.
Note: Data validation can be time consuming and might hinder the migration performance. It cannot be used when the source system has active writes.
Performance Analysis:
As part of performance analysis, a comprehensive performance study leveraging a 10TB BDI (Big Data Intelligence) workload was conducted to evaluate the capabilities of IBM Db2 Bridge for customers migrating from IIAS Sailfish to IBM’s P10 PCR.
Our test bench included an IIAS 1/3rd Rack as source and a P10CR Base Rack Large (BRL) as the target. The following key metrics were identified based on the test results:
- Db2 Bridge was able to attain an average of 1400GB/hr to 1550GB/hr in Data migration with the fastest table reaching upto 2700GB/hr.
- The Data migration rate is directly proportional to the number of partitions available on the source and how the data is partitioned in the database.
Figure 6: IBM Db2 Bridge performance charts
As we have found that the performance scales linearly with the number of partitions on the source, we can expect somewhere around 2.5 hours to migrate 10 TB of data from an IIAS Full Rack cluster to a P10 PCR BRL system which gives us a respectable 4TB/hr of data migration. These figures are based on uncompressed flat file equivalents, providing a consistent baseline for comparison, as compression efficiency can vary across different engines.
Conclusion:
IBM Db2 Bridge is designed to scale efficiently, whether migrating a single large database of 1,000 terabytes or thousands of smaller databases, while maintaining data integrity and performance. This blog post illustrated one such use case—migrating from IBM Integrated Analytics System (IIAS) to the Power10 Private Cloud Rack (P10 PCR)—demonstrating how the tool can support complex migrations with reliability and efficiency.
About the Authors
Sairaman K is a Software Developer and techical lead for IBM Db2 Bridge at the IBM India Software Lab, with over 6 years of experience in Databases, Kubernetes and Cloud services. He holds a Bachelor’s in Computer Science from the Thiagarajar college of Engineering, Madurai. Sairaman can be reached at sairamank@in.ibm.com
Nandhinee Dhevi Ramachandran is a Software Developer and technical lead at IBM India Software Lab, with over 12 years of experience in mainframe, databases and cloud services. Nandhinee can be reached at nandramh@in.ibm.com