Getting sick is a fact of life. If you live in the United States, paying for your doctor’s bill is normally your responsibility. Unless you decide to send a check via snail mail, you are probably going to pay off your bill with an online transaction or over the phone with the billing department. That normally means a debit or credit card.
I recently paid one of those bills for a sick kid… lucky me, right? The interesting part thing was how quickly my account was updated after the transaction. My account was already updated with the charge by the time I logged in and looked at it… which was well under a minute from the time the transaction processed. We are effectively talking near real time. a couple of seconds or less milliseconds for a set of transactions. The only delay was my ability to quickly access my account.
How is that possible? While I can’t comment on how a transaction works between companies, the technology behind a company receiving the charge and updating an account is Data Replication or specifically Change Data Capture. When a highly transactional database has a change, such as a payment, a row is updated. That update is sent from one database to another database at near real-time.
Many of these transactions occur on the mainframe and using the DB2 on z/OS database. Many companies then replicate data to another database that is hosted on X86 architecture (think an Intel or AMD processor). This secondary database could be on-premises or in a cloud provider. The secondary Database might be opensource such as PostgreSQL, IBM’s DB2 or a hosted database such as SingleStore.
An extremely high-level view of the flow of data from the mainframe to the end user is below.
Assuming the source table has already been replicated to the target database, CDC only sends incremental changes from the database transaction logs from the source to the target. This ensures near real-time updates and reduces network bandwidth usage (reduces congestion). It also saves money if the changes are being sent out of a cloud’s VPC.
CDC implements these changes through three SQL commands: Insert, update, and delete. Depending on the database used, a row might be inserted into the table to add a new product at a store. If a customer purchased that product then an update command could change the amount of inventory on hand. Finally, when the product is no longer being sold, a the delete command can used to delete the row from the database… Hopefully, your database administrator is careful enough when using a delete command and deletes the desired row and not delete an entire table. Without the proper safeguards, any change data capture technology could propagate that change to your target database and you might experience data loss and have a really bad day.
Making an update happens within a blink of an eye and if you think about it at scale, is impressive. While I hope my transaction was powered by IBM Data Replication, the power of Change Data Capture is truly fantastic.
Check out theproduct page to learn more about IBM Data Replication!