NOTE: Table definitions and SQL code that will appear in this post are examples, not intended to be copied, reused or implemented. It would require a study of your data and some work to build the adequate table for your business.
Contents
Executive Summary
Transaction linking, establishing meaningful connections between individual payment events across accounts, customers, channels, and time, is a foundational problem in retail banking fraud and AML compliance. Linked transactions reveal money laundering typologies (structuring, layering, mule networks), expose fraud rings, and are the evidentiary backbone of every Suspicious Activity Report (SAR).
The academic and industry ML community converged on graph-based techniques as the dominant approach: model accounts and transactions as nodes in a network, model fund flows as directed edges, then apply Graph Neural Networks (GNNs) or network-scoring algorithms to detect suspicious subgraph patterns. IBM’s own research produced both a benchmark synthetic AML dataset and the Financial Crimes Insight platform’s graph analytics component to operationalise this.
IBM SQL Data Insights (SQL DI) for Db2 13 for z/OS offers a complementary and architecturally simpler path: train a semantic vector embedding model directly on the transaction table,in-place on the mainframe, and then expose transaction proximity through four native Db2 AI scalar functions. The result is a probabilistic transaction linking layer that requires no separate graph infrastructure, no data movement, and no labelled training data.
This document grounds the use case in published benchmark datasets and ML techniques, then shows how each industry technique maps to SQL DI query patterns and how they complement each other in a retail bank architecture.
The Transaction Linking Problem in Retail Banking
Why Transaction Linking Is Hard
A retail bank processes tens of millions of transactions per day across payment rails (SWIFT, SEPA, FPS, ACH, card networks). Each transaction is recorded as an independent row. Detecting that a series of individually innocuous payments form a coordinated scheme requires reasoning across dimensions that SQL WHERE clauses and threshold rule-engines cannot capture:
| Structuring / smurfing |
15 cash deposits of £9,800 each across 10 days, each below the reporting threshold, but collectively evidencing criminal structuring |
| Mule account chains |
Funds flow A → B → 12 sub-accounts → cash out; no single hop is suspicious; the chain is |
| Round-tripping / layering |
Funds leave the bank, travel through correspondent banks, return to a related account — disguising origin |
| Temporal burst clustering |
High-frequency transactions in a narrow window at unusual hours to multiple new payees |
| Merchant collusion fraud |
Retailer and cardholder coordinate refund fraud — transactions mirror each other in amount and timing |
| Cross-product linkage |
Mortgage drawdown immediately funds a series of high-risk outward transfers — product boundaries obscure the chain |
| Multi-customer orchestration |
Transactions initiated by different customers share merchant, amount, or timestamp patterns — invisible without cross-customer comparison |
Regulatory and Business Context
- FATF Recommendations / 6AMLD: Transaction monitoring must detect all three stages of money laundering like placement (introducing cash into the financial system), layering (concealing the trail through complex transactions), and integration (reintroducing laundered funds as legitimate). The UN estimates $800 billion–$2 trillion is laundered globally each year (2–5% of global GDP).
- PSD2 / Open Banking fraud controls: Real-time payment screening requires knowing whether an outbound payment resembles previously confirmed fraud cases.
- FinCEN / BSA (US), FCA / PSR (UK): SARs require documented evidence of suspicious transaction chains. Regulators expect behavioural analytics, not only threshold rules.
- GDPR data minimisation: Analytics must run on the existing transactional store, not a copy on a data lake , exactly the SQL DI architecture.
Industry Benchmark Datasets
Understanding transaction linking requires grounding in the public datasets that the ML research community uses to benchmark detection algorithms. These datasets directly inform the feature engineering and model design choices relevant to SQL DI column configuration.
IBM AML Synthetic Transaction Dataset (Kaggle, 2023)
The most significant publicly available benchmark for bank transaction AML was published by IBM Research in 2023 alongside the paper “Realistic Synthetic Financial Transactions for Anti-Money Laundering Models” (Atasu et al., arXiv:2306.16424). It is available on Kaggle as “IBM Transactions for Anti-Money Laundering (AML)”.
| Generator |
Agent-based synthetic generator calibrated to real bank transaction distributions |
| Scale |
Millions of transactions across multiple simulated banks |
| Laundering patterns |
8 explicit typologies: fan-out, fan-in, cycle, scatter-gather, gather-scatter, U-shape, random, and stack |
| Key columns |
Timestamp, From Bank, Account (originator), To Bank, Account (beneficiary), Amount Received, Receiving Currency, Amount Paid, Payment Currency, Payment Format, Is Laundering (label) |
| Task |
Binary transaction classification: is this transaction part of a laundering scheme? |
| ML baseline |
GNN (GraphSAGE / GCN), XGBoost, Random Forest on edge features |
This dataset is the direct analogue of the table that SQL DI would be trained on. Its column set maps almost perfectly to the SQL DI TXNDB.PAYMENT_TRANSACTION schema developed in this document, confirming the column selection is grounded in real research practice.
Elliptic Bitcoin Transaction Dataset (2019)
Published by Weber et al. (“Anti-Money Laundering in Bitcoin: Experimenting with Graph Convolutional Networks for Financial Forensics”, arXiv:1908.02591), this is the first large publicly labelled transaction graph dataset.
| Source |
Bitcoin blockchain transaction graph |
| Size |
203,769 nodes (transactions), 234,355 directed edges (payment flows) |
| Features |
166 node features per transaction (local + aggregated neighbourhood features) |
| Labels |
Illicit / licit / unknown (partial labelling , real-world condition) |
| Key finding |
Random Forest outperforms GCN on this dataset; GCN excels when neighbourhood aggregation is properly designed; both outperform Logistic Regression significantly |
| Industry lesson |
Structured tabular features (amount, timing, fee) are strong signal even without graph structure; graph structure adds lift for multi-hop patterns |
PaySim Mobile Money Simulation Dataset (Kaggle)
Simulated mobile money transactions modelled after real African mobile money network data, originally published by Lopez-Rojas et al. (2016). Available on Kaggle as “Synthetic Financial Datasets For Fraud Detection”.
| Size |
~6.3 million transactions over 30 simulated days |
| Transaction types |
CASH-IN, CASH-OUT, DEBIT, PAYMENT, TRANSFER |
| Fraud columns |
isFraud, isFlaggedFraud (rule-triggered flag) |
| Key columns |
step (hour), type, amount, nameOrig, oldbalanceOrg, newbalanceOrig, nameDest, oldbalanceDest, newbalanceDest |
| Industry lesson |
Balance deltas (newbalance - oldbalance - amount) are powerful fraud features; TRANSFER and CASH-OUT are the only fraud-bearing transaction types in this dataset, type filtering is a critical blocking step |
Credit Card Fraud Detection Dataset (Kaggle / ULB)
Anonymised real European credit card transactions from 2013, by Leborgne and Gégout (Université Libre de Bruxelles). One of the most-used fraud detection benchmarks.
| Size |
284,807 transactions, 492 frauds (0.172% positive rate) |
| Features |
V1–V28 (PCA-transformed, anonymised), Time, Amount |
| Challenge |
Extreme class imbalance; requires SMOTE, class weighting, or anomaly detection framing |
| Industry lesson |
Anomaly detection (Isolation Forest, Autoencoder reconstruction error) outperforms supervised classification when labels are sparse, directly relevant to SQL DI’s AI_SIMILARITY dissimilarity mode |
Conventional Techniques vs. SQL DI Approach
Conventional Techniques (Industry Baseline)
| Rule-based TMS |
Threshold rules: amount > X, frequency > Y, payee country in watchlist Z |
All major AML platforms |
>95% false-positive rate; no semantic proximity; evaded by staying just under thresholds |
| Random Forest / XGBoost on transaction features |
Supervised classifier on engineered features (velocity, amount ratio, MCC entropy) |
Elliptic (Weber 2019), PaySim |
Requires labelled fraud data; does not capture multi-hop chain semantics |
| Graph Convolutional Networks (GCN / GraphSAGE) |
Accounts as nodes, transactions as edges; node classification via neighbourhood aggregation |
Elliptic (Weber 2019), IBM AML (Atasu 2023) |
Requires dedicated graph infrastructure; batch-only or near-batch; complex MLOps |
| Temporal Graph Networks (TGN) |
Dynamic graph that evolves as transactions arrive; memory module per node |
Rozemberczki et al. 2021 |
Requires real-time streaming infrastructure; design specific interfaces for invocation |
| Autoencoder / Isolation Forest |
Unsupervised anomaly detection, reconstruct transaction; high reconstruction error = anomaly |
ULB Credit Card dataset |
Point anomaly only; does not capture relational/chain structure |
| Network / community detection (Louvain, label propagation) |
Build transaction graph; find connected components or dense subgraphs |
IBM FCI Graph Analytics |
Batch snapshots; stale by definition |
| Node2Vec / DeepWalk |
Random-walk-based node embedding, accounts/transactions embedded by graph neighbourhood |
General GNN literature |
Requires full graph materialisation; separate vector store |
| Word2Vec on transaction sequences |
Treat customer transaction history as a “sentence”; each transaction = a “word”; embed per customer |
Bahnsen et al. 2016; PaySim adaptations |
Sequence-level, not cross-customer; requires NLP pipeline outside the database |
The SQL DI Advantage and Mapping to Industry Techniques
SQL-DI trains a database embedding on the full row semantics of the transaction table. The resulting vector space is conceptually equivalent to a Word2Vec / transaction2vec embedding, but trained on the complete multi-column relational row, not just transaction sequences, and executed entirely within Db2.
| Random Forest / XGBoost similarity scoring |
AI_SIMILARITY |
Finds rows with semantically similar feature combinations without requiring labels |
| Anomaly detection (Autoencoder, Isolation Forest) |
AI_SIMILARITY (dissimilarity mode) |
Returns transactions maximally far from a reference, equivalent to high reconstruction error |
| Community detection / cluster membership |
AI_SEMANTIC_CLUSTER |
Seeds = known fraud/mule transactions; returns cluster affinity score |
| Network score (graph analytics) |
AI_SEMANTIC_CLUSTER (multi-seed) |
Multi-seed cluster approximates the “neighbourhood aggregation” of GCN without a graph store |
| Centroid-based statistical profiling |
AI_COMMONALITY |
Scores distance from the dataset centroid, structuring detection |
| Analogy-based pattern transfer |
AI_ANALOGY |
Tests whether a new transaction pair mirrors a known clean or known-suspicious relationship |
Key difference from GNNs: SQL DI does NOT model the transaction graph topology (edges, hops, connectivity). It models the row-level semantic similarity of individual transaction records. This means:
- SQL DI is stronger at finding transactions with similar attributes (same payment rail, similar amount band, same counterparty country) across the entire table but does not stick to individual values, you can achieve that with regular SQL.
- GNNs are stronger at detecting multi-hop structural patterns (A→B→C fund flows) in the transaction graph.
- The two approaches are complementary, not competing. SQL DI delivers the “semantic similarity layer”; IBM Financial Crimes Insight’s graph component (Db2 Graph) delivers the “network topology layer”. Together they cover both the attribute and the structural dimensions.
Target Data Model for a Retail Bank
Core Table: TXNDB.PAYMENT_TRANSACTION
Inspired by the IBM AML synthetic dataset schema (Atasu et al., 2023), this table holds the canonical transaction fact row. Column types map directly to the features that research has identified as most discriminative for transaction linking.
TRANSACTION_ID |
VARCHAR(36) |
key |
Unique row identifier; not semantically scored |
ACCOUNT_ID |
VARCHAR(20) |
categorical |
Originator account, groups transactions by source (IBM AML: Account) |
CUSTOMER_ID |
VARCHAR(20) |
categorical |
Account owner, enables cross-account customer linking |
TO_BANK |
VARCHAR(16) |
categorical |
Receiving bank identifier (IBM AML: To Bank) |
COUNTERPARTY_ACCOUNT |
VARCHAR(34) |
categorical |
Beneficiary IBAN/account, high discriminator (IBM AML: Account dest) |
COUNTERPARTY_BANK_BIC |
CHAR(11) |
categorical |
BIC of receiving bank |
COUNTERPARTY_COUNTRY |
CHAR(3) |
categorical |
ISO 3166-1 alpha-3 destination country |
PAYMENT_FORMAT |
VARCHAR(12) |
categorical |
ACH / Cheque / Credit Card / Bitcoin / Wire, from IBM AML dataset |
PAYMENT_METHOD |
CHAR(4) |
categorical |
WIRE / DRCT / STDG / BACS / CARD |
TRANSACTION_TYPE |
CHAR(3) |
categorical |
CR / DR / RFD / FEE |
AMOUNT_BAND |
SMALLINT |
categorical |
Pre-bucketed amount band (1–15); each band is a discrete token, stable boundaries, threshold-aware, explainable to regulators |
CURRENCY_PAID |
CHAR(3) |
categorical |
ISO 4217 currency of originator (IBM AML: Payment Currency) |
CURRENCY_RECEIVED |
CHAR(3) |
categorical |
ISO 4217 currency of beneficiary (IBM AML: Receiving Currency) |
IS_CROSS_CURRENCY |
CHAR(1) |
categorical |
Y/N — currency conversion flag |
MERCHANT_CATEGORY_CODE |
CHAR(4) |
categorical |
ISO 18245 MCC (0000 for non-card) |
MERCHANT_COUNTRY |
CHAR(3) |
categorical |
Merchant/terminal country |
CHANNEL |
CHAR(3) |
categorical |
MOB / WEB / ATM / BRN / IVR |
TXN_HOUR_BAND |
SMALLINT |
numeric |
Hour of day 0–23 (temporal burst detection) |
TXN_DAY_OF_WEEK |
SMALLINT |
numeric |
Day of week 1–7 |
IS_CROSS_BORDER |
CHAR(1) |
categorical |
Y/N international vs. domestic |
IS_HIGH_RISK_COUNTRY |
CHAR(1) |
categorical |
Y/N FATF high-risk jurisdiction flag |
IS_FIRST_TXN_TO_COUNTERPARTY |
CHAR(1) |
categorical |
Y/N first-time payee — key feature in PaySim research |
LAUNDERING_FLAG |
CHAR(1) |
categorical |
Y/N confirmed laundering label (IBM AML: Is Laundering) |
TXN_DESCRIPTION |
VARCHAR(140) |
text (Pro) |
SEPA remittance info / free-text narrative — PyAIDB only |
REMITTANCE_INFO |
VARCHAR(512) |
text (Pro) |
ISO 20022 structured remittance info — PyAIDB only |
Note: TXN_DESCRIPTION and REMITTANCE_INFO require SQL DI Pro with PyAIDB (APAR PH69885, SQL DI 1.1.4). In base SQL DI, exclude these columns. The structured columns alone provide strong semantic linking capability as validated on the IBM AML and Elliptic datasets but having additional data in unstructured columns strengthens the accuracy of the solution.
On AMOUNT_BAND: The IBM AML dataset uses raw decimal amounts. The ULB Credit Card dataset uses raw Amount. Research consistently shows that amount magnitude bands are more discriminative for linking than exact amounts, because launderers vary exact amounts while staying in the same magnitude range. Configure AMOUNT_BAND as categorical so each pre-bucketed band is a discrete semantic token with stable, threshold-aware boundaries.
See Appendix A. How AMOUNT_BAND and base10Cluster Work in SQL DI for a full technical explanation.
Enriched View for Transaction Linking
CREATE VIEW TXNDB.TXN_LINKING_VIEW AS
SELECT
T.TRANSACTION_ID,
T.ACCOUNT_ID,
T.CUSTOMER_ID,
T.TO_BANK,
T.COUNTERPARTY_ACCOUNT,
T.COUNTERPARTY_BANK_BIC,
T.COUNTERPARTY_COUNTRY,
T.PAYMENT_FORMAT,
T.PAYMENT_METHOD,
T.TRANSACTION_TYPE,
-- Log-scale amount band (research-grounded: IBM AML, Elliptic)
SMALLINT(CASE
WHEN T.AMOUNT_LOCAL <= 0 THEN 1
WHEN T.AMOUNT_LOCAL <= 10 THEN 2
WHEN T.AMOUNT_LOCAL <= 50 THEN 3
WHEN T.AMOUNT_LOCAL <= 100 THEN 4
WHEN T.AMOUNT_LOCAL <= 250 THEN 5
WHEN T.AMOUNT_LOCAL <= 500 THEN 6
WHEN T.AMOUNT_LOCAL <= 1000 THEN 7
WHEN T.AMOUNT_LOCAL <= 2500 THEN 8
WHEN T.AMOUNT_LOCAL <= 5000 THEN 9
WHEN T.AMOUNT_LOCAL <= 9500 THEN 10
WHEN T.AMOUNT_LOCAL <= 10000 THEN 11 -- threshold band
WHEN T.AMOUNT_LOCAL <= 25000 THEN 12
WHEN T.AMOUNT_LOCAL <= 50000 THEN 13
WHEN T.AMOUNT_LOCAL <= 100000 THEN 14
ELSE 15
END) AS AMOUNT_BAND,
T.CURRENCY_PAID,
T.CURRENCY_RECEIVED,
T.IS_CROSS_CURRENCY,
T.MERCHANT_CATEGORY_CODE,
T.MERCHANT_COUNTRY,
T.CHANNEL,
HOUR(T.TRANSACTION_TIMESTAMP) AS TXN_HOUR_BAND,
DAYOFWEEK(T.TRANSACTION_TIMESTAMP) AS TXN_DAY_OF_WEEK,
T.IS_CROSS_BORDER,
T.IS_HIGH_RISK_COUNTRY,
T.IS_FIRST_TXN_TO_COUNTERPARTY,
T.LAUNDERING_FLAG,
T.TXN_DESCRIPTION, -- Pro only
T.REMITTANCE_INFO -- Pro only
FROM TXNDB.PAYMENT_TRANSACTION T;
SQL DI AI Object Configuration
Enabling the AI Object (base SQL DI)
-- Register the AI object against the transaction view.
-- Column configuration specified during enablement via SQL DI Web UI, REST API, or CLI.
-- SQL DI data type assignment:
-- key: TRANSACTION_ID
-- categorical: ACCOUNT_ID, CUSTOMER_ID, TO_BANK, COUNTERPARTY_ACCOUNT,
-- COUNTERPARTY_BANK_BIC, COUNTERPARTY_COUNTRY, PAYMENT_FORMAT,
-- PAYMENT_METHOD, TRANSACTION_TYPE, CURRENCY_PAID,
-- CURRENCY_RECEIVED, IS_CROSS_CURRENCY, MERCHANT_CATEGORY_CODE,
-- MERCHANT_COUNTRY, CHANNEL, IS_CROSS_BORDER,
-- IS_HIGH_RISK_COUNTRY, IS_FIRST_TXN_TO_COUNTERPARTY,
-- LAUNDERING_FLAG
-- categorical: (also) AMOUNT_BAND
-- numeric: TXN_HOUR_BAND, TXN_DAY_OF_WEEK
Enabling Text Columns (SQL DI Pro, APAR PH69885)
After PH69885 PTF: TXN_DESCRIPTION and REMITTANCE_INFO use COLTYPE = 'text' Verify column configuration after enabling:
Transaction Linking Query Patterns
Each pattern below maps to a known industry technique validated on public datasets.
Pattern 1 — Find Transactions Matching a Confirmed Fraud Modus Operandi
Industry equivalent: Random Forest / XGBoost similarity scoring; nearest-neighbour search on transaction feature vectors.
Use case: A transaction is confirmed fraud (e.g., authorised push-payment scam or a labelled IBM AML laundering transaction). Find all other transactions with the same semantic profile, same payment format, amount band, counterparty country, and channel — without specifying a rule for each dimension.
-- Top 25 transactions most similar to confirmed fraud case TXN-20240315-00987654
SELECT
AI_SIMILARITY(T.TRANSACTION_ID, 'TXN-20240315-00987654') AS SIMILARITY_SCORE,
T.TRANSACTION_ID,
T.ACCOUNT_ID,
T.CUSTOMER_ID,
T.COUNTERPARTY_ACCOUNT,
T.COUNTERPARTY_COUNTRY,
T.PAYMENT_FORMAT,
T.AMOUNT_BAND,
T.CHANNEL,
T.LAUNDERING_FLAG
FROM TXNDB.TXN_LINKING_VIEW T
WHERE T.TRANSACTION_ID <> 'TXN-20240315-00987654'
ORDER BY SIMILARITY_SCORE DESC
FETCH FIRST 25 ROWS ONLY;
Score thresholds (calibrate per bank):
| > 0.90 |
Near-identical semantic profile, high-priority SAR candidate |
| 0.75–0.90 |
Closely related, include in investigation case file |
| < 0.65 |
Different semantic profile, no immediate link |
Pattern 2 — Structuring Detection via AI_COMMONALITY
Industry equivalent: Statistical centroid profiling; detection of transactions engineered to blend into the population (the key challenge of the IBM AML dataset’s fan-in and scatter-gather laundering patterns).
Use case: Structuring transactions are specifically designed to appear “average” — they cluster near the centroid of the legitimate transaction distribution. High AI_COMMONALITY scores combined with repetition per account expose this pattern.
-- High commonality = statistically typical transaction → structuring signal
-- Accounts with many high-typicality transactions are structuring candidates
SELECT
AI_COMMONALITY(T.TRANSACTION_ID) AS TYPICALITY_SCORE,
T.ACCOUNT_ID,
T.CUSTOMER_ID,
T.AMOUNT_BAND,
T.TXN_HOUR_BAND,
T.COUNTERPARTY_COUNTRY,
T.PAYMENT_FORMAT,
T.IS_FIRST_TXN_TO_COUNTERPARTY,
T.LAUNDERING_FLAG
FROM TXNDB.TXN_LINKING_VIEW T
WHERE T.LAUNDERING_FLAG = 'N'
AND T.IS_HIGH_RISK_COUNTRY = 'N'
ORDER BY TYPICALITY_SCORE DESC
FETCH FIRST 100 ROWS ONLY;
Downstream aggregation: Group this result by ACCOUNT_ID and count rows with TYPICALITY_SCORE > 0.80. Accounts with > N such rows within a time window match the structuring typology from the IBM AML dataset’s scatter-gather and fan-in patterns.
Pattern 3 — Mule Network Detection via Semantic Cluster Membership
Industry equivalent: GCN / GraphSAGE node classification; community detection in the transaction graph; IBM Financial Crimes Insight graph analytics network score.
Use case: Given confirmed mule transactions (seed set), find all transactions across the portfolio that are semantically clustered with the mule profile. The multi-seed AI_SEMANTIC_CLUSTER approximates the neighbourhood aggregation step of a GCN without requiring a graph materialisation.
-- Semantic cluster membership against confirmed mule transaction seeds
-- Seeds represent the IBM AML "fan-out" or "gather-scatter" laundering patterns
SELECT
AI_SEMANTIC_CLUSTER(
T.TRANSACTION_ID
USING MODEL COLUMN TRANSACTION_ID,
'TXN-MULE-SEED-001' USING MODEL COLUMN TRANSACTION_ID,
'TXN-MULE-SEED-002' USING MODEL COLUMN TRANSACTION_ID,
'TXN-MULE-SEED-003' USING MODEL COLUMN TRANSACTION_ID
) AS MULE_CLUSTER_SCORE,
T.TRANSACTION_ID,
T.ACCOUNT_ID,
T.CUSTOMER_ID,
T.COUNTERPARTY_ACCOUNT,
T.COUNTERPARTY_COUNTRY,
T.PAYMENT_FORMAT,
T.AMOUNT_BAND,
T.LAUNDERING_FLAG
FROM TXNDB.TXN_LINKING_VIEW T
WHERE T.TRANSACTION_ID NOT IN (
'TXN-MULE-SEED-001', 'TXN-MULE-SEED-002', 'TXN-MULE-SEED-003'
)
ORDER BY MULE_CLUSTER_SCORE DESC
FETCH FIRST 50 ROWS ONLY;
Why multi-seed is better: AI_SEMANTIC_CLUSTER scores the candidate against the entire seed cluster simultaneously. This mirrors the GraphSAGE aggregation insight from the Elliptic dataset, where aggregating over multiple neighbourhood nodes was more robust than pairwise comparisons, especially under the partial-label conditions typical of real bank data.
Pattern 4 — Real-Time Anomaly Scoring via Dissimilarity
Industry equivalent: Isolation Forest / Autoencoder reconstruction error on individual transactions; validated on ULB Credit Card Fraud dataset.
Use case: For each new transaction, score it against the customer’s established behavioural norm. Transactions maximally dissimilar from the customer’s history are the highest-risk anomalies, account takeover, novel fraud typology, first use of a stolen card. This is the SQL DI equivalent of Isolation Forest’s anomaly score.
-- Top 10 most anomalous transactions for customer CUST-00123456
-- vs. their established transaction profile
SELECT
AI_SIMILARITY(T.TRANSACTION_ID, 'TXN-PROFILE-CUST-00123456', 'DISSIMILARITY')
AS ANOMALY_SCORE,
T.TRANSACTION_ID,
T.ACCOUNT_ID,
T.COUNTERPARTY_ACCOUNT,
T.COUNTERPARTY_COUNTRY,
T.PAYMENT_FORMAT,
T.AMOUNT_BAND,
T.CHANNEL,
T.TXN_HOUR_BAND,
T.IS_FIRST_TXN_TO_COUNTERPARTY
FROM TXNDB.TXN_LINKING_VIEW T
WHERE T.CUSTOMER_ID = 'CUST-00123456'
ORDER BY ANOMALY_SCORE DESC
FETCH FIRST 10 ROWS ONLY;
Note on the profile reference: TXN-PROFILE-CUST-00123456 is the TRANSACTION_ID of a representative transaction from the customer’s history, or the most recent transaction used as a “current state” anchor. The 'DISSIMILARITY' argument inverts the cosine similarity, consult the Db2 13 SQL reference for the exact syntax at your SQL DI release level.
Pattern 5 — Cross-Account Relationship Verification via Analogy
Industry equivalent: Rule-based transfer-relationship verification; analogy reasoning over transaction pair relationships.
Use case: A known clean payment relationship exists: Customer A regularly sends funds to Customer B (confirmed personal relationship, verified KYC). Use AI_ANALOGY to test whether a similar-looking fund flow from Customer C to Customer D follows the same semantic relationship, or whether the analogy breaks down, signalling disguised layering.
-- Does TXN-NEW-001 (C outbound) relate to TXN-NEW-002 (D inbound) in the same
-- way that TXN-KNOWN-A (A outbound) relates to TXN-KNOWN-B (B inbound)?
SELECT
AI_ANALOGY(
'TXN-KNOWN-A' USING MODEL COLUMN TRANSACTION_ID,
'TXN-KNOWN-B' USING MODEL COLUMN TRANSACTION_ID,
'TXN-NEW-001' USING MODEL COLUMN TRANSACTION_ID,
'TXN-NEW-002' USING MODEL COLUMN TRANSACTION_ID
) AS ANALOGY_SCORE
FROM SYSIBM.SYSDUMMY1;
- High score (→ 1.0): New pair mirrors the known clean relationship, proceed.
- Low score (→ 0.0): Relationship is semantically different despite surface similarity, flag for investigation. Effective at detecting IBM AML “U-shape” and “cycle” laundering typologies where amounts/rails mimic legitimate flows.
Pattern 6 — Multi-Hop Fund Flow Chain (Iterative Similarity Traversal)
Industry equivalent: GNN multi-hop propagation; IBM FCI graph analytics with configurable hop depth; Db2 Graph network score.
Use case: Trace a suspected layering chain hop by hop. Each hop uses the top-scoring transactions from the previous hop as seeds for the next. This is the SQL DI approximation of multi-hop graph traversal, applied iteratively at the application layer.
Hop 1 — Find transactions semantically similar to a confirmed fraud seed:
-- Hop 1: find transactions from accounts linked to the seed fraud transaction
WITH SEED AS (
SELECT 'TXN-FRAUD-A001' AS SEED_TXN_ID
)
SELECT
AI_SIMILARITY(T.TRANSACTION_ID, S.SEED_TXN_ID) AS HOP1_SCORE,
T.TRANSACTION_ID,
T.ACCOUNT_ID,
T.COUNTERPARTY_ACCOUNT,
T.COUNTERPARTY_COUNTRY,
T.PAYMENT_FORMAT,
T.AMOUNT_BAND,
T.TRANSACTION_TYPE
FROM TXNDB.TXN_LINKING_VIEW T,
SEED S
WHERE T.ACCOUNT_ID IN (
SELECT COUNTERPARTY_ACCOUNT
FROM TXNDB.PAYMENT_TRANSACTION
WHERE TRANSACTION_ID = 'TXN-FRAUD-A001'
)
AND T.TRANSACTION_ID <> S.SEED_TXN_ID
ORDER BY HOP1_SCORE DESC
FETCH FIRST 20 ROWS ONLY;
The application substitutes the TRANSACTION_ID values from the top hop-1 results as seeds for hop 2, repeating until scores fall below a configurable threshold or a maximum hop count is reached. This replicates the IBM FCI graph analytics “expand on demand” traversal model documented in the Financial Crimes Insight platform.
Column Influence and Discriminator Scores
Understanding which columns dominate the semantic space is essential for tuning the model and providing regulatory explainability. Research on the IBM AML and Elliptic datasets confirms which features are most discriminative for laundering detection.
-- Column influence and discriminator scores for the transaction linking model
SELECT
COLUMN_NAME,
COLTYPE,
INFLUENCE_SCORE,
DISCRIMINATOR_SCORE
FROM SYSAIDB.SYSAICOLUMNCONFIG
WHERE OBJECT_SCHEMA = 'TXNDB'
AND OBJECT_NAME = 'TXN_LINKING_VIEW'
ORDER BY DISCRIMINATOR_SCORE DESC;
Research-grounded expected profile:
COUNTERPARTY_ACCOUNT |
Very high |
Near-unique IBANs — highest cardinality column |
COUNTERPARTY_BANK_BIC |
High |
Thousands of distinct BICs globally |
ACCOUNT_ID |
High |
Many distinct account identifiers |
PAYMENT_FORMAT |
Medium-high |
IBM AML dataset: format is the single most discriminative feature for laundering typology |
MERCHANT_CATEGORY_CODE |
Medium-high |
~500 MCCs — good semantic grouping |
COUNTERPARTY_COUNTRY |
Medium |
~200 countries; FATF high-risk countries form tight semantic clusters |
AMOUNT_BAND |
Medium |
15 bands configured as categorical; each band is a discrete semantic token, proximity learned from co-occurrence with payment format, counterparty country, etc. |
TXN_HOUR_BAND |
Low-medium |
24 values — temporal burst fingerprint |
IS_HIGH_RISK_COUNTRY |
Low |
Binary — low discriminator but high influence when set to Y |
LAUNDERING_FLAG |
Low |
Binary — very high influence when Y labels are present and clean |
TXN_DESCRIPTION (Pro) |
Variable |
Highest when SWIFT MT messages contain diverse remittance narratives |
Model Management and Incremental Retraining (SQL DI Pro)
Transaction tables are the highest-velocity datasets in a retail bank. The IBM AML dataset simulates continuous transaction generation, its agent-based generator mirrors the daily inflow pattern a real bank faces. With SQL DI Pro (PyAIDB), the model absorbs new confirmed fraud patterns through incremental retraining without a full rebuild:
-- Retraining history for the transaction linking model
SELECT
OBJECT_SCHEMA,
OBJECT_NAME,
RETRAIN_TYPE, -- 'F' = full, 'I' = incremental (Pro)
START_TIME,
END_TIME,
CHAR(TIMESTAMPDIFF(2, CHAR(END_TIME - START_TIME))) AS DURATION_SECS
FROM SYSAIDB.SYSAIRETRAINLOG
WHERE OBJECT_SCHEMA = 'TXNDB'
AND OBJECT_NAME = 'TXN_LINKING_VIEW'
ORDER BY START_TIME DESC
FETCH FIRST 10 ROWS ONLY;
Research-grounded retraining cadence:
| Initial model build |
Full (F) |
Once at go-live |
Baseline training on full historical dataset (Elliptic, IBM AML pattern) |
| Daily new transaction volume |
Incremental (I) — Pro |
Every 4–8 hours |
Continuous transaction inflow, IBM AML agent-based simulation shows drift within hours |
| New fraud typology confirmed |
Full (F) or Incremental (I) |
After bulk labelling |
Adding new LAUNDERING_FLAG = 'Y' records requires absorbing new vocabulary |
| Payment format or MCC change |
Full (F) |
After data dictionary update |
New categorical vocabulary values unseen at training time return NULL, must retrain |
| Annual model quality review |
Full (F) |
Annually |
Elliptic dataset: model performance degrades over time as graph structure evolves |
Architecture: Transaction Linking Application Flow
Limitations and Complementary Techniques
SQL DI transaction linking is a semantic layer complement to graph analytics and rule-based systems, not a replacement. The Elliptic and IBM AML dataset research explicitly confirms where each technique excels.
| No graph topology |
SQL DI does not model directed edges (A→B→C chains); it measures row-level attribute similarity |
Elliptic: GCN outperforms RF on multi-hop patterns; RF outperforms GCN on single-node features |
Combine SQL DI (attribute layer) with IBM FCI Graph Analytics or Db2 recursive SQL (topology layer) |
| Amount granularity |
Raw decimals must be pre-bucketed; the model does not natively distinguish £9,800 from £9,750 |
IBM AML: structuring uses amounts near thresholds, narrow bands near €10,000 are critical |
Design log-scale bands with boundary at key regulatory thresholds |
| Temporal ordering not modelled |
SQL DI embeds each transaction in isolation; burst sequence patterns require time-series reasoning |
PaySim: step (hour) is a key fraud feature, burst detection needs sequence context |
Pre-compute TXN_HOUR_BAND and TXN_DAY_OF_WEEK; combine with a windowed rule engine |
| Unseen counterparty = NULL |
A new IBAN not in the training vocabulary returns NULL similarity |
PaySim: IS_FIRST_TXN_TO_COUNTERPARTY is a top fraud feature |
Treat NULL score as elevated risk; configure in application; use IS_FIRST_TXN_TO_COUNTERPARTY flag as a hard override |
| Extreme class imbalance |
Laundering is rare (<1% in IBM AML dataset); the model learns a heavily skewed distribution |
ULB CC: 0.172% fraud rate; standard training produces low recall |
Include LAUNDERING_FLAG as a categorical column; ensure labelled positives are well-represented in training data |
| High-volume table performance |
Cross-product scoring on 500M+ rows is impractical |
Standard recommendation across all datasets |
Block by PAYMENT_FORMAT, COUNTERPARTY_COUNTRY, or ACCOUNT_ID in WHERE before AI functions |
Data Preparation Recommendations
Grounded in the feature engineering patterns observed on the IBM AML, Elliptic, and PaySim benchmark datasets:
- Amount bucketing: Pre-compute
AMOUNT_BAND with boundary at key regulatory thresholds (€10,000 / $10,000 CTR, £5,000 enhanced due diligence). The IBM AML dataset shows structuring concentrates just below these boundaries, narrow bands there improve discrimination.
- Payment format normalisation: Standardise
PAYMENT_FORMAT values to a controlled vocabulary (ACH / Cheque / Credit Card / Bitcoin / Wire, as in the IBM AML dataset). Inconsistent format names fragment the vocabulary and suppress discriminator scores.
- Counterparty normalisation: Normalise IBAN (remove spaces, capitalise). BIC codes padded to 11 characters. Replace filler values (
UNKNOWN, XXXXXXXXX) with SQL NULL.
- Temporal feature engineering: Pre-compute
TXN_HOUR_BAND and TXN_DAY_OF_WEEK as SMALLINT columns. The PaySim dataset demonstrates that transaction hour is among the most discriminative features for fraud, do not pass raw TIMESTAMP.
- First-payee flag: Maintain
IS_FIRST_TXN_TO_COUNTERPARTY as a live CHAR(1) column, updated by a trigger or CDC process. PaySim research and IBM AML fan-out patterns both confirm first-payee status as a top-ranked feature.
- Laundering label freshness: Ensure
LAUNDERING_FLAG reflects confirmed current state. Stale or inconsistent labels are more harmful than omitting the column, use a view that joins the latest SAR disposition table rather than a static column.
- Transaction description cleaning (Pro): Remove ISO 20022 header boilerplate from
TXN_DESCRIPTION. PyAIDB’s transformer delivers maximum value on varied, semantically rich remittance information, SWIFT MT103 narrative fields are ideal candidates.
Summary Matrix — SQL DI Functions vs. Transaction Linking Scenarios
| Find transactions matching a known fraud modus operandi |
AI_SIMILARITY |
RF/XGBoost nearest-neighbour |
IBM AML, Elliptic |
| Detect structuring / blending-in transactions |
AI_COMMONALITY |
Statistical centroid profiling |
IBM AML scatter-gather, fan-in typologies |
| Identify mule network transactions from seed set |
AI_SEMANTIC_CLUSTER |
GCN neighbourhood aggregation |
Elliptic (Weber 2019) |
| Real-time anomaly scoring per customer profile |
AI_SIMILARITY (dissimilarity) |
Isolation Forest / Autoencoder |
ULB Credit Card Fraud dataset |
| Verify semantic relationship between transaction pairs |
AI_ANALOGY |
Rule-based relationship transfer |
IBM AML cycle / U-shape typologies |
| Multi-hop fund flow chain tracing |
AI_SIMILARITY (iterative) |
IBM FCI graph hop expansion |
IBM AML fan-out / gather-scatter typologies |
| Cross-table linkage (card + wire) |
AI_SIMILARITY with USING MODEL TABLE |
Cross-system graph federation |
Requires compatible column models on both tables |
References
Research Papers and Datasets
- Atasu, K. et al. (2023). Realistic Synthetic Financial Transactions for Anti-Money Laundering Models. arXiv:2306.16424. IBM Research. → IBM AML dataset on Kaggle
- Weber, M. et al. (2019). Anti-Money Laundering in Bitcoin: Experimenting with Graph Convolutional Networks for Financial Forensics. arXiv:1908.02591. → Elliptic dataset
- Lopez-Rojas, E. A., Elmir, A., & Axelsson, S. (2016). PaySim: A financial mobile money simulator for fraud detection. EMSS 2016. → PaySim dataset on Kaggle
- Dal Pozzolo, A. et al. (2015). Calibrating Probability with Undersampling for Unbalanced Classification. IEEE SSCI. → ULB Credit Card Fraud dataset on Kaggle
- FATF (2012, updated 2023). The FATF Recommendations. Financial Action Task Force. → AML typology reference
- MarketsandMarkets (2023). Transaction Monitoring Market — Global Forecast to 2028. USD 6.8 billion projection.
Appendix A — How AMOUNT_BAND and base10Cluster Work in SQL DI
Two strategies for representing transaction amounts
There are two distinct strategies for feeding transaction amounts into an SQL DI model, and they behave very differently.
Strategy A — Raw DECIMAL column configured as numeric (automatic clustering)
When a DECIMAL or FLOAT column is assigned SQL DI data type numeric, the training pipeline automatically invokes base10Cluster during the AI object enablement preprocessing phase. The algorithm runs in two steps:
- Binning: Each numeric value is assigned to a bin determined by its base-10 logarithm. For example, £9,845.50 → log₁₀ ≈ 3.99 → bin 3 (covering values in the 1,000–9,999 range). Values that are log-close land in the same bin.
- Redistribution: The bins are rebalanced to produce equal-density buckets across the actual data population. If 80% of transactions are under £1,000, those bins are subdivided more finely; sparse high-amount bins are merged. The resulting cluster boundaries are written to
SYSAIDB.SYSAICOLUMNCENTERS (CLUSTER_MIN, LABEL).
Each transaction amount is then replaced by a string token such as AMOUNT_LOCAL_cluster_7. That token is trained as a vocabulary item in ibm-data2vec alongside categorical tokens. The model learns that cluster_7 and cluster_8 are semantically proximate because they co-occur with similar merchants, counterparty countries, and payment formats.
You can inspect the cluster boundaries produced for any numeric column with:
SELECT
COLUMN_NAME,
CLUSTER_MIN,
LABEL
FROM SYSAIDB.SYSAICOLUMNCENTERS
WHERE MODEL_ID = (
SELECT MODEL_ID FROM SYSAIDB.SYSAIOBJECTS
WHERE SCHEMA = 'TXNDB' AND NAME = 'TXN_LINKING_VIEW'
)
AND COLUMN_NAME = 'AMOUNT_LOCAL'
ORDER BY CLUSTER_MIN ASC;
Strategy B — Pre-bucketed SMALLINT configured as categorical (the recommended approach)
Instead of relying on base10Cluster to decide boundaries, the AMOUNT_BAND column is pre-computed in the Db2 view using a CASE expression and then configured as categorical. Each band integer (1–15) becomes a discrete vocabulary token; proximity between adjacent bands is learned from their co-occurrence with the rest of the row (same PAYMENT_FORMAT, same COUNTERPARTY_COUNTRY, etc.), not from arithmetic distance.
Why Strategy B is better for AML transaction linking
| Threshold control |
base10Cluster log scale: £9,800 and £10,200 land in the same bin, the regulatory boundary at £10,000 is invisible to the model |
A band boundary placed at £10,000 makes £9,800 (band 10) and £10,200 (band 11) adjacent but distinct, the structuring threshold is preserved |
| Boundary stability |
Cluster boundaries shift between retrains as the population distribution changes |
Band assignments are static, no shift between retrains |
| Cardinality |
Millions of raw decimals before clustering; redistribution is data-driven and unpredictable on skewed distributions |
Exactly 15 bands with known semantic meaning |
| Explainability |
Cluster labels are opaque integers requiring inspection of SYSAIDB.SYSAICOLUMNCENTERS to interpret |
AMOUNT_BAND = 10 means “£9,500–£10,000”, directly explainable to compliance officers and regulators |
A subtlety: why not numeric on the pre-bucketed integer?
If AMOUNT_BAND (values 1–15) were configured as numeric, base10Cluster would still run, but on the integers 1–15. Because log₁₀(10) = 1.0 and log₁₀(15) = 1.18, the redistribution step would compress bands 10–15 into fewer clusters, blunting the threshold boundary that was deliberately designed into the view. Configuring as categorical bypasses base10Cluster entirely for this column, preserving every band as its own discrete token.
IBM’s documentation states: “Treat a column of numeric SQL data type as a SQL DI categorical type if the column contains 10 or fewer distinct values.” With 15 bands, the principle extends: any pre-bucketed integer column whose bands carry explicit semantic meaning should be treated as categorical, regardless of its SQL data type.