Does anyone face a problem of latency in replication/mirroring tables when a lot of archived log files are generated at source database. Latency goes up to 4-6 hours and we get data too late due to this problem. Latency issue occurs every business day (from Monday to Friday) when large data are generated. IIDR Management Console performance statistics show that problem with log reader (bottleneck due to log reader). Who did solve this problem or made IIDR to read logs faster?
1) What is your source database type?
2) Is your source engine installed on the local database server or cluster, or is it installed remotely on a separate server?If your source database is DB2 LUW or DB2 on z it could be that the log reader is the bottleneck because the archived transaction logs are having to be retrieved from secondary storage (.e.g tape) as they are no longer on-line.If your source engine is remote, then comms between the source database server and the source CDC server could well be the cause of the bottleneck.If your source engine is Oracle there might be contention between CDC reading the log and Oracle writing the redo log.RegardsRobert
1) What is your source database type?Both source and target database type is Oracle.
2) Is your source engine installed on the local database server or cluster, or is it installed remotely on a separate server?Source engine is installed on a separate server. Source engine reads archived/redo logs remotely by NFS mounted on server where source engine is installed.If your source engine is remote, then comms between the source database server and the source CDC server could well be the cause of the bottleneck.We'll check it.
Oracle's recommendation is to store all logs and data files on non-cached files systems for high volume environments. This is because Oracle implements its own caching, and therefore additional caching by the OS file system is not required, and could lower performance if enabled. However we do recommend storing archive logs can benefit from a separate cached file system which is not shared with the online logs. This way, disk access by CDC does not affect Oracle log writing throughput. This is because they are only written once and never read back by Oracle unless it is recovering during backup/restore whereas CDC benefits from archive logs being stored on a cached file system as this allows CDC to read the archive log out of cache if it was recently written.
Online logs are a source of disk contention in that both Oracle and CDC are accessing them simultaneously. Therefore, the operating system must share the underlying device between reader (CDC) and writer (Oracle). In this case, if the physical disks do not have sufficient IO operations per second, both products will be constrained. Oracle will find that writes will queue and take longer to complete, and CDC will stall on disk reads. So you may even consider having flash drive."You can also refer to the following IBM documentation:Disk contention for the CDC Replication log reader
If the NFS mount point is the cause of the problem and you need low latency (a few seconds) then you may need to consider installing CDC locally on the database server.Hope this helpsRobert
For example, by dd command:
dd if=/CBS/arch/CBS_1_2047896_844335453.arc of=/dev/null bs=256k
4318+1 records in
4318+1 records out
1132132352 bytes (1.1 GB) copied, 10.0693 s, 112 MB/s.Best wishes,Anuar
HelloYou probably need to look at the log reader and parser I/O metrics in the subscription performance perspective. Possibly an increase in the memory allocated to CDC might reduce any file I/O.