Introduction:
The LVM (Logical Volume Manager) always ensures data consistency among mirrored copies of a logical volume during normal I/O processing.
For every write to a logical volume, the LVM generates a write request for every mirror copy. A problem arises if the system crashes in the middle of processing a mirrored write (before all copies are written). If mirror write consistency recovery is enabled for a logical volume, the LVM keeps additional information to allow recovery of these inconsistent mirrors. Mirror write consistency recovery should be enabled for most mirrored logical volumes.
LVM provides two types of MWC (Mirror Write Consistency), Active and Passive. This blog is regarding the Active mirror write consistency.
Active MWC provides fast recovery at reboot time after a crash has occurred. However, under very heavy workloads, especially in the case of random writes write performance will degrade with Active MWC.
In production(customer) scenario, under heavy workloads, MWC cache may become bottleneck due to its limited size. The MWC cache size is fixed and cannot scale in proportion with the increasing workload. Hence may not be large enough to track all the active writes. So, this results in slowness in IO processing and appears as hang. At present, we do not have any method to detect and display such conditions.
Solution:
To help system administrators to detect such conditions and tune workload accordingly, LVM is enhanced to detect and display MWC cache full scenario.
lvmstat command is enhanced to display Active MWC statistics,
|
Column
|
Description
|
|
IO_CNT
|
Specifies the total number of requests.
|
|
FULL_CNT
|
Specifies the number of times the request was on hold because the cache was full.
|
|
INFLT_CNT
|
Specifies the number of times the request was on hold because the cache write operation was in progress.
|
Examples:
To enable the MWCC statistics for the volume group datavg, enter the following command:
lvmstat -v datavg -m -e
To disable the MWCC statistics for the volume group datavg, enter the following command:
lvmstat -v datavg -m -d
To clear the MWCC statistics for the volume group datavg, enter the following command:
lvmstat -v datavg -m -C
To display five MWCC statistics reports at a time for the volume group datavg at an interval of 10 seconds in comma-separated values (csv) file format, enter the following command:
lvmstat -v datavg -F m -s 10 5
Pre-requisites
This feature is supported from AIX 7.3 TL3 release and above.