AIX

 View Only

  • 1.  Shrinking JFS2 file sytem

    Posted Wed June 23, 2021 01:40 PM
    Hi all,
    shrinking jfs2 file system has been quite a long with us (AIX 6.1 as far as I remember) and always works fine for me. I haven't encounter any problems regarding corruption of data and haven't heard that someone has. imho it's not possible as chfs just move extents in logical volume and eventually will fail if there is not enough continuous free space, but still my client has concerns. I'm looking for confirmation that it's 100% safe for file system data. Has anyone seen official document/information that confirms it' safe?

    Jakub

    ------------------------------
    Jakub Pacowski
    ------------------------------


  • 2.  RE: Shrinking JFS2 file sytem

    Posted Wed June 23, 2021 03:57 PM
    Hi

    There's nothing 100% safe in computing. It's better to use 90% and let 10% for our old friend Murphy.

    About shrinking jfs2 file system  it's going to work as far as there's enough free space not only for data as for jfs2 log device (external or internal). 

    As always best practices is to keep our AIX systems with latest updates availables just in case.

    HTH


     


    César Daniel Delgado Ponce.

    Sistemas Operativos Seguridad Distribuidos (6181)
    +58 212 503 0619
    Twitter:@MercantilBanco
    YouTube: Mercantil Banco





    Original Message


  • 3.  RE: Shrinking JFS2 file sytem

    Posted Thu June 24, 2021 10:07 AM

    Here are some of the file system and storage related patches I have been struggling with this past year:

     

    IJ19600: SYSTEM CRASH WITH INTERRUPT STACK OVERFLOW WHILE DOING JFS2 IO APPLIES TO AIX 7200-03

    IJ24067: HANG IN DISK DRIVER IF LUN IS UN-MAPPED AND OPEN ATTEMPTED APPLIES TO AIX 7200-04

    IJ25122: IN RARE CASE FSCK MAY COMPLETE AND LEAVE THE METADATA CORRUPT APPLIES TO AIX 7200-04

    IJ25150: JFS2 DEFRAGFS -F <MNTPT> CMD MAY PUT FILE SYSTEM IN READ-ONLY APPLIES TO AIX 7200-04

    IJ25245: DEFRAGFS -F CAN CRASH IN DBBACKSPLIT WHEN BLOCKMAP IS CORRUPT APPLIES TO AIX 7200-04

    IJ29204: UNCONFIG OF NON-MPIO DISKS CAN RESULT IN CRASH.

    IJ30149: AIX CRASH AFTER USING IOO TO CHANGE DK_CLOSED_PATH_RECOVERY

     

    It is not always easy getting the latest patches from IBM.  I often have to wait while IBM tests multi-fixes for the current (n) and recent (n-1) supported levels of each AIX TL, even when there are official single fixes for known defects.  One would think a company leading their marketing for their operating system with "For over three decades, our customers have trusted AIX to run their most mission critical applications", would at least keep a rolling patch set with patches for all for known defects with validated patches.  I should not have to beg for security fixes affecting Confidentiality, Integrity, or Availability; especially when I repeatedly send system crash dumps and provide detailed problem reports.

     

    I also find it puzzling that IBM obscures defect tracking for customers by providing randomly numbered tracking numbers for each version level of their product even when the same defect is in multiple versions of AIX.  I don't know how much time I waste trying to cross reference which APAR numbers apply to which releases and which releases are affected by a defect and which releases have which fixes, and which patches (ifixes or filesets) are missing which previously patched defects.  I don't care if a fix is delivered as an i-fix (emgr) or fileset (installp) or RPM or other package management format, but I do need to know it has some level of validation for fixes of defects that have impacted me or are likely to impact me.  A single tracking number for each defect for all versions would be greatly appreciated by customers and would provide higher safety levels for customers using AIX for mission critical applications for which it is intended.

     

    (I have discussed this with way too many IBM "quality" managers and IBM "escalation" managers.)

     



    ------------------------------
    Edward Davignon
    ------------------------------

    Original Message


  • 4.  RE: Shrinking JFS2 file sytem

    Posted Wed June 23, 2021 05:20 PM
    Hi Jakub,

    it is completely safe to shrink a JFS2 filesystem, otherwise it would not be supported. :)
    However, as also the man page states, during shrinking the I/O will be suspended, so if you have a big filesystem it is recommended to shrink the filesystem in small amounts or during low workloads.

    Regards,
    Levente



    Original Message


  • 5.  RE: Shrinking JFS2 file sytem

    Posted Thu June 24, 2021 03:35 AM
    Thanks for the responses. Still looking for official note or document  (if exist :))

    ------------------------------
    Jakub Pacowski
    ------------------------------

    Original Message


  • 6.  RE: Shrinking JFS2 file sytem

    Posted Thu June 24, 2021 04:43 AM
    Edited by Jakub Pacowski Thu June 24, 2021 04:44 AM
    I've got official replay from @mr_nmon. Thank you Nigel for help.

    Hi,

    100% customer paranoia.

     

    This was a feature in AIX 6 (may be earlier look for JFS2 announcements) and I have been using it for decades.

    You might find it in the AIX strength to Strength document.

     

    JFS2 is extent based which makes it much easier = it does not have to move anything and the space does not need to be removed from the end.  The logical volume of the JFS2 has a PP size - you find that in the volume group 

     

    blue:/# lsvg rootvg
    VOLUME GROUP:       rootvg                   VG IDENTIFIER:  00f6052500004c0000000139ff5a6711
    VG STATE:           active                   PP SIZE:        256 megabyte(s)
    VG PERMISSION:      read/write               TOTAL PPs:      511 (130816 megabytes)
    MAX LVs:            256                      FREE PPs:       156 (39936 megabytes)
    LVs:                13                       USED PPs:       355 (90880 megabytes)
    OPEN LVs:           12                       QUORUM:         1 (Disabled)
    TOTAL PVs:          1                        VG DESCRIPTORS: 2
    STALE PVs:          0                        STALE PPs:      0
    ACTIVE PVs:         1                        AUTO ON:        yes
    MAX PPs per VG:     32512                                     
    MAX PPs per PV:     1016                     MAX PVs:        32
    LTG size (Dynamic): 256 kilobyte(s)          AUTO SYNC:      no
    HOT SPARE:          no                       BB POLICY:      relocatable 
    PV RESTRICTION:     none                     INFINITE RETRY: no
    DISK BLOCK SIZE:    512                      CRITICAL VG:    no
    FS SYNC OPTION:     no                       CRITICAL PVs:   no
    blue:/# 
     

    So all shrinking does is file an 256MB extent that has no space allocated from it for your JFS2 and remove it from the unused extent list and remove it from the LV.

     

    If your file system is too full it will refuse to shrink.

     

    IBM does not document the reliability of individual AIX commands!!

    That would be insane.

    All AIX commands are safe and backed up by AIX Support.

    To be blunt, if chfs was unsafe it would have been in the press and all over the web for the past 20 years.

     

    Of course, if an AIX user also had underlying disk problems they are unaware of then chfs might highlight that.

    So if your customer is in the habit of ignoring AIX warnings:

    Recommend they check their AIX systsem logs = errtp -a

    before using chfs.

     

    Nigel Griffiths



  • 7.  RE: Shrinking JFS2 file sytem

    Posted Thu June 24, 2021 10:47 AM

    How long shrinking a file system depends on how much data needs to be moved.  Keep in mind, applications and databases may need to wait for chfs to finish resizing a file system before the application can access the data or write to the data. That is, applications may hang until chfs finishes resizing a file system.

    When I do resize file systems in AIX with inline logs, I specify the logsize and size, so chfs does not automatically rescale logsize.  I use "lsfs -q" to get the log size.  I may scale the logsize with a separate chfs command or simple make it big enough that it should not need to be changed.  This reduces the time chfs keeps its locks on a file system.

    I wish there was a preview to determine how much data would need to be moved for a particular shrink file system operation.  This information would be valuable in determining how long a particular chfs size command would lock out access to the database while it moved data; it would allow us to determine what size decrements to decrease a file system to avoid affecting the application.  Better yet, the shrink command should operate on manageably-sized sets (i.e. chucks) of data and periodically commit its progress, release its locks, and wait while applications catch up, before starting its next set of processing.

    Keep in mind that for large file systems with many PVs on large VGs, it may take a long time to update all of the VGDAs, even for small changes changes to an LV.  This means that each chfs resize operation on a large file system may have a large, constant-time overhead associated with each invocation of chfs.

    These last two points make it difficult to determine what the optimal "small amounts" are for each shrink file system command.

    I typically use the sleep command to wait between chfs and LVM commands when applications are running on the server.

    I never shrink the office of record (i.e. the official copy of important information), without validation.  For OLTP DBs, typically I would pause a near real-time replica and flashcopy (i.e. array-based snapshot) the data, mount the copy on another server, resize the copy, unmount the copy, mount the copy, compare the copy with the original, unmount the original, and restart the replication process to replay the changes.  Or I would restore from backup on new volume groups and file systems and then rejoin it to a replica.

    Here are some commands examples:

    sudo lsfs -q /fs/name

    [...] inline log size: 512 [...]

    sudo chfs -a logsize=512 -a size=-50G /fs/name



    ------------------------------
    Edward Davignon
    ------------------------------

    Original Message


  • 8.  RE: Shrinking JFS2 file sytem

    Posted Thu June 24, 2021 05:50 PM
    shrinkfs (chfs) performance improvements are delivered in AIX 7.2 TL 5. Improved shrinkfs functionality minimizes the amount of time the filesystem is quiesced and also added a progress indicator to the chfs command output.

    ------------------------------
    Lakshmi Yadlapati
    ------------------------------

    Original Message