Informix

 View Only

  • 1.  Journaling file system

    Posted Fri March 19, 2021 10:57 AM

    Some Informix DBAs believe "journaling" should be turned off for file systems that hold Informix chunks. The idea is that Informix does its own journaling, so the operating system journaling is unnecessary and possibly harmful.

     

    http://infxms.blogspot.com/2012/04/file-systems-for-informix.html

     

    http://informix-myview.blogspot.com/2010/07/new-journaled-filesystem-rant.html

     

    We are using the ext4 file system type on our Linux machines. It has advantages over prior types ext2 and ext3 in that it allows larger files to be stored.

     

    Here is an article about how to know which type of file system is being used:

    https://www.tecmint.com/find-linux-filesystem-type/

     

    Here is an article about how to turn off journaling for an ext4 file system partition:

    https://foxutech.com/how-to-disable-enable-journaling/

     

    I'm hoping I can get some more comments on this subject. Does anyone know of documentation from an official IBM / Informix source on whether file system journaling should be off? If so, I want to share it with our Operations staff, because they are preparing a new machine that will have an Informix database.

     

    clerk11s

    John Dargan, Database Administrator II - Information Technology

    LAURA E. ROTH, Clerk of the Circuit Court

    Seventh Circuit, Volusia County, Florida

     

     

     

     

     


    #Informix


  • 2.  RE: Journaling file system

    Posted Fri March 19, 2021 11:20 AM

    John,

     

    You can also consider xfs as a valid option. Art Kagel has been looking deep into that option and it seems to present many advantages, in addition to being a more current technology

    • Although a journaling technique, it only considers metadata journaling, making it faster
    • Stripe-aware file system
    • Tunable filesystem block sizes from 512 to 64K
    • DIRECT_IO support built-in using DMA operations! VERY important to use DIRECT_IO as XFS uses lazy writes
    • XFS can be expanded without unmounting

      HE should have published this info in his blog, not sure he did       ��

      My 0.2 cents

     

    Eric Vercelletto
    Data Management Architect and Owner / Begooden IT Consulting
    KandooERP Founder and CTO
    IBM Champion 2013,2014,2015,2016,2017,2018,2019,2020
    ibm-champion-rgb-130px

    Tel:     +33(0) 298 51 3210
    Mob : +33(0)626 52 50 68
    skype: begooden-it
    Google Hangout: eric.vercelletto@begooden-it.com
    Email:     eric.vercelletto@begooden-it.com
    www :     http://www.vercelletto.com
    www  https://kandooerp.org

     

     




    Original Message


  • 3.  RE: Journaling file system

    Posted Fri March 19, 2021 11:46 AM
    Here are some resources that I know about:

    A page on the PostgreSQL site that talks about one of the logging modes of the system and recommends no using journaled filesystems:
    https://www.postgresql.org/docs/8.4/wal-intro.html

    Here's a discussion more generally about using journaled filesystems with applications that do their own logging:
    http://tinco.pair.com/bhaskar/gtm/doc/books/ao/OpenVMS_manual/ch06s01.html


    ------------------------------
    Art S. Kagel, President and Principal Consultant
    ASK Database Management Corp.
    www.askdbmgt.com
    ------------------------------

    Original Message


  • 4.  RE: Journaling file system

    Posted Fri March 19, 2021 11:55 AM

    The last time I tested this – a year ago – in the customer environment ext2 was still the faster of all the ext*, xfs examined.  However, they settled on xfs due to the ease of admin

     

    Cheers

    Paul

     




    Original Message


  • 5.  RE: Journaling file system

    Posted Fri March 19, 2021 11:47 AM

    Some thoughts on this subjects maybe not directly related to the question whether to use "journaling"  or not.

    Recent hardware saw significant improvements  in CPU processing capabilities, amounts of available memory and even disk performance - if we take into account (enterprise) SSD, NVMe, or all-flash storage. With current hardware, in my opinion, the emphasis needs to be made either on avoiding  I/O as much as possible (larger buffer pool) or using all-flash storage solutions to have "balanced" configuration that would take full advantage of the increased CPU power. Otherwise, as soon as Informix touches disk (aside from logging) - performance would be greatly impacted, with fairly little difference of whether the file system is journalled or not.

    On the matter of turning off journaling for ext4 or other FS. Journaling mostly comes into play when the system goes down unexpectedly due to sudden power loss. In such case - after restart - without journal there will be a need to run file system check (fsck - executed before system can mount the fs) which can take long time, depending on the underlying storage speed and size/contents of the file system. Note that Informix logging  is not a replacement for FS journal - Informix will not even be able to access the logs and start it's recovery until the file system is made consistent by fsck and mounted. Sometimes fsck will fail - and require administrator's assistance to make a decision - so the system may not even come up without intervention from the operator.

    Journaled files system, from practical experience, takes seconds to replay log when restarted after crash, and system becomes alive much faster.

    Therefore - if system can never crash - yes, the journal can be safely disabled, and yes, it will result in (slightly) improved I/O performance. However any significant amount of I/O will slow down RDBMS, journal or no journal, unless storage is really fast (NVMe or all-flash) and in that case overhead from journaling is also negligible.

    Note that "raw devices" don't have these problems (unless write cache to storage device is enabled without equivalent of battery backup)

    To conclude - for overall safety reasons, personally I'd avoid turning off journal. If I/O is a problem - just use adequate amount of memory for the Informix buffer pool (+ direct I/O, of course), if there is still not enough RAM and I/O significantly affects performance - consider investing into all-flash storage.



    ------------------------------
    Vladimir Kolobrodov
    ------------------------------

    Original Message


  • 6.  RE: Journaling file system

    Posted Fri March 19, 2021 05:54 PM
    Vladimir:

    Here are my thoughts:

    First, I would not use EXT4 with journaling enabled unless it is mounted with journal_data_ordered because the original default of journal_data_writeback is dangerous. 

    Next, I hear your concerns about startup after a crash, but on a filesystem that is dedicated to Informix chunks, there is rarely any change to the metadata. New files added to the FS are rare and there is a relatively long period between creating a new chun file and using it for data. Existing chunks are rarely expanded on most systems. Yes, there are systems that often extend chunks and allocate new chunks from storage pools which either expands a pool file or creates a new chunk file, I know that, but again, I believe that the possibility of a system crashing immediately after adding to the dbspaces and after beginning to put important data into them, but before the FS can safely update itself on disk, especially with DIRECT_IO enabled, is vanishingly unlikely. 

    For these reasons, if I have to work with a journaling filesystem my preference is XFS or JFS because they are meta-data only journaled and, as I noted, the meta-data doesn't change very often and when it does, it will be a single simple entry. These filesystems are not being used by applications that write, extend, and delete files willy nilly constantly. Filesystems holding Informix chunks are highly stable with few changes to the meta data. 

    Now, meta-data only journals are at risk for data file corruption, but Informix running with DIRECT_IO is writing directly to disk and writing logs before data, so the risk of data loss is minimal. 

    That's my take. Someone has to explain to me where the failure is in my logic here.




    ------------------------------
    Art S. Kagel, President and Principal Consultant
    ASK Database Management Corp.
    www.askdbmgt.com
    ------------------------------

    Original Message


  • 7.  RE: Journaling file system

    Posted Fri March 19, 2021 11:58 AM

    If they are preparing a machine for you then you have an opportunity to some testing .... And then argue from a position of having data.

     




    Original Message