Informix

Hi Stavros, 
it's more about how the filesystem is configured. I prefer to have one filesystem for the cookfile chunks, but with 8-16 LUNS in a 128K stripe size. Also node to use DIRECT_IO 0x5 and so much BUFFERPOOL in different page sizes you can have on the machine. 
Best Regards, Henri

Stavros;

I would have one dbspace for rootdbs and temp dbsoaces, a PLOG dbspace for the physical log, a separate dbspace for your logical logs, then at least one dbspace for tables and another at a wide pagesize for any non-trivially small indexes. How many filesystems you use for that is up to you.

Remember that between 30 & 50% of your IOs will be to the logical logs and, depending on your apps, up to 30% to the temp dbspaces!

I agree with Henri about setting DIRECT_IO to include your temp dbspaces.

Art

Hello,

Thank you both for your answers.

@Art: Actually the instance is configured already similarly to the way you described:

Dbspaces
address          number   flags      fchunk   nchunks  pgsize   flags    owner    name
4bb61028         1        0x1        1        1        2048     N  BA    informix rootdbs
4bdf3660         2        0x2001     2        1        2048     N TBA    informix tempdbs
4bdf38a0         3        0x1        3        1        2048     N  BA    informix physdbs
4bdf3ae0         4        0x1        4        1        2048     N  BA    informix logdbs
4bdf3d20         5        0x8001     5        1        2048     N SBA    informix sbspace
4c5f5028         6        0x1        6        1        2048     N  BA    informix admindbs
4c5f5268         7        0x1        7        1        16384    N  BA    informix sopradbs
 7 active, 2047 maximum

Chunks
address          chunk/dbs     offset     size       free       bpages     flags pathname
4bb61268         1      1      0          150000     110812                PO-B-- /IDS/rootdbs_chk01
4c5f54a8         2      2      2          10239998   9731845               PO-B-- /IDS/tempdbs_chk01
4c5f6028         3      3      2          10239998   2375625               PO-B-- /IDS/physdbs_chk01
4c5f7028         4      4      2          15359998   11984945              PO-B-- /IDS/logdbs_chk01
4c5f8028         5      5      2          1023998    955002     955002     POSB-- /IDS/sbspacesdbs_chk01
                                 Metadata 68943      51302      68943
4c5f9028         6      6      2          511998     424720                PO-B-- /IDS/admindbs_chk01
4c5fa028         7      7      8          39321343   33202348              PO-B-- /IDS/sopradbs_chk01

In the current disk layout all system dbspaces (all except sopradbs in the output above) are lvs created in the same volume group and sopradbs (data dbspace) is created on a different volume group.

So i was wondering if the change from raw device to cooked files is not related to how the filesystem(s) should be structured or if there is something that i need to keep in mind for production systems. In a testing environment that is used only for dbexport and backup purposes i just migrated all chunks to the same filesystem and the instance is fine. But in this case there is no application accessing the database so it is not comparable to a production server.

@Henri@Art: Actually i read about DIRECT_IO in a similar thread opened in the community (see below)
raw disk support deprecated in RHEL 9.x | Informix
In this thread, the following was mentioned:

"With DIRECT_IO the performance hit versus RAW is down from 20-25% to barely 5% and sometimes less. 

However, that has been highly variable and I don't think anyone has figured out what effects the performance. Some systems do not perform any better with DIRECT_IO enabled than with it disabled using file system chunks and some even perform better without DIRECT_IO. So, you should do some major testing with a substantially identical host and storage with as close to the production database and operations as possible"

With that in mind i am wondering what the most "conservative" approach would be, meaning if i should start without DIRECT_IO and use it if i noticed any performance issues or if it would be better to set it up in the same day of the migration from raw disks to cooked files.

Best Regards,

Stavros

Stavros:

Yes, that arrangement looks mostly OK. I would use a PLOG type space for the physical log rather than a "normal" dbspace, but the benefit of doing that is small, so not a big deal.

So, on Linux, you no longer have access to RAW devices on most releases, that is true, but if you  upgrade to the latest v14.10 or to v15.0 three is DIRECT_IO support for using cooked devices in addition to using filesystem chunks as long as the pagesize of the dbspace is a multiple of 4K (support for cooked devices that are created with a 4K paging was not included in v14.10 until recently, so you probably need to upgrade). 
I would want the system, data, and index dbspaces on separate filesystems if I am using filesystem chunks, so I would set up three filesystems, perhaps on separate LVMs.

Yes, some users have found that using DIRECT_IO is slower than not enabling it, but in general, most installations do find DIRECT_IO to be faster than using AIO VPs as long as you configure sufficient KAIO resources in the OS.

Art

Hi,

I would start with DIRECT_IO turned on, keep it similar to the existing setup.

This is x86_64?

Which filesystem type? I have seen issues with XFS and writes to adjacent 2K pages on the same  4K block so I would use ext4.

Regards,

David.

Hello,

@Art: I like your idea regarding the filesystem separation. I have another UAT system which is closer to my PROD environment. Of course there are only a handful of users connected but the batch processes that are running on PROD are also executed in UAT and i think i can get a stronger indication of what to implement in PROD. The good news is that support for RHEL8 does not end until 2029 so i think i can make the transition on my UAT first and even let the system run for a few months before deciding on what to do on PROD.

I know that i will have to update informix version as well but at this point i would not want to mix the 2 (disk layout transition + informix update). But definitely this is the next task to perform.

@David: Yes this is a x86_64 running on xfs filesystems. Since there is support for ext4 filesystems in RHEL9 i think i would create the new filesystems on ext4 just to be on the safe side.

Best Regards,

Stavros

Stavros:

I STRONGLY disagree with David on using EXT4! Its copy-on-write strategy fragments file systems which can cause performance issues on magnetic drives and accelerate aging of SSD and other flash based drives, and you cannot disable that feature. In addition, EXT4 still writes meta-data changes before writing the corresponding data changes which has the potential to corrupt data. Finally EXT4 journaling is an unnecessary overhead for a file system that will only be used for databases that are already logging, and you cannot disable the journal safely (as Paul noted).

I personally like XFS, but if David is right and there are issues with that, my next choices would be JFS2 or EXT3 with journaling and copy-on-write disabled.

Hmm i think the next step would be to implement something in practice in order some measurable data from my environment.

I know that things do not always work the same way in different environments, so for the UAT i will keep the xfs filesystems as i am doing now with the raw disks.

Since i have the luxury of monitoring the system for quite a long time before the rollout in PROD it will give me enough time to determine if a change in filesystem type is justified or not

We manage many long-standing systems with IDS on Linux using block device logical volumes with DIRECT_IO 5. Running "onstat -d" proves it is using direct IO with the last chunk flag being "D" in all cases. No problems at all. The chunks are better protected not being in file systems, and don't get included in file system backups (level 0 archives are preferable). Expecting flak from some people on this thread...

Regards,
Doug

The only problem wilh using block devices with direct_io is if the block device is defined with a [age size that is a 4K. That was not supported until v15 and the later releases of v14,10 but only for dbspaces whose page size is a multiple of 4K. Not sure where it came in in v14.10, but rather recent. Block devices with 2K or smaller page size has been supported.

Art

As Art says, you will need a recent IDS version for DIRECT_IO with 4K block devices (Advanced Format storage):

https://www.ibm.com/docs/en/informix-servers/14.10.0?topic=parameters-direct-io-configuration-parameter-unix

https://www.ibm.com/docs/en/informix-servers/14.10.0?topic=io-kernel-asynchronous

https://en.wikipedia.org/wiki/Advanced_Format

That's from 14.10.xC13 or 15.0.1.x.

Use "onstat -d" to check it's working.

Hello,

@Art@Doug: Regarding the block device page size, are you referring to the page size set when creating the informix chunk or the block size of the lun that is created on storage system? If it is the second, i executed smartctl -a /dev/sde and the logical block size reported is 512 bytes.

Best Regards,

Stavros

Hi Stavros.

We did mean the disk device block size. If that is 512, then you have no problems with chunks defined on LUNs with DIRECT_IO 5 and any Informix page size.

Best practice is to create symbolic links to your LUNs and use those links when creating the chunks with "onspaces" or admin API SQL. That way, you have better migration options in the future. Also, you will need a "udev" rule so the LUN devices have owner and group "informix" with permissions like "chmod 660".