Hi, Henri,
you are absolutely right about new hardware and I/O tech. The problem however remains. If a single (local) device is being used for storage, by Informix or not, it will fail eventually. In some cases it will fail after whole system or storage has been upgraded, however it's very likely to fail when the system is still in operation if the system remains in use for several years.
For longer use scenarios there is a need to build some degree of redundancy into the storage solution. With the external storage (SAN), be it based on flash, NVMe, hybrid, of even just disks, such redundancy usually comes with the device, the extra thing to do would be to use multipathing - which would remove single point of failure on the adapter/port level.
If, for various reasons, external storage solution is not desired, it does make sense to create storage redundancy within local system. And here David is right to use SVM or Informix mirroring for the piece of mind.
The argument, I guess, emerged from mentioning possibility of using higher RAID levels (5,6), which had (in the past!) quite bad reputation in terms of performance and most importantly the condition known as the "write hole" - which may cause data corruption if power loss happens during write.
If the decision is made to use redundant storage solution local to the system, next question will be - which level of RAID is more appropriate?
As usual - there is no straight answer to this question, my comment was only to highlight the fact that stated deficiencies of the higher RAID may not apply for the current hardware, not to say that RAID-6 is better than mirror. (Each solution is "better" in it's own way)
To illustrate - let's go through several conceptual examples, which use locally attached storage only.
If you run on SPARCstation 10 or 20 - pretty much your ONLY possible solution will be mirror or SVM.
Interestingly, if you run on system with the modern motherboard supporting only 2 NVMe drives - the mirror would also be a reasonable choice.
Note that is both these cases it's not possible to replace failed mirror device without shutting down the system (in SPARCstation it may be possible only if the failed device is external)
However, if, for example, one has 2U chassis with 8 to 24 disk bays, let's say DELL R740, with the high-end storage controller and more than 6 disks to be used for storage, and these disks are SSD or better, it can be demonstrated that pretty much all the bad things said about RAID-6 may be true but are not practically applicable.
In fact using separate SSDs with Informix (no redundancy on the device level!) versus volumes carved from RAID-6 (double parity!) over same SSDs demonstrated better performance for the latter. As for the write hole - deployments with UPS, current design of the system's power supplies, and the controller with flash cache and supercapacitor most likely make it a non-issue. Though I cannot say that I had direct experience in this matter, as the occasions when busy systems went down on me due to power loss were extremely rare.
That's all I was meaning to convey when mentioning RAID-6 as a possible solution, since the old-school arguments for NEVER considering these RAID levels may not be fully applicable in 21 century.
------------------------------
Vladimir Kolobrodov
------------------------------
Original Message:
Sent: Fri October 02, 2020 04:48 AM
From: Henri Cujass
Subject: RAID6?
Hi David, Vladi, Art,
nice discussion but old fashion. I suggest a "waitless i/o/" by using NVMe/Flash storage (local or not). The throughput will depend from RAM and CPU frequence in this case, only. My new test with 14.10 EE, ARM64, Ubuntu on NVIDIA Jeston Xavier NX => 400.000 inserts/second, 50 seconds for a parallel index build on a 100 Mio records table and 650.000 scanread/second. That's the future on a <1.000 $ system :-)
Best regards,
Henri
------------------------------
Henri Cujass
leolo IT, CTO
Germany
Original Message:
Sent: Fri October 02, 2020 03:55 AM
From: David Grove
Subject: RAID6?
Part way through the tech support case, I decided to disregard SVM, and use the built-in Informix mirroring. Just for my own edification, I wanted to follow-up with the tech support case to see if there was any firm conclusion available concerning the safety of SVM. It seemed to me, from information presented from tech support, that Informix didn't do the kind of write-on-write that was most likely what Oracle had in mind in their caution in the 'man metainit' pages. But, I couldn't be sure because I didn't know for sure what they had in mind.
In any event, I decided that Informix mirroring could do enough of what we want, and with guaranteed safety, so that's the way we're going.
I really do appreciate everyone's thoughts.
At the end of the day, though, as long as there are suitable alternatives ("suitable" meaning effective, safe, and economic), I think we'll continue to pass on RAID5 & RAID6, etc.
In this present case, we find Informix mirroring to be suitable.
With full respect to all,
DG
P.S. Vladimir, I meant no disrespect to you or Informix tech support. (Note that I acknowledged in my initial post that your first recommendation was to use SAN.)
P.P.S. I guess I am sort of a pessimist, at heart. I always feel that the old principle, "If something can go wrong, it will go wrong," is lurking behind me. So, even if something is unlikely, if there is a way to avoid the possibility (rather than rely on the unlikelihood of an occurrence), I'll choose that. I think there is sort of a saying among our friends, the physicists, to the effect that (unlikely) phenomena which are not prohibited are guaranteed, eventually, to occur.
------------------------------
David Grove
Original Message:
Sent: Thu October 01, 2020 10:12 PM
From: Art Kagel
Subject: RAID6?
David:
I have not given up researching. RAID6 is only marginally better than RAID5 as far as data safety and is even SLOOOWWWERRR for write performance than RAID5 since two checksums have to be calculated and written to two different drives. Stick with RAID10.
That said, you should probably not be using SVM for mirroring RAW chunks or any chunk being used with RAW or DIRECT_IO due to the write-on-write issue. There I agree with HCL tech support. SVM is software mirroring which is the slowest and least reliable. It would be best to use a better LVM that does mirroring properly in parallel or, as suggested, use a good SAN that implements mirroring properly in firmware, again with parallel writes.
If you MUST use SVM, I would suggest using SVM to create two stripes then use Informix mirroring to mirror them effectively creating a RAID01 array which is as fast and as reliable and nearly as safe as a proper RAID10, just slower to recover from since the entire stripe has to be recovered.
Art
Art S. Kagel, President and Principal Consultant
ASK Database Management
Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves.
Original Message:
Sent: 10/1/2020 7:20:00 PM
From: David Grove
Subject: RAID6?
I'm lazy. So, when I make a decision on some things, I stop making explicit efforts to continue following or researching it.
An example of this is the whole RAID thing. Years ago, I read much of Art's material, and decided that RAID10 was "it" for the systems that I am responsible for. So, I went forward thinking that RAID levels were something I didn't have to agonize over any more. "Settled science", as it were. :)
Imagine my mild surprise when, in a recent support case with Informix Tech Support, in which I was asking about the safety of using Solaris SVM for Informix mirroring, and, in the final case update, they suggested using RAID 6.
(After reading Oracle's "WARNING" near the bottom of the man pages for "metainit', in which they caution about the susceptibility of SVM to write-on-write situations, I opened a priority 4 tech support case to inquire whether using SVM mirrors (configured as RAID10) was a potential risk.)
After several months, they concluded that the answer really depended on Oracle's envisioned use case, and Informix was unable to give a definitive answer without more detail about Oracle's use case that motivated the warning.
But, they did suggest that a good solution would be, first, external SAN storage; and, second (if external storage was not available) using internal drives configured as RAID6.
They also suggested, as a possible third choice, ZFS, but cautioned that it was resource intensive, and might not be suitable to use on the same system as Informix.
What caught my attention was their recommendation to use RAID-6.
Has the world changed? Do I need to revisit and reinvestigate this whole RAID thing (which I thought I had settled in my own mind years ago)?
I don't want to be one of "those guys" who, being resistant to change and new knowledge, are forever living in a static past.
Curious,
DG
------------------------------
David Grove
------------------------------
#Informix