Originally posted by: SystemAdmin

Fellows, i did some performance tests using compress and gzip. And the result was described below.

Do someone knows why CIO and DIO are so worst than JFS2?
I know FS buffer is extremely important for this issue, but do someone knows how to improve gzip and/or compress with CIO/DIO?

How to tune an AIX system to well perform the aplication (SAS) + cp/mv/gzip/compress/etc.. + dont overuse the LRUD deamon?

I enabled CIO on all application (SAS) FSs and did very well, but GZIP/COMPRESS didnt...

Is it a paradigm? hehe

Compress version:
(criticalbkp) >lslpp -l bos.rte.archive
Fileset Level State Description

---

Path: /usr/lib/objrepos
bos.rte.archive 5.3.8.1 COMMITTED Archive Commands

gzip version:
(criticalbkp) >gzip -V
gzip 1.3.12
Copyright (C) 2007 Free Software Foundation, Inc.
Copyright (C) 1993 Jean-loup Gailly.
This is free software. You may redistribute copies of it under the terms of
the GNU General Public License <http://www.gnu.org/licenses/gpl.html>.
There is NO WARRANTY, to the extent permitted by law.
Written by Jean-loup Gailly.

AIX Level:
(criticalbkp) >oslevel -s
5300-02-00-0000
Server configuration:
P595 - 2,3Ghz - 2 CORES - 44GB memory - SMT enabled
Standalone partition. The tests were did without concurrency
Vmstat sample:
(criticalbkp) />vmstat 1

System configuration: lcpu=4 mem=45312MB ent=1.00

kthr memory page faults cpu

---

-----------

---

------------

---

r b avm fre re pi po fr sr cy in sy cs us sy id wa pc ec
0 0 421602 10550097 0 0 0 0 0 0 8 65 185 0 1 99 0 0.01 0.9
0 0 421602 10550097 0 0 0 0 0 0 10 359 229 0 0 99 0 0.01 0.8
Test Set:
For all tests I compressed and decompressed 2 files (1GB and 5GB), under the following filesystems, umounting and mounting between each test.
1 - JFS2 ( named /teste-normal )
2 - JFS2 + CIO ( named /testec )
3 - JFS2 + rbrw ( named /tester )
4 - JFS2 + DIO ( named /tested )

Test example:
timex gzip File1GB (SAS file);
timex compress File5GB (SAS file);
etc...

Sumary Results:
Average gzip rate: 88% (1GB file), 77% (5GB file)
Average compress rate: 78% (1GB file), 73% (5GB file)

(real,user and sys seconds of cpu - just the major discrepancies of compressing are shown)
JFS2: compress lasts half the gzip real time and consumes almost the half user time

CIO: compress lasts 2-3X the gzip real time, consumes 6-8X user time, but a half sys time ...??

RBRW: compress lasts less gzip real time, consumes 1/3 less user time, but 1/3 more sys time ..?

DIO: compress lasts 2-3X the gzip real time, consumes 1/3 less user time, but 6X sys time ...??

Crazy results!!
If you compare the times among the FSs types the results are still awkward...
What is the solution? Maybe tunning even more the vmo and ioo?
What are your opinion?
I attached the complete test case results.

Best regards
Igor da Silveira Franco
#AIX-Forum

Originally posted by: dukessd

lslpp -l bos.rte.archive > bos.rte.archive 5.3.8.1

this looks like a 5.3 TL08 SP1 fileset.

oslevel -s > 5300-02-00-0000

but the system is at 5.3 TL02 !!!!

Sort this out first.
#AIX-Forum

Originally posted by: SystemAdmin

Hi!
Not all AIX filesets must be updated to become a TL2.

Like this example (TL8):
(ubbdwp02) /home/work/prod>oslevel -r
5300-08

(ubbdwp02) /home/work/prod>lslpp -l | more
...
X11.adt.bitmaps 5.3.0.0 COMMITTED AIXwindows Application
Development Toolkit Bitmap
Files
X11.adt.imake 5.3.7.0 COMMITTED AIXwindows Application
Development Toolkit imake
X11.adt.include 5.3.8.0 COMMITTED AIXwindows Application
Development Toolkit Include
Files
X11.adt.lib 5.3.0.0 COMMITTED AIXwindows Application
Development Toolkit Libraries
X11.apps.aixterm 5.3.8.0 COMMITTED AIXwindows aixterm Application
X11.apps.clients 5.3.0.60 COMMITTED AIXwindows Client Applications
X11.apps.config 5.3.0.50 COMMITTED AIXwindows Configuration
Applications
X11.apps.custom 5.3.0.0 COMMITTED AIXwindows Customizing Tool
X11.apps.msmit 5.3.8.0 COMMITTED AIXwindows msmit Application
X11.apps.rte 5.3.8.0 COMMITTED AIXwindows Runtime
Configuration Applications
X11.apps.util 5.3.8.0 COMMITTED AIXwindows Utility
Applications
X11.apps.xdm 5.3.8.0 COMMITTED AIXwindows xdm Application
X11.apps.xterm 5.3.8.1 COMMITTED AIXwindows xterm Application
X11.base.common 5.3.0.0 COMMITTED AIXwindows Runtime Common
Directories
X11.base.lib 5.3.8.0 COMMITTED AIXwindows Runtime Libraries
X11.base.rte 5.3.8.1 COMMITTED AIXwindows Runtime Environment
X11.base.smt 5.3.0.30 COMMITTED AIXwindows Runtime Shared
Memory Transport
X11.base.xpconfig 5.3.0.0 COMMITTED Xprint Configuration Files
#AIX-Forum

Originally posted by: j.gann

just to give you a hint: cio and dio are access methods which bypass the file system cache. they are intended for applications (DBMSes e.g.) which do their own caching and by knowing the semantics of their data (index, logs, whatever) can cache data much more successfully than the aix filesystem cache can. gzip and compress are not such applications. cio/dio is preferrably specified by an application which knows what it is doing at time of open()ing a file. the mount options cio and dio are somewhat crude shortcuts which force every open() of files on this filesystem to be in cio/dio mode.

Joachim
#AIX-Forum

Originally posted by: MarkTaylor

optimal block size for CIO is somewhere around 1MB I think and normal operations at std block sizes can take up to 7 times longer ..

HTH
Mark Taylor
#AIX-Forum

Originally posted by: SystemAdmin

Hi Mark!
Do you know how to enable 1MB block size in a FS???
I think JFS2 can be set up to 4KB block size.

Best Regards
Igor Franco
#AIX-Forum

Originally posted by: nagger

You could try dd'ing with large blocks off the disk and piping to the compress commands.
dd would then be some sort of "read ahead cache program" for you!

You could the dd back onto the disks but I am not so sure that will help.

Haven't tried this myself but it may help, ta N
#AIX-Forum

Originally posted by: SystemAdmin

Joachim, I know CIO/DIO doesnt use FS cache. The problem is, how do I perform with SAS (and indeed perform using CIO) and casual operations like cp, mv, compress, gzip and so on, that are dependent to cache in some cases?

Regards
#AIX-Forum