Originally posted by: sajid1

AIX 5.3
Oracle 10.2.0.4

I/O whenever reach between 90% to 100% of hdisks the system went hanged. What I explored in topas while I/0 goes to 90% the sr and sy is very high in the output of vmstat and also pageout and Sios are very high. Please suggest. Find below some outputs for your reference.

$ lsvg -o
datavg
rootvg

$ lspv
hdisk0 00046e630b53dbb6 rootvg active
hdisk1 00046e630b681f2e rootvg active
hdisk2 00046e636502ecbf datavg active
hdisk3 00046e63b525c2b8 datavg active

$ lspv -l hdisk0
hdisk0:
LV NAME LPs PPs DISTRIBUTION MOUNT POINT
hd10opt 1 1 00..00..01..00..00 /opt
hd1 4 4 00..00..04..00..00 /home
lg_dumplv 8 8 00..08..00..00..00 N/A
hd5 1 1 01..00..00..00..00 N/A
hd8 1 1 00..00..01..00..00 N/A
hd6 77 77 00..77..00..00..00 N/A
hd2 12 12 00..00..12..00..00 /usr
hd4 4 4 00..00..04..00..00 /
hd3 24 24 00..00..24..00..00 /tmp
hd9var 8 8 00..00..08..00..00 /var

$ lspv -l hdisk1
hdisk1:
LV NAME LPs PPs DISTRIBUTION MOUNT POINT
hd10opt 1 1 00..00..01..00..00 /opt
hd1 4 4 00..00..04..00..00 /home
lv02 7 7 00..07..00..00..00 /mkcd/cd_images
loglv01 1 1 00..01..00..00..00 N/A
hd5 1 1 01..00..00..00..00 N/A
hd8 1 1 00..00..01..00..00 N/A
hd6 77 77 00..77..00..00..00 N/A
hd2 12 12 00..00..12..00..00 /usr
hd4 4 4 00..00..04..00..00 /
hd3 24 24 00..00..24..00..00 /tmp
hd9var 8 8 00..00..08..00..00 /var

$ lspv -l hdisk3
hdisk3:
LV NAME LPs PPs DISTRIBUTION MOUNT POINT
fslv00 350 350 00..217..83..00..50 /vol01
fslv01 570 570 218..00..00..190..162 /vol02
loglv00 1 1 00..01..00..00..00 N/A
fslv02 155 155 00..00..134..21..00 /vol03

Thanks
#AIX-Forum

Originally posted by: alethad

Identify the disks that have the high usage. What's on those disks specifically that is bringing them to it's knees? What are the users doing specifically. High disk usage can cause the system to pause.

You stated the paging is high? Have you checked it? How much paging do you have and how much RAM?
If you don't have enough paging the system can pause to wait for processes to finish.

I would suggest looking into Nmon to help you diagnose your issue. Performance cannot be determined by one or two commands. You need to look at the whole picture. It is possible to have more than one performance issue and not know it. Or more than one contributing factor to the issue.

Good luck.

You've got to continue to grow, or you're just like last night's cornbread -- stale & dry Loretta Lynn alethad
#AIX-Forum

Originally posted by: sajid1

I am working on that since some days back but could not reach the conclusion. I will try to use nmon as you said and let me know. As you need to know about paging and RAM please find below.

$ lsps -a
Page Space Physical Volume Volume Group Size %Used Active Auto Type
hd6 hdisk0 rootvg 20480MB 17 yes yes lv

$ lsattr -El sys0 | grep realmem
realmem 16318464 Amount of usable physical memory in Kbytes
False

Many Thanks
#AIX-Forum

Originally posted by: alethad

One more thing.....Is this a database server such as Oracle or DB2?
If so have you tuned the system for it?

You've got to continue to grow, or you're just like last night's cornbread -- stale & dry Loretta Lynn alethad
#AIX-Forum

Originally posted by: sajid1

One more thing.....Is this a database server such as Oracle or DB2?
--Yes it is Oracle
If so have you tuned the system for it?
--No

Thanks
#AIX-Forum

Originally posted by: sajid1

In topas there are page in and page out if it is high (high means how much value) then do it needs more RAM?

I observed in topas page in and out is high but paging space percentage is not changing it is always same used and free?
#AIX-Forum

Originally posted by: alethad

Before you start going any further. Go check out this document and read it. It is very good and just got updated back in the fall. You'll want to download a copy for yourself to refer back to it for future use.
The actual tunables you are concerned with are on page 32. Go tune your system first and see what settles out.

http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100883

If you don't know how to tune your Oracle get your DBA to do it for you.
Good luck.

You've got to continue to grow, or you're just like last night's cornbread -- stale & dry Loretta Lynn alethad
#AIX-Forum