Originally posted by: SystemAdmin

Dear All

i got error yesterday on my power P510 , and after rebooting the system hangs on AIX logo (when system starts) .
After investigation and check all my HD from SMS , i found that HD that holds the rootvg and AIX installation is FAILED .
(the server connected to a SAN storage contains Oracle Application ) , but no errors from SAN SCSI Disks .

I made a new installation on a new HD .. and when login , I did NOT find the :
oraclevg (Oracle data Volume group )and the 3 Filesystems was related to PV assigned to this VG .

I only find the rootvg (new) with only one HD that had the installation .

This is a Production Server ( i have backup from Oracle files but not AIX SYStem backup ) ..

Any help to restore the Previous VG and FS without losing data ..
Sorry , Iam not experienced enough in AIX .

Please ASSIST ASAP

Thanks
Abuzeid
#AIX-Forum

Originally posted by: SystemAdmin

Hi Friends ...

Waiting your assistance ..
Its a production server , and i need a scenario to restore previous VG ,PV and FS configuration .

Thanks
#AIX-Forum

Originally posted by: orphy

It's not a very good idea to wait for someone on this forum to help you recover a prod system!

Don't you have AIX support for this prod system? If so, you should have called them with a sev 1 PMR by now (800-IBM-SERV) If not, assuming there is no change on the SAN side, you should have the Oracle hdisk back after the OS reinstall. If all you had on this box was rootvg and oraclevg, do a lspv to find out the name of the unassigned PV then do an importvg -y oraclevg hdiskX. You should then be able to mount your Oracle filesystems. You shouldn't have lost any Oracle data other than those on flight ones when the box crashed.

You should plan on doing mksysb periodically.
Orphy
#AIX-Forum

Originally posted by: SystemAdmin

Hi Orphy

Thanks a lot for your immediate response .
As i understood from your previous reply , that i can restore my data (FS) using importvg -y oraclevg hdiskX.
That mean i should Know what exactly PV was assigned to this Oracle VG . (what to do if not , especially i have 9 HD in the SAN storage ???)

Could you give me a detailed steps to do that .

Thanks
Abuzeid
#AIX-Forum

Originally posted by: orphy

Are all nine PVs belonging to oraclevg? If so, just pick any unused one and do a importvg. Then do an "lsvg -l oraclevg" too see if those are the oracle FS in that VG. If so, you go on to mount the FS. If you don't see the Oracle FS, you have more than 2 VGs on this box. You then need to do a "varyoffvg oraclevg" and an "exportvg oraclevg". Repeat the same step using another PV. You will eventually find the PV belonging to oraclevg. Chances are that you have more than one PVs in the oraclevg but all you need is one PV for importvg since the VGDA on each hdisk knows all the PVs in the same VG. The importvg/exportvg won't harm your user data so no worry about trying it.
Orphy
#AIX-Forum

Originally posted by: ColombianJoker

Hello.

Can you tell us:
How many VGs you had before?
Can you show us the output of:
lspv
lqueryvg -Atp $(lsdev -c disk -F name | tail -n1)
#AIX-Forum

Originally posted by: SystemAdmin

Hello ,

i think i facing a problem regarding to the SSA SAN storage ,,
i cannot see or detect the SAN physical volumes from command #lspv or from smit tool .

How to ensure the SAN storage connected properly ? (from command line or smit) may this help to solve my issue .

do you need me to upload any command result or any log file ?
Thanks
Abuzeid
#AIX-Forum

Originally posted by: orphy

Do you know if you have AIX support for this box? If so, call them! If you are not sure, call them to check.

Are these 7133 SSA? I haven't touched them for a long time so don't remember a whole lot about them. You can run cfgmgr to see if it complains about missing any device drivers since you reinstalled AIX. Check the hardware to make sure they are good (i.e. no fault lights, still power up, etc).

Do you see any errors when you run errpt?
Orphy
#AIX-Forum

Originally posted by: SystemAdmin

Hi abuzeid_maher,

It seems you need to install Powerpath and ODM on your Server, as you said, you made fresh install the Server after it failed.
Once you installed the Powerpath and ODM, you may now run commands to detect your SAN disks, anyway, you didn't mention if your SAN is Clariion or DMX.

Regards,
Zieggy
#AIX-Forum

Originally posted by: SystemAdmin

Hi , all
Now i can detect the SSA SAN storage (replacing Fiber cables).
when booting in maintenance mode , i can see the VGs .

Also , i made a fsck as per note :http://www-01.ibm.com/support/docview.wss?rs=111&context=SWG10&dc=DA480&dc=DB520&q1=LED+518&uid=isg3T1000131&loc=en_US&cs=utf-8&lang=en

fsck /dev/hd4
fsck /dev/hd2
fsck /dev/hd3
fsck /dev/hd9var
fsck /dev/hd1

and i could mount (/ , tmp , usr , and var filesystems) .

I found that the hostname for the server is changed to localhost !!!
i change it from smit and sync;sync;sync;reboot

after normal booting , the previous error disappear.
but after code 0c32 the display has no code and system stuck and no response.(i tried many times).

any advice .

thanks
#AIX-Forum

Originally posted by: orphy

You were able to boot up AIX without any issue before. So are you saying now that somehow rootvg got in trouble after you you replaced some cables? Do you have a console/monitor attached to this server? If so, answer the question on the console to select that as the console display. After you boot up the box, use "smitty hostname" to change the hostname.
Orphy
#AIX-Forum

Originally posted by: SystemAdmin

hi ,
each time i booted in service mode , i found the hostname back to localhost ,
and didnot changed ,
also, i couldn't up the en0 lan (may be in single user mode) .
how to permanent change the hostname from maintenance mode ?

thanks
#AIX-Forum

Originally posted by: SystemAdmin

hello ,

the status now is :
aix booting and code 0c32 appear on the code display .
it supposed to get aix screen (services starting ) , but the screen go offline .
and stuck in this ..
as you said , its a console issue . could you tell me how to fix it please ..
can i set the console from SMS menu ??? or from where ?

thanks
#AIX-Forum

Originally posted by: SystemAdmin

Hello ALL

I installed a new AIX on another HD .
and i can see all SAN SSA disks ,but the both old Volume groups not shown for sure .
i need to restore my volume groups (oravg and datavg) with my old configuration and Filesystems .
The first VG (oravg) was on one ligical volume lv0 and contains (8 physical volumes HD)on the SSA SAN .
i tried to create a VG with the same old name oravg but he asked me to assign a physical volume to it .
Does this will affect my data on the HDs , and as i told you i have multiple volumes HD was assigned to this VG ,
how to add them all ?
could you please tell me how to restore this in details .
I am sorry , but its a critical issue now ...
Please Assist ..

Regards

Abuzeid
#AIX-Forum

Originally posted by: orphy

Do not make a VG with those PVs or your data will be gone! Did you ever try my suggestions (posted Apr 28, 2012 11:51:47 AM above) using importvg/exportvg? If not, you should.
Orphy
#AIX-Forum

Originally posted by: SystemAdmin

Hello ,

ok ,i will use importvg .
but i have mutiple physical volumes assigned to these volumegroups .
could you provide me with commands >

Thanks
#AIX-Forum

Originally posted by: orphy

Don't worry about the number of disks you have in oraclevg. Run these commands.

lspv # find the first unassigned PV, e.g. hdisk1
importvg -y oravg hdisk1
lsvg -l oraclevg

If you find lv0, you can mount the oracle filesystem and move on to do lspv to find the other unused PVs to importvg for datavg. If you don't see lv0, this is probably datavg. Then do these

varyoffvg oravg
exportvg oravg
importvg -y datavg hdisk1
lsvg -l datavg
mount ...

Once you have datavg, use lspv to find the other 8 unused PVs (HDs) and use one of them to do importvg for oravg as mentioned above.
Orphy
#AIX-Forum

Originally posted by: orphy

I meant oravg if my phone changed it to oraclevg for me so just use oravg.
Orphy
#AIX-Forum

Originally posted by: SystemAdmin

Hello Orphy ..

i got this error when try to varyon the volume group:
0516-082 varyonvg unable to access a special device ..

Thanks
#AIX-Forum

Originally posted by: orphy

What was the varyonvg for? Can you show me outputs of these commands?

history
lsdev -Ccdisk
lspv
Orphy
#AIX-Forum

Originally posted by: SystemAdmin

Hello Orphy ,

Good News : the OLD system is up and running now ..
(i use diagnostics from service mode and found a problem related to Monitor , after change it
the AIX start in Gui mode ).

BUT , the BAD news is : i could NOT find my oracle File systems /oradata and /oratech ..

---

Here are the requested commands output :

bash-3.2# lsdev -Ccdisk
hdisk0 Available 1S-08-00-5,0 16 Bit LVD SCSI Disk Drive
hdisk1 Available 1S-08-00-8,0 16 Bit LVD SCSI Disk Drive
hdisk2 Available 1n-08-L SSA Logical Disk Drive
hdisk3 Available 1n-08-L SSA Logical Disk Drive
hdisk4 Available 1n-08-L SSA Logical Disk Drive
hdisk5 Available 1n-08-L SSA Logical Disk Drive
hdisk6 Available 1n-08-L SSA Logical Disk Drive
hdisk7 Available 1n-08-L SSA Logical Disk Drive
hdisk8 Defined 1S-08-00-4,0 16 Bit LVD SCSI Disk Drive
hdisk9 Available 1S-08-00-4,0 Other SCSI Disk Drive
hdisk10 Available 1S-08-00-3,0 Other SCSI Disk Drive
hdisk11 Available 1n-08-L SSA Logical Disk Drive
hdisk12 Available 1n-08-L SSA Logical Disk Drive
hdisk13 Available 1n-08-L SSA Logical Disk Drive
hdisk14 Available 1n-08-L SSA Logical Disk Drive

---

bash-3.2# lspv
hdisk0 005fbbaf82025c36 rootvg
hdisk1 005fbbafa44aa990 rootvg
hdisk2 005fbbafc376f887 oravg
hdisk3 005fbbafd5591e3c oravg
hdisk4 005fbbaf7692daf5 oravg
hdisk5 005fbbaf76933c62 oravg
hdisk6 005fbbaf3d8dd464 oravg
hdisk7 005fbbaf0a9b49e4 rootvg
hdisk9 005fbbafb58c961d datavg
hdisk10 005fbbaff33a9a29 datavg
hdisk11 005582dae5772092 None
hdisk12 005582dae5a9cf7c None
hdisk13 005582dae5c1a85d None
hdisk14 005582dabbd16004 None

---

bash-3.2# df
Filesystem 512-blocks Free %Used Iused %Iused Mounted on
/dev/hd4 655360 578632 12% 1747 2% /
/dev/hd2 8388608 6879896 18% 20960 2% /usr
/dev/hd9var 131072 31512 76% 2063 13% /var
/dev/hd3 7077888 1493696 79% 705 1% /tmp
/dev/hd1 131072 126808 4% 22 1% /home
/proc - - - - - /proc
/dev/hd10opt 393216 339792 14% 481 1% /opt
/dev/lv01 110100480 24655256 78% 882498 25% /oraapp

---

Check this also :

bash-3.2# mount -a
mount: 0506-324 Cannot mount /dev/hd1 on /home: The requested resource is busy.
mount: 0506-324 Cannot mount /proc on /proc: The requested resource is busy.
mount: 0506-324 Cannot mount /dev/hd10opt on /opt: The requested resource is busy.
mount: 0506-324 Cannot mount /dev/lv02 on /oratech: A file or directory in the path name does not exist.
mount: 0506-324 Cannot mount /dev/lv00 on /oradata: A file or directory in the path name does not exist.
mount: 0506-324 Cannot mount /dev/lv01 on /oraapp: The requested resource is busy.
mount: 0506-324 Cannot mount /dev/datalv on /data: A file or directory in the path name does not exist.
bash-3.2#

I could not find the /dev/lv02 ,/dev/lv00 ,/dev/datalv ..

Waiting your Assistance , and thanks alot

Regards
Abuzeid
#AIX-Forum

Originally posted by: orphy

Are you sure you don't have another VG containing /oradata and /oratech? Would they be in hdisk11-14? Those SSA disks are unassigned and you only have 5 PVs in oravg but you said oravg is supposed to have 8. Why not try importvg using hdisk11-14 to see what you have there?

If you can't find them there, you should just recreate them and restore. I think you mentioned having backups.

Orphy
#AIX-Forum

Originally posted by: SystemAdmin

Please find the attached file contains /etc/filesystems

Thanks
#AIX-Forum

Originally posted by: SystemAdmin

Hello ALL ,

thanks for you all , especially Orphy .

I used redefinevg command to make varyonvg work , then synclvodm .
After that i mount -a and then found all my data under filesystems ..
you can contact me for further details ..

Thanks again ..

Abuzeid
#AIX-Forum

Originally posted by: orphy

Good for you! After everything is settled down, you should put a backup/recovery plan in place. I suggest doing a mksysb at least once a month and maybe run a set of command to save your system info for future reference. If you don't want to figure out what to save, look into the snap command and simply save its output which usually have enough info on another box. Good luck with your new venture!
Orphy
#AIX-Forum

Originally posted by: SystemAdmin

Thanks ALL for support .

and i will schedule a system backup monthly , and databackup weekly ..

Regards

Abuzeid
#AIX-Forum