Originally posted by: oddjob

Hi,
I've been working on a migration project where I've been moving data from an ESS array to an HP XP array. The project is nearly complete but I've hit a snag with the last fc port on the last server I'm changing.

The server is a VIO box running at level 1.4.1.2-FP-9.2. and has a dual port fibre card. Port FCS1 has been successfully connected to the SAN, however port FCS0 won't connect through to the array although it is logging on to the fibre switch.

I've been asked by the storage guys to reboot the server.

As this is a live server, the change request to do a reboot will take weeks to sort out.

Any ideas???

I have removed all "fcs0" devices and rediscovered them using cfgmgr.
lsattr suggests all OK:

$lsattr -El fscsi0
attach switch How this adapter is CONNECTED False
dyntrk no Dynamic Tracking of FC Devices True
fc_err_recov delayed_fail FC Fabric Event Error RECOVERY Policy True
scsi_id 0x29e00 Adapter SCSI ID False
sw_fc_class 3 FC Class for Fabric True
#AIX-Forum

Originally posted by: oddjob

Just spoken to the Storage guy again. Aparantly, I'm logging on to the fabric successfully, but not the SAN. The second port is successfully logging onto the SAN without issue and I see devices down that single path.

Have started the change request for a reboot but if anybody has any ideas that will save me this pain then I'd be most grateful.
#AIX-Forum

Originally posted by: SystemAdmin

If you have done a rmdev -dRl fcs1 and then a cfgmgr a reboot will not help.

You say you see the disks on fcs0.

I am going to assume since this is not your first server being done the mpio driver for the XP storaqge is installed and you are using Active/Active setup.

If you do an lspaths , what does it dhow you ?

Does errpt say anything ?

I do both Storage , San and AIX where I am so I will ask a few things.

But first,

STORAGE most be created and assigned to the FCS's
Normally your storage team will slice and dice some LUNs for you to use. (( it appears that works there are some on the other fcs adapter )
Send them the wwpn of the adapter that has no storage ( lscfg -vl fcs0 | grep -i network ) and havethem validate it is correct

SAN Fabric
Now a ZONe must be create for this server and the new storage
If you have tow fabrics this must be done for both fabrics
Have them check that they have the correct wwpn in the zone and it has been added to the config

Server configs
Use cfgmgr -v to see if there are any strange things happening when the fcs adapters are probed.

IN the past these are the issues we have hit

1. The errpr shows that there are errors woth the fcs
REPLACE THE CARD
2. Mistypoe in the wwpn
Check the FABRIC and STORAGE to make sure the wwpn is correct
3. Zone created but not added to SAN configuration
Make sure the new zone is added to the configuration
4. Check storage to make LUNs are assign to both fibre cards
Storage team needs to validate the adapters are assigned
5. Check the adapter on the storage team is defined correctly.
IBM uses reports LUNs not walk the scsi bus.
Have the storage team checkl the adapter is correctly defined

Hope this helps
They will then assign on the XP the LUNs to the server.
#AIX-Forum

Originally posted by: oddjob

Both ports are on a single dual-port card. The mpio driver is installed and running, lspath shows the disks down the working path. There are no errors in "errpt".

I've run cfgmgr -v and there are no obvious errors

The storage team have double checked their config. The WWN has been confirmed as it can be seen logging on to the fabric.

The same procedure has been used on 3 other VIO servers and 6 lpars, all without issue.
#AIX-Forum

Originally posted by: tony.evans

Can't help specifically, but I do want to say, I've had two instances where a single HBA refused to show any storage.

In both cases, rmdev -dl didn't make any difference. Neither did a reboot. However, powering down the LPAR, and powering it back on, did make a difference (the adapter logged in correctly).

I can't comment on why, or if it's applicable, but if you do get the change raised, don't shutdown -Fr. Do a shutdown -h, wait 5 minutes, and then power the LPAR back on via the HMC.
#AIX-Forum

Originally posted by: Holgervk

>In both cases, rmdev -dl didn't make any difference. Neither did a reboot. However, powering down the LPAR, and powering it back on, did make a difference (the adapter logged in correctly).

I also made this same experience.

However, later I´ve noticed that it is enough to remove (with rmdev) the disks, the adapter AND the pci-slot of the adapter. Then, the adapter will be powercycled when running cfgmgr again.

Same happens, of course, when powering down the LPAR. However, removing the pcislot saves (down-)time.
#AIX-Forum

Originally posted by: tony.evans

That makes sense, my conclusion at the time was that power cycling the adapter (via the slot) was the solution, and that for some reason (maybe firmware) the adapter had failed to negotiate correctly with the SAN the first time, but wasn't able to start again from scratch correctly.
#AIX-Forum

Originally posted by: oddjob

I've raised my change for rebooting the server (which will really be halting and restarting the server but they'll only ask more questions...)

In the past, I'm sure I've been able to rmdev recursively from the PCI card itself as suggested. However, I've checked the VIO server and I can't see a device for the card (although I can see the hardware path)...

root@adm101/ $lsslot -c pci

1. Slot Description Device(s)

U787F.001.DPM2BYC-P1-C1 PCI-X capable, 64 bit, 133MHz slot ent2 ent3
U787F.001.DPM2BYC-P1-C2 PCI-X capable, 32 bit, 66MHz slot Empty
U787F.001.DPM2BYC-P1-C3 PCI-X capable, 32 bit, 66MHz slot Empty
U787F.001.DPM2BYC-P1-C4 PCI-X capable, 64 bit, 266MHz slot fcs0 fcs1
U787F.001.DPM2BYC-P1-C5 PCI-X capable, 64 bit, 133MHz slot Empty
U787F.001.DPM2BYC-P1-C6 PCI-X capable, 64 bit, 133MHz slot Empty

root@adm101/ $lsslot -c slot

1. Slot Description Device(s)

U787F.001.DPM2BYC-P1-C1 Logical I/O Slot pci10 ent2 ent3
U787F.001.DPM2BYC-P1-C2 Logical I/O Slot pci11
U787F.001.DPM2BYC-P1-C3 Logical I/O Slot pci5
U787F.001.DPM2BYC-P1-C5 Logical I/O Slot pci6
U787F.001.DPM2BYC-P1-C6 Logical I/O Slot pci7
U787F.001.DPM2BYC-P1-T5 Logical I/O Slot pci9 ent0 ent1
U787F.001.DPM2BYC-P1-T7 Logical I/O Slot pci8 usbhc0 usbhc1
U787F.001.DPM2BYC-P1-T10 Logical I/O Slot pci4 sisscsia0
U787F.001.DPM2BYC-P1-T12 Logical I/O Slot pci3 ide0
U9131.52A.066F35G-V1-C0 Virtual I/O Slot vsa0

Begining to think thatI'm losing the plot :(
#AIX-Forum

Originally posted by: oddjob

OK, I am losing the plot... I was looking at the wrong server :-P

Once (if) my change gets approved, I'll try recursively deleting pci1 and then running cfgmgr. If that doesn't work, I'll halt the box, go for a coffee and then bring it back up again. If that doesn't work, I'll log a call. And cry.

$lsslot -c slot | grep fcs
U787B.001.DNWFWVR-P1-C4 Logical I/O Slot pci1 fcs0 fcs1

$lsdev -Cc disk
hdisk48 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk49 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk50 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk51 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk52 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk53 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk54 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk55 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk56 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk57 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk58 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk59 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk60 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk61 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk62 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk63 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk64 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk65 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk66 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk67 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk68 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk69 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk70 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)
hdisk71 Available 01-09-02 XP MPIO Disk XP24000 (Fibre)

$lspath
Enabled hdisk48 fscsi1
Enabled hdisk49 fscsi1
Enabled hdisk50 fscsi1
Enabled hdisk51 fscsi1
Enabled hdisk52 fscsi1
Enabled hdisk53 fscsi1
Enabled hdisk54 fscsi1
Enabled hdisk55 fscsi1
Enabled hdisk56 fscsi1
Enabled hdisk57 fscsi1
Enabled hdisk58 fscsi1
Enabled hdisk59 fscsi1
Enabled hdisk60 fscsi1
Enabled hdisk61 fscsi1
Enabled hdisk62 fscsi1
Enabled hdisk63 fscsi1
Enabled hdisk64 fscsi1
Enabled hdisk65 fscsi1
Enabled hdisk66 fscsi1
Enabled hdisk67 fscsi1
Enabled hdisk68 fscsi1
Enabled hdisk69 fscsi1
Enabled hdisk70 fscsi1
Enabled hdisk71 fscsi1
#AIX-Forum

Originally posted by: jvk

Do you use all default values? What do show cmds like:

• lsattr -El fcs0
• lsattr -El fcs1
• lscfg -vpl fcs0

Next block/unblock switch port where fscsi0 is connected to, run the cfgmgr again.
Is it seen as F port there?
#AIX-Forum

Originally posted by: oddjob

All,
Thanks for your assistance - I've arranged an outage to complete the suggested tasks.
#AIX-Forum