Originally posted by: SystemAdmin

I am hoping somebody with Flashcopy experience on AIX may be able to help. We have had a few issues with the scripts we have been running to import Flashcopies of our databases to our P series host nightly to dump to tape; mostly centering around repository sizing which I have pretty much resolved.

I am struggling with a couple of legacy issues though. First of all we need to add some more disk (LUNS) from the array, and I am concerned about conflicts with pvids on the scripted Flashcopies.

Currently our import/export runs something like this :

Import:

• Recreate Flashcopy with SMcli(FAStT700)
• LUN Mapping with SMcli(FAStT700)
• hot_add (AIX cfgmgr) to import the disk devices to the host
• chdev -l hdisk -a pv=clear (AIX) to clear the source LVM data structures from the FC's
• recreatevg (AIX) to recreate the Flashcopy Volume Group
• mount filesystems

Export:

• umount filesystems (AIX)
• varyoffvg (AIX) varyoff the FC volume group
• exportvg (AIX) export the FC volume group
• rmdev -l hdisk -d (AIX) completely remove the FC devices from the ODM
• Remove LUN mapping with SMcli(FAStT700)
• Disable the Flashcopy with SMcli (FAStT700)

Problem is as we completely remove the FC pvs from AIX each time, they only keep the same pvids each time for scripting purposes because we never add any other volumes to the system.

Somebody on another forum suggested that we should only remove the FCs from AIX using rmdev -l hdisk (no -d flag) in order that they remain defined in the ODM. That theory seems to work; but unfortunately when re-importing the volumes as soon as you run 'chdev -l hdisk -a pv=clear'I can not get the 'defined' volumes to switch back to 'available', and subsequently re-running hot_add (cfgmgr) just brings the FCs on line with new pvids, leaving the original ones as 'defined'.

I have tried throwing a 'chdev -l hdisk -a pv=yes' into the equation and even a 'mkdev -l hdisk' into the mix in various orders but nothing seems to work. Has anybody got any ideas how I can effectively 'hardcode' the FC pvids in the ODM and reuse the same ones each time ?

The other issues we have experienced are failures due to either 'chdev -l hdisk -a pv=clear' fails; or corrupt jfs superblocks when mounting the imported volumes on the host, requiring an fsck or re-running the snapshot.

We shut down the database prior to recreating the FCs to quiesce i/o; but I was wondering if additionally we should be (a) running a synch call prior to recreating the FCs to flush the disk buffers and (b) running a full fsck prior to mounting the volumes as a matter of course.

Does anybody have any ideas/experience here ?

Sorry for the long post - but I'm an HP guy historically and I'm just getting to grips with this Flashcopy stuff; and unfortunately IBM won't ratify our 'home grown' scripts under our support agreement.

I already posted this in the Storage Forum last week and got no response so I thought I'd try it here too. Thanks.

Originally posted by: bhead

Ive written Korn shell scripts for flashcopy for ESScli (ES800) and DScli (DS8000) so Im not 100% sure what I can tell you will be of any help. Your also backlevel on AIX. Assuming SMcli is pretty much doing the same thing as well as your level of AIX I can point out some differences in how Im doing this.

First thing I noticed is your clearing out the pvid which isnt necessary because the recreatevg command does that for you:

grep "pv=clear" /usr/sbin/recreatevg
chdev -l $pv -a pv=clear 1>/dev/null

Or at least it does as far back as 5.2 ML06. You can do the same grep command on the recreatevg script on your system to verify.

Second thing is we do not remove the hdisk or the vpath drives from the ODM. Once you dump your data off to tape you should be able to just varyoffvg and exportvg. As long as the drives dont belong to an active volume group the drives arent in use. I run a flashcopy to drives that are in the ODM, but I make sure the FC filesystems are unmounted, FC volume group has been varied off and exported.

In my script I use a parm file with a list of drives to use to create the flashcopy volume group. When we need to add drives I just add a primary to the source volume group, then add the flashcopy drive name to the parm file. Build a new incremental flashcopy task on the ESS box for the new pair of drives and rebuild the consistency group flash copy task adding in the new drive pair task.

When it comes time to create the new volume group I read in the parm file to get the disk drive names. My parm file has one line for each drive so I have to create a temp file with all the drives on one line, cat the temp file into a variable and then use that variable for the list of drives in the recreatevg command.

I would highly recommend running a fsck y command against each of the newly created filesystems. After that mount up the new filesystems and kick off your backup. Once the backup completes just unmount filesystems. Add a varyoffvg and exportvg at the beginning of your flashcopy backup script. That way if you need to restore prior to the next backup all you have to do is mount up your flash copy filesystems and copy disk to disk without having to recreate the volume group again.

The only time Ive ran into problems with this is when weve upgraded AIX and installed a newer version of SDD. Sometimes the drives come back in with different vpath/hdisk names. So prior to an upgrade I do a lsvpcfg command and save that off to a file. Do it again after the upgrade/reconfigure and compare the drive serial numbers to make sure the drive names havent changed. If they have then youll just need to update your parm file.

If you never remove your hdisk or vpath drives from the ODM then theyll always be the same.

Again, I only know that this has worked in my environment, hardware and software are different so I cant be sure this will help and could possible cause you a problem. Might want to verify with IBM first.

Originally posted by: jdsdavis

Fosteria,
I saw your post on unix.com too, which had more details.

Basically, if you freeze your filesystems, you shouldn't have filesystem issues. I'm pasting in text for both JFS2 and DS4k snapshotting. I haven't tested for syntax errors, and some of the commands are adapted from yours.

[b]Example 1: AIX JFS2 snapshotting
################################[/b]
code
### Prepare for snapshotting
mkdir /snapshots
mkdir /snapshots/u01
mkdir /snapshots/u02
mkdir /snapshots/u03
mkdir /snapshots/u04
mklv -y snapu01 datavg 4
mklv -y snapu02 datavg 4
mklv -y snapu03 datavg 4
mklv -y snapu04 datavg 4
### quiesce oracle
echo "alter system set resource_manager_plan='SYSPLAN' scope=both;
connect / as sysdba
alter system quiesce restricted;
select active_state from v$instance;" | sqlplus
### Quiesce the filesystems
sync;sync;sync;sleep 5
chfs -a freeze=30 /u01
chfs -a freeze=29 /u02
chfs -a freeze=28 /u03
chfs -a freeze=27 /u04
### Snapshot the filesystems
snapshot -o snapfrom=/u01 /dev/snapu01
snapshot -o snapfrom=/u02 /dev/snapu02
snapshot -o snapfrom=/u03 /dev/snapu03
snapshot -o snapfrom=/u04 /dev/snapu04
### Resume I/O
chfs -a freeze=0 /u01
chfs -a freeze=0 /u02
chfs -a freeze=0 /u03
chfs -a freeze=0 /u04
echo "connect / as sysdba
alter system unquiesce;
select active_state from v$instance;" | sqlplus
### Mount the snapshots
mount -v jfs2 -o snapshot /dev/snapu01 /snapshots/u01
mount -v jfs2 -o snapshot /dev/snapu02 /snapshots/u02
mount -v jfs2 -o snapshot /dev/snapu03 /snapshots/u03
mount -v jfs2 -o snapshot /dev/snapu04 /snapshots/u04
### Back up the snapshots
dsmc s /snapshots/
### Release the snapshots
umount /snapshots/u01
umount /snapshots/u02
umount /snapshots/u03
umount /snapshots/u04
snapshot -d /dev/snapu01
snapshot -d /dev/snapu02
snapshot -d /dev/snapu03
snapshot -d /dev/snapu04
rmdir /snapshots/*
rmdir /snapshots
[/code]

[b]Example 2: DS4000 flash copy
############################[/b]
code ### NOTE: Much of this could be SSHed over from the TSM server
### Preparation
mkdir /snapshots
mkdir /snapshots/u01
mkdir /snapshots/u02
mkdir /snapshots/u03
mkdir /snapshots/u04

### quiesce oracle
echo "alter system set resource_manager_plan='SYSPLAN' scope=both;
connect / as sysdba
alter system quiesce restricted;
select active_state from v$instance;" | sqlplus

### Quiesce the filesystems
sync;sync;sync;sleep 5
chfs -a freeze=30 /u01
chfs -a freeze=29 /u02
chfs -a freeze=28 /u03
chfs -a freeze=27 /u04

### Snapshot the filesystems
SMcli -n "DS4700" -c 'recreateFlashcopy logicalDrive ;'

### Resume I/O
chfs -a freeze=0 /u01
chfs -a freeze=0 /u02
chfs -a freeze=0 /u03
chfs -a freeze=0 /u04
echo "connect / as sysdba
alter system unquiesce;
select active_state from v$instance;" | sqlplus

### Mount the snapshots
### NOTE: This portion could be done on the TSM server if zoned.
### The benefit would be no processing overhead on DBPROD.
SMcli -n "DS4700" -c 'create mapping logicalDrive="DBPROD_1"
logicalUnitNumber=30 host="DBPROD";'
hot_add
recreatevg -y fcdatavg hdisk2
mount -v jfs2 -oro /dev/??? /snapshots/u01
mount -v jfs2 -oro /dev/??? /snapshots/u02
mount -v jfs2 -oro /dev/??? /snapshots/u03
mount -v jfs2 -oro /dev/??? /snapshots/u04

### Back up the snapshots
dsmc s /snapshots/
### Release the snapshots
umount /snapshots/u01
umount /snapshots/u02
umount /snapshots/u03
umount /snapshots/u04
varyoffvg fcdatavg
export fcdatavg
rmdev -l hdisk2
Smcli -n "DS4700" -c 'delete mapping logicalDrive="DBPROD_1"
host ="DBPROD";'
SMcli -n "DS4700" -c 'disableFlashCopy logicalDrive
;'
rmdir /snapshots/*
rmdir /snapshots
[/code]