Originally posted by: SystemAdmin

Hi

I have created a disk heartbeat volumegroup. However, when I try to vary it on in concurrent mode, I receive the following error:

1. varyonvg -c hbeatvg

srcsrqt failed errno : SRC_NSVR
Subsystem gsclvmd is not active
tellclvmd: request failed rc = -9036 SRC_NSVR
0516-1334 varyonvg: The command /usr/sbin/tellclvmd returned an error.

I tried starting gsclvmd, but nothing happens

I am using HACMP 5.3 on AIX 5.3 TL5. Incidentally, bos.clvm is installed (only thing that i have found on google pointing to a solution, which wasn't so in my case)

Any help greatly appreciated
#AIX-Forum

Originally posted by: alethad

Did you create this VG as an "Enhanced Capable" or concurrent VG? You can't vary it on as concurrent if you didn't.

Someone else asked a similar question about the gsclvmd the other day. You might take a look at it to see if it pertains to your issue.
If you look to see what cluster services are running do you see gsclvmd in the list? If not try to start it thru the hacmp smit menus.

I guess you have your reasons but I was wondering why you were creating you heartbeat disks in their own VG? It isn't normally necessary to do that.
#AIX-Forum

Originally posted by: SystemAdmin

You generally want to have a small disk as your heartbeat disk in a seperate VG in the event that the Resource Group disk you choose to heartbeat though is deleted
#AIX-Forum

Originally posted by: alethad

You don't need to do that if you have a redundant heartbeat disk in another VG & on another SAN/loop than the other heartbeat disk.
A heartbeat is only used as a route for the other nodes to check each other anyway. Hence my curiousity as to why have it set aside in its own VG since the heartbeat disk doesn't need disk space. It does everything in a non-data portion of the disk.
#AIX-Forum

Originally posted by: tony.evans

There are performance considerations to using LUNs for both data and heartbeating.

From

http://www-03.ibm.com/systems/p/advantages/ha/faq/tech.html#11

Question: Can I use an existing Enhanced Concurrent Mode volume group for disk heartbeat? Or do I need to define a new one?

Answer: To achieve the highest levels of availability under the widest range of failure scenarios, the best practice would be to configure one disk heartbeat connection per physical disk enclosure (or LUN).

The heartbeat operation itself involves reading and writing messages from a non-data area of the shared disk. Although the space used for heartbeat messages does not decrease the space available for the application (it is in the reserved area of the disk) there is some overhead when the disk seeks back and forth between the reserved area and the application data area.

If you configure the disk heartbeat path using the same disk and vg as is used by the application, the best practice is to select a disk which does not have frequently accessed or performance critical application data: although the disk heartbeat overhead is small (2-4 seeks/sec), it could potentially impact application performance or, conversely, excess application access could cause the disk hb connection to appear to go up and down.

Ultimately the decision of which disk and volume group to use for heartbeat depends on what makes sense for your shared disk environment and management procedures. For example, using a separate vg just for heartbeat isolates the heartbeat from the application data, but adds another volume group that has to be maintained (during upgrades, changes, etc) and consumes another LUN.

If you decide on a separate vg for heartbeat, it does not need to be included in an HACMP resource group, however, the CSPOC utilities use a resource group node list as the set of nodes to perform operations: including the vg in a resource group with just the (sub)set of nodes connected to the disk will let you take advantage of the CSPOC functions. You can also define and use a disk which is not part of any volume group, though such a setup would have to be manually configured and maintained.
#AIX-Forum

Originally posted by: SystemAdmin

Subsystem gsclvmd is not active....

http://www.ibm.com/developerworks/forums/thread.jspa?threadID=192811
#AIX-Forum

Originally posted by: SystemAdmin

Unfortunately, it won't start any cluster services because of this issue with the disk, therefore, I can't start any services through HACMP at all.
#AIX-Forum

Originally posted by: alethad

Have you checked your cluster log files for any further or extended messages? anything in the hacmp.out?

You didn't answer my original question. Did you create this VG as an "Enhanced Concurrent" VG?

There should be no reason why the other cluster services shouldn't start. They do not depend on the gsclvmd daemon in order to start. You should at least have the clstrmgrES running. If not then check your cluster logs there's got to be something else wrong.

You may just need to call support.
#AIX-Forum

Originally posted by: SystemAdmin

Hi

Indeed, the vg is created as Enhanced Concurrent capable and the cluster manager daemon is running on both nodes. Fair enough, I'll raise a call

Thanks for your suggestions anyhow
#AIX-Forum

Originally posted by: tony.evans

Can I ask a different question, why are you trying to vary it on?
#AIX-Forum

Originally posted by: SystemAdmin

Hello again

I've narrowed it down to the following. When I remove the heartbeat disk as a communications device, restart cluster services, I can varyonvg my disk and sync the two nodes (cluster services work fine). However, when I add it back in as a communication device, this seems to screw up the cluster services and complains that the devices are not in enhanced concurrent mode, even though lspv shows the vg to be in concurrent mode.

The reason that I am trying to vary it on in concurrent mode is for the above, namely, that I can't get the heartbeat device to work when I "Verify and Synchronize" for the reason that the VG is not in Enhanced Concurrent mode, even though it is?!?!?
#AIX-Forum

Originally posted by: tony.evans

The volume group doesn't need to be varied on for the LUN to be used to heartbeat, if the LUN is only used for heartbeating.

So I'm not sure why it doesn't sync, but it's not because it's not varied on. You can use lssrc -ls topsvcs to check if heartbeats are working over a device.

(I included some output from one of my clusters, but broke the formatting, you'll have to trust me).

Message was edited by: tony.evans
#AIX-Forum

Originally posted by: IBMFORUMS

according to HACMP doc, the diskhb should be attached to a concurrent volume group and it should be varied off in both node.
did you test the diskhb by:

dhb_read -p /dev/hdiskX -r // for read on one node
dhb_read -p /dev/hdiskX -t // for transmit from the other node
#AIX-Forum

Originally posted by: SystemAdmin

I had the same error on one of my clusters today, too. SAN disks can be online as ECM vg without a diskhb while dhb_read test works but clver fails with ERROR if diskhb devices are configured. It turned out to be a problem with the HACMP 5.3 fileset levels which where at 5.3.0.0 when symptoms where as mdraja described them. Problem was resolved when cluster was updated to 5.3.0.5 i.e. a fileset level that includes IY75002.
#AIX-Forum