Content Management and Capture

 View Only
Expand all | Collapse all

Looking for HA fileshare solution for Content Engine on Windows ND cluster

  • 1.  Looking for HA fileshare solution for Content Engine on Windows ND cluster

    Posted Thu December 31, 2020 02:46 AM
    Hello everyone.

    we have been successfully deploying Content Engine with WAS ND clusters on AIX using GPFS as a shared highly available filesystem.

    Now we need to deploy CE with WAS ND on Windows 2019 and we are looking for the most efficient, easy to configure (no third party licensed products pls), shared-higly available filesystem for CE filestores.

    Our Windows Servers are VMs on VMware infrastructure.

    Any ideas from colleagues that use Windows servers?

    BR
    Christos

    ------------------------------
    Christos Chorattides
    Datatech
    ------------------------------


  • 2.  RE: Looking for HA fileshare solution for Content Engine on Windows ND cluster

    Posted Fri January 08, 2021 04:03 AM
    Hello,

    As far as I know, there is no GPFS-like feature on Windows Server.
    Generally what we do is to share a NAS on the same Windows drive for each Windows Server. NAS is shared via CIFS or NFS protocol which handled files and locks.
    Then on your Windows drive you can create folders for different filestores or set up new drives for each NAS share you want.

    Unlike GPFS this solution is not using SAN but NAS so it is less fast than GPFS. But for document management purpose it's generally sufficient.

    Hope it helps,

    ------------------------------
    Florian KIEBEL
    Practice Leader
    Amexio
    ------------------------------



  • 3.  RE: Looking for HA fileshare solution for Content Engine on Windows ND cluster

    Posted Fri January 08, 2021 11:08 AM
    Hi Florian,

    thanks for your input.

    I was actually looking for a highly available filesystem.

    Seems the only HA option for Windows is a dedicated File Server Cluster.

    BTW, I think IBM has a GPFS port for Windows but I don't know anything about licensing.

    Thanks!
    Christos

    ------------------------------
    Christos Chorattides
    Datatech
    ------------------------------



  • 4.  RE: Looking for HA fileshare solution for Content Engine on Windows ND cluster
    Best Answer

    Posted Fri January 08, 2021 11:51 AM
    Have you considered using an Advanced Storage Area with two file system storage devices?

    See https://www.ibm.com/support/knowledgecenter/SSNW2F_5.5.0/com.ibm.p8.planprepare.doc/p8ppi267.htm

    ------------------------------
    Mike
    ------------------------------



  • 5.  RE: Looking for HA fileshare solution for Content Engine on Windows ND cluster

    Posted Sun January 10, 2021 03:25 AM
    Edited by Christos Chorattides Sun January 10, 2021 10:25 AM
    What a great idea Michael! Thanks!.

    I have never used Advanced Storage Areas before. I knew they had something to do with Cloud Storage and Replication but I didn't think I could use with a WAS Cluster to have a highly available storage.

    I tested two areas-two devises and seems to wok fine! Have you used this in a production environment?

    I have a few questions, if you know. When I tested with the second Storage Area down, system was operating fine. After bringing up the second storage, it took some time to synchronize. What is the default interval in which the first storage area checks and synchronizes the second storage area after it comes up? Is it configurable?

    Also on "normal" Storage Areas we can add multiple Storage Areas to the same Storage Policy, creating a storage farm to improve throughput and implement size and file number constrains for each Storage Area. Can we implement something similar to Advanced Storage Areas?

    Can the Bulk Content Move Sweeps move content from "normal" Storage Areas to Advanced Storage Areas?

    BR

    ------------------------------
    Christos Chorattides
    Datatech
    ------------------------------



  • 6.  RE: Looking for HA fileshare solution for Content Engine on Windows ND cluster

    IBM Champion
    Posted Mon January 11, 2021 09:51 AM
    >Can the Bulk Content Move Sweeps move content from "normal" Storage Areas to Advanced Storage Areas?
    Sure, no problem. We did 240 million docs from IS CFS do advanced storage area...


    ------------------------------
    Gerold Krommer
    ------------------------------



  • 7.  RE: Looking for HA fileshare solution for Content Engine on Windows ND cluster

    Posted Mon January 11, 2021 10:50 AM
    Thanks Gerold!

    Glad to see some familial names from the IS era!

    BR

    ------------------------------
    Christos Chorattides
    Datatech
    ------------------------------



  • 8.  RE: Looking for HA fileshare solution for Content Engine on Windows ND cluster

    IBM Champion
    Posted Tue January 12, 2021 04:13 AM
    Yes, its been a while.

    We are about to remove the last traces of IS here with our customers. Last years have been a pain to operate it. The product has surfaced around 1986(!) on FDOS and has been on an iron lung for the recent years...

    Hear you soon,
    /Gerold

    ------------------------------
    Gerold Krommer
    ------------------------------



  • 9.  RE: Looking for HA fileshare solution for Content Engine on Windows ND cluster

    Posted Tue January 12, 2021 05:03 AM
    To answer the first question, asynchronous replication (to the second storage device) is operated by a queue sweep, which can be tuned/scheduled in various ways in common with all queue sweeps (refer to the documentation). However, in the case of a down connection a couple of additional factors apply:

    - in general within the advanced storage logic there is a delay between attempts to re-restablish a connection to a storage device which was unavailable which defaults to 5 minutes. This can be overridden through the JVM argument Advanced.ConnectivityRefreshSeconds.
    - advanced storage queue items which cannot be processed due to a down connection are subject to deferral for a period which defaults to two minutes beyond the connectivity refresh period. This can also be overridden by the JVM argument Advanced.Queue.Entry.Defer.Seconds (which would be set to an absolute time in seconds).

    The idea behind this is to avoid thrashing and unnecessary failure-retry of queue items, and we don't recommend changing either setting.

    Regarding the second question, Gerold already somewhat answered. But to be very clear, bulk move can move from any type of storage area to any other type.

    ------------------------------
    Mike
    ------------------------------



  • 10.  RE: Looking for HA fileshare solution for Content Engine on Windows ND cluster

    Posted Wed January 13, 2021 05:10 AM
    Thank you Mike,

    you missed my second question about Advanced Storage farms.

    Although I suppose we can create multiple ASAs and assign them to the same Storage Policy, I am not sure how to configure them to increase throughput at the primary storage (or location). What I need to achieve is to have three primary ASAs where only one copy of the content will be stored, the primary ASAs will be load balanced for better throughput, and the same should happen to the secondary storage where the content should be replicated, again one copy in three load balanced ASAs.

    BR

    ------------------------------
    Christos Chorattides
    Datatech
    ------------------------------



  • 11.  RE: Looking for HA fileshare solution for Content Engine on Windows ND cluster

    Posted Wed January 13, 2021 07:46 AM
    If I understand correctly what you're asking for, you can do this exactly as you described:
    - create three ASAs each with one primary relica and one secondary
    - create a storage policy which selects those three ASAs
    - define that storage policy as the default for your document classes.
    Incoming content will then be distributed across the three ASAs, exactly as you've evidently understood can be done with legacy file stores.

    (Storage policies, like bulk move sweeps, are agnostic as to the type of the selected storage areas, so you can even mix area types).

    ------------------------------
    Mike
    ------------------------------



  • 12.  RE: Looking for HA fileshare solution for Content Engine on Windows ND cluster

    Posted Thu January 14, 2021 03:37 AM
    Edited by Christos Chorattides Thu January 14, 2021 03:41 AM

    Original Message:
    Sent: Wed January 13, 2021 07:45 AM
    From: Michael Seaman
    Subject: Looking for HA fileshare solution for Content Engine on Windows ND cluster

    If I understand correctly what you're asking for, you can do this exactly as you described:
    - create three ASAs each with one primary relica and one secondary
    - create a storage policy which selects those three ASAs
    - define that storage policy as the default for your document classes.
    Incoming content will then be distributed across the three ASAs, exactly as you've evidently understood can be done with legacy file stores.

    (Storage policies, like bulk move sweeps, are agnostic as to the type of the selected storage areas, so you can even mix area types).

    ------------------------------
    Mike



  • 13.  RE: Looking for HA fileshare solution for Content Engine on Windows ND cluster

    Posted Thu January 14, 2021 03:38 AM
    Thanks Mike!

    Is there a Technote or paper that describes ASAs  in detail? Knowledge center has some ASA info under "Storage Content" and "IT administrator installation tasks" but information it's a bit scattered and high level.

    BR

    ------------------------------
    Christos Chorattides
    Datatech
    ------------------------------



  • 14.  RE: Looking for HA fileshare solution for Content Engine on Windows ND cluster

    Posted Fri January 15, 2021 09:41 AM
    As far as I am aware, the only published material is that in the knowledge centre.

    ------------------------------
    Mike
    ------------------------------