IBM Security Verify

 View Only
  • 1.  isam_cli/isva_cli -c reload all i isav-config container -- unexpected error

    Posted Wed October 27, 2021 10:20 AM
    Hi,
    We are running ISAV 10.0.2.0 in a Kubernetes bare-metal cluster.

    after adding and configuring the DSC to our (working) Cluster, we see the following in the container shell:

    [isam@verify-access-1-3-isvaconfig-c85f9f848-snhkd /]$ isam_cli -c reload all
    sh: /usr/sbin/ip: No such file or directory
    sh: /usr/sbin/ip: No such file or directory
    sh: /usr/sbin/ip: No such file or directory
    sh: /usr/sbin/iptables-restore: No such file or directory
    sh: /usr/sbin/ip6tables-restore: No such file or directory
    2021-10-27-16:03:35.809+02:00I----- 0x35E5102B WebSEAL-Mgmt-API FATAL pdz general ZProperties.cpp 742 0x7f1edfbc4940
    HPDPZ0043E An access function failed for configuration file /var/felb/etc. The access function was access() and return code was 2.
    Unexpected error

    Unfortunately we cannot be certain if this error occurred because of the DSC addition or not.
    After installing, the DSC (and hopefully configuring it correctly) pages/areas in the wrp that we protect with a TOTP access rule no longer works.
    The rest of the site works as before.

    Is this in any way related?

    If not I will open a new discussion regarding the DSC/TOTP problems we are seeing.

    Thanks in advance

    ------------------------------
    Anders Domeij
    CGI Sweden AB
    ------------------------------


  • 2.  RE: isam_cli/isva_cli -c reload all i isav-config container -- unexpected error

    Posted Wed October 27, 2021 04:28 PM

    Anders,

    The error which you are seeing is not related to your TOTP issue.  It looks like, based on the container name, that you are attempting to run the reload command from within the configuration container.  This command is not really designed to be run from within the configuration container as the configuration container should always be running with the latest configuration.

    So, you can ignore this error and will instead need to focus on the TOTP issue.

    I hope that this helps.





    Scott A. Exton
    Senior Software Engineer
    Chief Programmer - IBM Security Access Manager

    IBM Master Inventor



    "Anders Domeij via IBM Community" ---28/10/2021 12:19:48 am---Hi, We are running ISAV 10.0.2.0 in a Kubernetes bare-metal cluster.






  • 3.  RE: isam_cli/isva_cli -c reload all i isav-config container -- unexpected error

    Posted Thu October 28, 2021 03:32 AM
    Thanks for the reply.
    Now you have me confused though
    >This command is not really designed to be run from within the configuration container
    The 10.0.2 documentation says:
    "Note: The CLI commands are only available in the main verify-access image and are not available in the verify-access-runtime, verify-access-wrp images, and verify-access-dsc images."
    So what does the documentation refer to the "main" image?
    Or are we barking up the wrong tree?
    The above note is from https://www.ibm.com/docs/en/sva/10.0.2?topic=support-cli-in-docker-environment.
    Is the loading of the config file "automatic" or are the new (10.0.2.0) "reload runtime" och "reload policy" CLI commands what we should use in a clustered environment.
    Also the information text after publishing that alerts me to restart the runtime environments doesn't feel appropriate 'wording' in a clustered environment.
    Thanks


    ------------------------------
    Anders Domeij
    CGI Sweden AB
    ------------------------------



  • 4.  RE: isam_cli/isva_cli -c reload all i isav-config container -- unexpected error

    Posted Thu October 28, 2021 05:34 AM
    Hi Anders,

    In previous versions of Verify Access containers it was possible to reload/restart all containers by using "exec" and issuing the isam_cli reload command.  In 10.0.2.0 (using the lightweight containers) this isn't possible.  You must restart the worker containers to pick up new configuration.

    The easiest (but disruptive) way to do this is to simply delete the running pod for the component that needs to be restarted.  Kubernetes will detect the missing pod and start a new one:

    kubectl delete pod <pod to delete>

    In a production environment you would want to use built-in Kubernetes capabilities to perform a "rolling restart" of the deployment.  This means a new pod is started with new configuration before the old one is stopped.  One way to trigger a rolling restart is to change/add an arbitrary setting within the deployment configuration.  For example you could add a dummy environment variable.

    You can make a  simple edit to a deployment configuration using:

    kubectl edit deploy <deployment>

    This will open YAML in default editor.  When you save changes are applied.

    I don't think any of this explains the errors you are seeing attempting to run the reload command in your configuration container - although, as Scott says below, not sure that you ever really need to use that command there anyway.  I would guess something in your Kubernetes host system is locked down and causing the issue seen.  It's unknown whether it's responsible for the other issues you are seeing.  A support call is probably needed for the analysis to determine that.

    Perhaps the issues with TOTP are caused because you need to manually restart all components in your environment following configuration updates to enable DSC/TOTP configuration.

    Jon.

    ------------------------------
    Jon Harry
    Consulting IT Security Specialist
    IBM
    ------------------------------



  • 5.  RE: isam_cli/isva_cli -c reload all i isav-config container -- unexpected error

    Posted Wed October 27, 2021 04:37 PM

    Anders,

    I just had another quick look and something very strange is going on in your environment.  The reload command should work successfully in a configuration container - even if it doesn't really do anything useful.  In my environment the reload command works fine, even if I enable the DSC.  You might need to open a support ticket to get the support team to investigate the issue.

    Thanks.



    Scott A. Exton
    Senior Software Engineer
    Chief Programmer - IBM Security Access Manager

    IBM Master Inventor



    "Anders Domeij via IBM Community" ---28/10/2021 12:19:48 am---Hi, We are running ISAV 10.0.2.0 in a Kubernetes bare-metal cluster.






  • 6.  RE: isam_cli/isva_cli -c reload all i isav-config container -- unexpected error

    Posted Mon November 01, 2021 11:16 AM
    Hi Scott & Jon

    We took  a more radical approach :-)
    We uninstalled 10.0.3 and deleted the 10.0.3 snapshot from from the shared volume.
    Then we started (again) from scratch with a fresh installation of 10.0.3., but this time with DSC enabled.
    This gave us  back a working 10.0.3 version of what we had in 10.0.2 with 2 DSC containers in the not ready state.
    Now we are going to take incremental baby steps and restrain ourselves to making just one (and only one) change at a time.
    We'll start by trying to get an Ingress in front of the cluster (and abandon the Nodeport approach we used in 10.0.2) for load-balancing multiple instances of WRP(s) in the cluster.
    Once that is complete we'll start configuring the DSC for Session handling of multiple WRP instances.
    Rgds

    ------------------------------
    Anders Domeij
    CGI Sweden AB
    ------------------------------



  • 7.  RE: isam_cli/isva_cli -c reload all i isav-config container -- unexpected error

    Posted Mon November 01, 2021 01:02 PM
    Hi Anders,

    You mention v10.0.3.0 which is not released yet (except for as part of Early Access Program).  If you're seeing issues with that version then it could be related to a bug in the pre-release code as likely as an issue with your configuration or process.

    Jon.

    ------------------------------
    Jon Harry
    Consulting IT Security Specialist
    IBM
    ------------------------------



  • 8.  RE: isam_cli/isva_cli -c reload all i isav-config container -- unexpected error

    Posted Tue November 02, 2021 04:46 AM
    Darn !
    I was referring to the Helm versions 1.0.2 and 1.0.3 but got carried away when typing zeros.
    Sorry for the confusion.
    Rgds

    ------------------------------
    Anders Domeij
    CGI Sweden AB
    ------------------------------



  • 9.  RE: isam_cli/isva_cli -c reload all i isav-config container -- unexpected error

    Posted Tue November 02, 2021 06:33 AM
    Hi Anders,

    I think you're referring the the versions of the ISVA Helm Charts:
      v1.2.0 - Initial chart created for Verify Access using all-in-one image (stable)
      v1.2.1 - Updated chart for Verify Access using all-in-one image (stable)
      v1.3.0 - Chart created for Verify Access 10.0.2.0 using new "lightweight" containers (incubator)

    With regard to DSC in the v1.3.0 chart, worth noting that the lightweight DSC containers listen on port 9443 and 9444 (instead on 443 and 444).  You need to make sure these are the ports you specify when configuring the DSC (under cluster configuration).

    Actually, here's the full release notes from v1.3.0 chart:
    -----

    # What's new in verify-access v1.3.0 Chart

    The following enhancements have been made:

    * Support for function-specific containers

    * Support to set service name (which is hostname)

    * Added startupProbe for config and runtime containers

    * Allow service type to be set per WRP instance

    * Improvements to NOTES.txt

    # Documentation

    For detailed documentation instructions go to [https://www.ibm.com/support/knowledgecenter/en/SSPREK/welcome.html](https://www.ibm.com/support/knowledgecenter/en/SSPREK/welcome.html).

    # Required configuration changes for lightweight containers

    The new lightweight runtime container in v10.0.2.0 does not listen on port 443.  Instead it listens on port 9443.  You will need to update junctions and WRP configuration items that reference the runtime to set this new port.

    The new lightweight DSC containers in v10.0.2.0 do not listen on ports 443 and 444. Instead they listen on ports 9443 and 9444.  You will need to update you cluster configuration to reflect this change.

    The new lightweight reverse proxy container in v10.0.2.0 does not listen on port 443.  Instead it listens on port 9443.  You may need to update ingress or loadbalancer configurations to reflect this.

    ----

    Also, since changes were made in v1.2.1 too, here's the release notes from that version too:
    ---

    # What's new in verify-access v1.2.1 Chart

    The following enhancements have been made:

    * Ability to set timezone for containers

    * Ability to set port for NodePorts

    * Change to preferred antiAffinity to allow for in-place upgrade

    * Mark PVCs as "keep" so not deleted with release

    * Change to Reverse Proxy definition to allow per-instance settings

    # Values migration

    In previous versions the Reverse Proxy instances were defined with an array containing a list of instance names. e.g.

    ```

    - rp1

    - rp2

    ```

    In this version, the array must now contain a list of instances with attributes defined for each instance.  The `name` attribute is required.  Other optional attributes are `nodePort` and `replicas` e.g.
    ```

    - name: rp1

    - name: rp2

      replicas: 2

      nodePort: 30443
    ```

    ---


    Waiting with interest to hear how you get on with the ingress configuration.  Hopefully it will be simple enough to get working.


    Jon.

    ------------------------------
    Jon Harry
    Consulting IT Security Specialist
    IBM
    ------------------------------