IBM Verify

IBM Verify

Join this online user group to communicate across Security product users and IBM experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Expand all | Collapse all

Secret Server Stops Working once in a while

  • 1.  Secret Server Stops Working once in a while

    Posted Thu October 22, 2020 07:57 AM

    We are using Security Verify privilege V10.8, the server is newly deployed and no secrets and no users are currently using this solution. We are facing an issue where once in a week, Secret Server stops working as this server is a build phase and the server remains inactive so most of the time. User can login and view their secrets but unable to establish connections using Putty, RDP etc. Email notifications stops working .

    But when we restart IIS server using iisreset command, everything get backs to normal.

    First i thought, it might be an issue mentioned in the below Technote but all the configuration settings mentioned are already in place

    https://docs.thycotic.com/ss/10.9.0/troubleshooting/change-iis-worker-process

    Is there any guide which can provide some information about Tuning IIS Web Server



    #Support
    #SupportMigration
    #Verify


  • 2.  RE: Secret Server Stops Working once in a while

    Posted Thu October 22, 2020 07:55 PM

    Problems with IIS (resource exhaustion, app pool crashes, etc.) would show up as HTTP 400/500 error pages, so this sounds like a back-end connection issue to SQL Server. Search the SS.log(s) for "System.Data.SqlClient.SqlException (0x80131904)." If you see any, then that's the root cause.

    Unfortunately, if you are losing the connection, then there's nothing you can do other than run IISRESET. Secret Server doesn't always recover from these events for some reason. To make matters worse, the event doesn't show up in Secret Server's System Log, so you can't create a trigger to alert you when it happens. The only way to tell is if you open SS.log or the UI stops working. Both are sub-optimal choices to say the least.

    Therefore, the best way to fix is to meet with the DBAs and ask why the connection is dropping so often in the first place. Are they rebooting it every week and not telling you? If they are, then you're options are: 1) tell them to stop; 2) tell them to either run IISRESET themselves (or tell you so you can do it) when SQL is back up ; or 3) create a separate SQL cluster specifically for Secret Server that they also should not reboot every week. Another possibility is an intermittent network issue, so you should invite someone from the Network group to the meeting as well. Resolving the SQL issue will more than likely make your Secret Server problem go away.

    Finally, something else that may help is to enable enable "Multi-Subnet Failover" if you're using an Availability Group for the SQL Server cluster. It's in the Secret Server Database Configuration page (https://hostname/Setup/Database). That will reduce the delay in connecting to a new SQL cluster instance when Secret Server encounters a connection failure. This might help it stay sync'd to SQL and not require a reset. More info on the setting here: https://docs.microsoft.com/en-us/sql/sql-server/failover-clusters/windows/sql-server-multi-subnet-clustering-sql-server?view=sql-server-ver15#DNS



    #Support
    #SupportMigration
    #Verify