IBM Guardium

IBM Guardium

Join this online user group to communicate across Security product users and IBM experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Back to discussions
Expand all | Collapse all

Universal Connector MariaDB on AWS RDS

  • 1.  Universal Connector MariaDB on AWS RDS

    Posted Thu May 08, 2025 11:29 AM

    @Wendy Zemba Thanks for the reply Wendy. I don't see anything under s-tap status. no universal connector. but i have checked it has been enabled and configuration was saved successfully have couple of questions

    1.I have not setup any policy for UC , do i have setup a policy first before logs can be pulled from cloudwatch?

    2.What will be the troubleshooting steps? where should i start from? 

    I read somewhere that Universal connector logs cannot be retrieved through normal sessions reports. since logs pulled through universal connector are stored in a table at the backend and it has to be pulled to collector through reports. But i am not sure if its correct or not. I would appreciate any help. 

    Thanks 



    ------------------------------
    sara rehan
    ------------------------------


  • 2.  RE: Universal Connector MariaDB on AWS RDS

    Posted Thu May 08, 2025 01:01 PM
    Edited by Wendy Zemba Fri May 09, 2025 08:37 AM

    Hi @sara rehan,

    Just because it saved ok, doesn't mean it's correct or working, the evaluation at save simply looks for issues with the construct of the code, it doesn't test for successful communication or anything. Since it doesn't show up in the STAP status table that tells me logs aren't getting to the collector.

    As far as debugging, there's a troubleshooting tool in 11.5+: Troubleshooting tool. 

    If that doesn't work, let me know and I'll share some CLI commands and where to look at logs in the fileserver.



    ------------------------------
    Wendy Zemba
    Sr. Consultant, Data Protection
    Converge Technology Solutions
    wendy.zemba@convergetp.com

    Need help with your Guardium deployment? Contact me directly to discuss engagement opportunities. Currently serving North America.
    ------------------------------

    Original Message


  • 3.  RE: Universal Connector MariaDB on AWS RDS

    Posted Fri May 09, 2025 08:35 AM
    Edited by Wendy Zemba Fri May 09, 2025 08:37 AM

    Hi @sara rehan,

    Here's how to debug the Universal Connector and view the logs.

    1. Login to the Collector CLI
    2. grdapi set_universal_connector_log_level uc_debug_level=debug
    3. Log into the Collector UI, navigate to Universal Connector, open the UC configuration and re-save it.
    4. From the Collector CLI, start fileserver.
    5. From fileserver, navigate to: 

      logs >  opt-ibm-guardium-log/ > uc_container_log/ > uc-logstash.log

    6. From there, there's a lot of different directions it could go based on what you find in the log.
    7. Remove debug (note, restarting the UC will set it back to the default, which is info, but I like to keep set it to error): 

      grdapi set_universal_connector_log_level uc_debug_level=error

    Regarding the policy. For testing purposes it is helpful to have a policy that contains, at minimum, a Log Masked Details rule so you can validate that the SQLs were captured, but you will get a session record by default.



    ------------------------------
    Wendy Zemba
    Sr. Consultant, Data Protection
    Converge Technology Solutions
    wendy.zemba@convergetp.com

    Need help with your Guardium deployment? Contact me directly to discuss engagement opportunities. Currently serving North America.
    ------------------------------

    Original Message


  • 4.  RE: Universal Connector MariaDB on AWS RDS

    Posted Fri May 09, 2025 09:05 AM
    Hi Wendy , 

    Thanks for sharing the steps 

    I ran the troubleshooting and it basically checks in two logs uc-logs task.log and logstash-plain.log and in the first one it says internal error: invalid syntax and the second one has no errors . But it doesn't tell me which line has syntax error . 


    Sent from my iPhoneX



    Original Message


  • 5.  RE: Universal Connector MariaDB on AWS RDS

    Posted Thu May 22, 2025 02:53 PM
    Hi Wendy, 

    I am still having issues connecting UC with aws rds. Do we need any bidirectional port opening to use UC with aws rds?  We have run the allow domain command as well . But the aws team telling us that their access key and secret access key hasn't been used not even once so something is stopping UC to connect to aws rds . What could it be ? 

    Thank 

    Sara 
    Sent from my iPhoneX

    On May 9, 2025, at 2:05 PM, sara rashid <sarah_sphinx@hotmail.com> wrote:

     Hi Wendy , 

    Thanks for sharing the steps 

    I ran the troubleshooting and it basically checks in two logs uc-logs task.log and logstash-plain.log and in the first one it says internal error: invalid syntax and the second one has no errors . But it doesn't tell me which line has syntax error . 


    Sent from my iPhoneX



    Original Message


  • 6.  RE: Universal Connector MariaDB on AWS RDS

    Posted Thu May 22, 2025 04:53 PM

    @sara rehan,

    Sorry to hear you are still having issues. 

    Guardium UC is a "pull", so one way from Guardium to the CloudWatch group is required.

      Is your Collector and the CloudWatch group in different aws accounts? If so, did you follow the configuration for cross account IAM Role? https://github.com/IBM/universal-connectors/blob/main/input-plugin/logstash-input-cloudwatch-logs/SettingsForRoleArn.md

      I've also run into a situation where the VPC wasn't opened between the two accounts.



      ------------------------------
      Wendy Zemba
      Sr. Consultant, Data Protection
      Converge Technology Solutions
      wendy.zemba@convergetp.com

      Need help with your Guardium deployment? Contact me directly to discuss engagement opportunities. Currently serving North America.
      ------------------------------

      Original Message


    1. 7.  RE: Universal Connector MariaDB on AWS RDS

      Posted Thu May 22, 2025 05:33 PM

      is this any better? 

      Collector is not on cloud, its virtual appliance. this is why we are using access and secret access key. Do we have to allow outbound port 443 from collector?



      ------------------------------
      sara rehan
      ------------------------------

      Original Message


    2. 8.  RE: Universal Connector MariaDB on AWS RDS

      Posted Fri May 23, 2025 08:18 AM

      Hi @sara rehan,

      Many configurations use access and secret access key even for non-hybrid infrastructure, but I think you're in the right place, troubleshooting networking issues at this point. Yes, in the case of Collector on-premise to CloudWatch, you need communication opened Collector to CloudWatch over port 443.



      ------------------------------
      Wendy Zemba
      Sr. Consultant, Data Protection
      Converge Technology Solutions
      wendy.zemba@convergetp.com

      Need help with your Guardium deployment? Contact me directly to discuss engagement opportunities. Currently serving North America.
      ------------------------------

      Original Message


    3. 9.  RE: Universal Connector MariaDB on AWS RDS

      Posted 15 days ago
      Hi Wendy,

      Yes so finally we have port 443 opened bi directional . From On prem collector to aws . We had to setup a vpc as well. But I still couldn't see any sessions coming through and aws rds team has confirmed the IAM user created has never been accessed. I am using Logstash Maria db configurations from GitHub to pull logs from cloud watch. Below is the script I'm using 

      cloudwatch_logs { 	log_group => [ "/aws/rds/instance/test/postgresql"] 	start_position => "beginning" 	access_key_id => "<Enter the access key id>" 	secret_access_key => "<<Enter the secret access key id>>" 	region => "ap-south-1" #Default value: us-east-1 	interval => 60 	codec => null

      add_field => {"account_id" => "<ACCOUNT_ID>"}

      type => "aws_mysql"

      Aws rds using Mariadb plugin but I think logs that are being sent are not JSON logs. I m keeping filter as empty at the moment. But I have not setup any configuration for output. Do I have to do that as well or I can do that later once I start getting sessions?

      Thanks for your help.

      Regards 

      Sara 

      Sent from my iPhoneX



      Original Message


    4. 10.  RE: Universal Connector MariaDB on AWS RDS

      Posted 13 days ago
      Edited by Wendy Zemba 13 days ago

      Hi @sara rehan,

      You need to use the input and filter that matches your DBMS type, in your case MariaDB, correct? Looks like you're using MYSQL.

      https://github.com/IBM/universal-connectors/blob/main/filter-plugin/logstash-filter-mariadb-aws-guardium/README.md

      Below depicts what you would put in for a MariaDB filter. Note that you need to include the availability zone of your region, I added "a", but you need to confirm. You also need to change the parameters represented by the < >'s (highlighted yellow).

      cloudwatch_logs {
          log_group => ["/aws/rds/instance/test/postgresql"]
          region => "ap-south-1a"
          codec => plain
          sincedb_path => "NUL"
          access_key_id => "<ACCESS_KEY>"
          secret_access_key => "<SECRET_ACCESS_KEY>"
          type => "mariadb"
          event_filter => ''
          start_position => "end"
          add_field => {"account_id" => "<ACCOUNT_ID>"}

      I'm not 100% how it will function if you don't include the filter, but it's easy enough to drop in there. The only thing you would change in it, without direction from support, would be the type. The type parameter in the filter needs to match the type value you enter in to the above input section, in this example "mariadb". 



      ------------------------------
      Wendy Zemba
      Sr. Consultant, Data Protection
      Converge Technology Solutions
      wendy.zemba@convergetp.com

      Need help with your Guardium deployment? Contact me directly to discuss engagement opportunities. Currently serving North America.
      ------------------------------

      Original Message


    5. 11.  RE: Universal Connector MariaDB on AWS RDS

      Posted 10 days ago

      Hi,

      I have been using simple filter with type =>mysql . Since I don't have full access to CLI, I am limited to what I can check. Network team confirmed that they can see traffic flow form collector to vpc endpoint and back but somehow not pulling logs since access key in aws rds has not been accessed when I run the troubleshooting tool it doesn't give me much information. I'm just not sure why is it not accessing the logs in cloudwatch. 



      ------------------------------
      sara rehan
      ------------------------------

      Original Message


    6. 12.  RE: Universal Connector MariaDB on AWS RDS

      Posted 9 days ago

      @sara rehan,

      At this point, I recommend performing some hands-on troubleshooting. If you haven't already, consider opening a support case for further assistance.



      ------------------------------
      Wendy Zemba
      Sr. Consultant, Data Protection
      Converge Technology Solutions
      wendy.zemba@convergetp.com

      Need help with your Guardium deployment? Contact me directly to discuss engagement opportunities. Currently serving North America.
      ------------------------------

      Original Message


    Related Content