Open Source Development

Power Open Source Development

Explore the open source tools and capabilities for building and deploying modern applications on IBM Power platforms including AIX, IBM i, and Linux.


#Power


#Power

 View Only

  • 1.  rsyslog 8.19.50.1 resend all logs to rsyslog master after hardware upgrade

    Posted Mon March 21, 2022 06:55 AM
    ​Hi team,

    We have some migration task monthly, e.g. Upgrade AIX OS box from Power8 to Power9, and also upgraded the OS level.
    The datavg is assigned by SAN Storage, so the log will be consistent with the old system.

    However, the rsyslog process will resend all the existing logs after the migration is completed, whether it has been forward to the rsyslog master or not before.
    so if we take the migrations at the same time for several systems, will hit the performance issue for the rsyslog master.

    that's why we try to relocate imfile-stat files to avoid the duplicated logs/message send again. but it does not work. Please advice.

    Thank you so much for your help.

    --------------------
    Michael Lam
    --------------------

    ------------------------------
    Michael Lam
    ------------------------------

    #AIXOpenSource


  • 2.  RE: rsyslog 8.19.50.1 resend all logs to rsyslog master after hardware upgrade

    Posted Fri March 25, 2022 07:13 AM
    Hi Michael,

    Can you elaborate a bit about the process of migration? Are you performing LPM? Or is it something else? Does it involve rebooting the lpar?

    Also, can you share your config file? Is it configured to send the messages to a remote machine?

    Have you tried stopping rsyslogd daemon before the migration and start it again after migration?

    Regards,
    Srinivas.

    ------------------------------
    SRINIVAS GUNDURAO
    ------------------------------

    Original Message


  • 3.  RE: rsyslog 8.19.50.1 resend all logs to rsyslog master after hardware upgrade

    Posted Mon March 28, 2022 09:42 PM
      |   view attached
    Hi Srinivas,

    Q: Can you elaborate a bit about the process of migration? Are you performing LPM? Or is it something else? Does it involve rebooting the lpar?
    A:
    Not LPM, we used two different physical LPARs. We restored the mksysb from Power8 to Power9 for system VG. For datavg, we used SAN replication and imported the replicated SAN copy(datavg) in Power9 machine.
    While we started up the machine after system and datavg migration in Power9 hosts, it resend all the messages to rsyslog master server but not the delta.

    Q: Also, can you share your config file?
    Yes, please find the attachment.
    Q: Is it configured to send the messages to a remote machine?
    Yes.
    # Redirect message to rsyslog_master1 and 2 port 45927 for DEV
    local6.* @@rsyslog_master1:45927;withfilename
    local6.* @@rsyslog_master2:45927;withfilename

    Q: Have you tried stopping rsyslogd daemon before the migration and start it again after migration?
    Yes, we did.

    Thank you very much for your help.


    ------------------------------
    Michael Lam
    ------------------------------

    Attachment(s)

    txt
    rsyslog case 20220329.txt   2 KB 1 version
    Original Message


  • 4.  RE: rsyslog 8.19.50.1 resend all logs to rsyslog master after hardware upgrade

    Posted Fri April 08, 2022 05:29 AM
    Hi Michael,

    I suspect this has got to do with stale state file(https://www.rsyslog.com/doc/v8-stable/configuration/modules/imfile.html#state-files). Not sure at what stage you are stopping rsyslog, but I think you should stop it before you take the mksysb image on power 8 machine. This way the state file should have correct offset till which the messages have been sent to the rsyslog master. After you start the power 9 machine, rsyslog should start sending from this offset now, without duplicating the messages.

    Also check to see if setting the following parameters would help:

    https://www.rsyslog.com/doc/v8-stable/configuration/modules/imfile.html#persiststateaftersubmission
    https://www.rsyslog.com/doc/v8-stable/configuration/modules/imfile.html#persiststateinterval

    Regards,
    Srinivas.

    ------------------------------
    SRINIVAS GUNDURAO
    ------------------------------

    Original Message


Related Content