Informix

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

Hi Mark,

The "B" in the 3rd position means that the logical log has been backed up, which is good!

The log backup will be triggered by the script referenced in your onconfig file with the parameter ALARMPROGRAM.  It is called each time a logical log is filled or logical log is switched.  The other possibility is that LTAPEDEV is set to /dev/null (definitely not recommended) in which case the logical log will not be backed up, yet the flag will still show that it has.

The logical log will switch when full, but there are other operations that will trigger a log switch and may be part of your restart process or a manual switch (onmode -l) and may be part of your ontape backup process also, which is why you see some logical logs used but less than 100% used.  If you look at your Informix log file (MSGPATH), you will see a record there of each time a logical log is backed up and you may be able to see from that some event around when the particular log backup was performed.

03/30/23 08:30:49  Logical Log 5375644 Complete, timestamp: 0xd1531780.

03/30/23 08:30:49  Logical Log 5375644 - Backup Started

03/30/23 08:30:50  Logical Log 5375644 - Backup Completed

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

HI Mike (and Art), thanks for the replies and information - really appreciate that.  Checking Alarmprogram looks like there is an entry BACKUPLOGS=Y as well as BACKUP_CMD="ontape -a -d". So the answer would be continuous backup is set and logs are being backed once used then, aye?  With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?

# PUBLIC SECTION : CONFIGURATION VARIABLES
#                                         
# ########################################

BACKUPLOGS=Y

Then at night we also run the ontape -s L 0 to create a Level 0 file (which we use for restores to test) as well as then a full backup of the logs again.  But this is not essential given we're running continuous log backups anyway?

many thanks!

Mark

ps. Can confirm we don't have LTAPEDEV set to /dev/null either. 

Hi Mark,

The "B" in the 3rd position means that the logical log has been backed up, which is good!

The log backup will be triggered by the script referenced in your onconfig file with the parameter ALARMPROGRAM.  It is called each time a logical log is filled or logical log is switched.  The other possibility is that LTAPEDEV is set to /dev/null (definitely not recommended) in which case the logical log will not be backed up, yet the flag will still show that it has.

The logical log will switch when full, but there are other operations that will trigger a log switch and may be part of your restart process or a manual switch (onmode -l) and may be part of your ontape backup process also, which is why you see some logical logs used but less than 100% used.  If you look at your Informix log file (MSGPATH), you will see a record there of each time a logical log is backed up and you may be able to see from that some event around when the particular log backup was performed.

03/30/23 08:30:49  Logical Log 5375644 Complete, timestamp: 0xd1531780.

03/30/23 08:30:49  Logical Log 5375644 - Backup Started

03/30/23 08:30:50  Logical Log 5375644 - Backup Completed

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

I see that you are using ontape to do the system archiving. Just make sure that the ALARMPROGRAM was changed to use ontape as well to back up the logs (the default in there is to use onbar). There should be a line:

BACKUP_CMD="ontape -a"

not 

BACKUP_CMD="onbar -b -l"

You cannot mix ontape and onbar backups, so if the server is archived using ontape, then the logs need to use ontape as well. Simlarly for using onbar. You have to be consistent.

Art

HI Mike (and Art), thanks for the replies and information - really appreciate that.  Checking Alarmprogram looks like there is an entry BACKUPLOGS=Y as well as BACKUP_CMD="ontape -a -d". So the answer would be continuous backup is set and logs are being backed once used then, aye?  With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?

# PUBLIC SECTION : CONFIGURATION VARIABLES
#                                         
# ########################################

BACKUPLOGS=Y

Then at night we also run the ontape -s L 0 to create a Level 0 file (which we use for restores to test) as well as then a full backup of the logs again.  But this is not essential given we're running continuous log backups anyway?

many thanks!

Mark

ps. Can confirm we don't have LTAPEDEV set to /dev/null either. 

Hi Mark,

The "B" in the 3rd position means that the logical log has been backed up, which is good!

The log backup will be triggered by the script referenced in your onconfig file with the parameter ALARMPROGRAM.  It is called each time a logical log is filled or logical log is switched.  The other possibility is that LTAPEDEV is set to /dev/null (definitely not recommended) in which case the logical log will not be backed up, yet the flag will still show that it has.

The logical log will switch when full, but there are other operations that will trigger a log switch and may be part of your restart process or a manual switch (onmode -l) and may be part of your ontape backup process also, which is why you see some logical logs used but less than 100% used.  If you look at your Informix log file (MSGPATH), you will see a record there of each time a logical log is backed up and you may be able to see from that some event around when the particular log backup was performed.

03/30/23 08:30:49  Logical Log 5375644 Complete, timestamp: 0xd1531780.

03/30/23 08:30:49  Logical Log 5375644 - Backup Started

03/30/23 08:30:50  Logical Log 5375644 - Backup Completed

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

HI Mike (and Art), thanks for the replies and information - really appreciate that.  Checking Alarmprogram looks like there is an entry BACKUPLOGS=Y as well as BACKUP_CMD="ontape -a -d". So the answer would be continuous backup is set and logs are being backed once used then, aye?  With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?

# PUBLIC SECTION : CONFIGURATION VARIABLES
#                                         
# ########################################

BACKUPLOGS=Y

Then at night we also run the ontape -s L 0 to create a Level 0 file (which we use for restores to test) as well as then a full backup of the logs again.  But this is not essential given we're running continuous log backups anyway?

many thanks!

Mark

ps. Can confirm we don't have LTAPEDEV set to /dev/null either. 

Hi Mark,

The "B" in the 3rd position means that the logical log has been backed up, which is good!

The log backup will be triggered by the script referenced in your onconfig file with the parameter ALARMPROGRAM.  It is called each time a logical log is filled or logical log is switched.  The other possibility is that LTAPEDEV is set to /dev/null (definitely not recommended) in which case the logical log will not be backed up, yet the flag will still show that it has.

The logical log will switch when full, but there are other operations that will trigger a log switch and may be part of your restart process or a manual switch (onmode -l) and may be part of your ontape backup process also, which is why you see some logical logs used but less than 100% used.  If you look at your Informix log file (MSGPATH), you will see a record there of each time a logical log is backed up and you may be able to see from that some event around when the particular log backup was performed.

03/30/23 08:30:49  Logical Log 5375644 Complete, timestamp: 0xd1531780.

03/30/23 08:30:49  Logical Log 5375644 - Backup Started

03/30/23 08:30:50  Logical Log 5375644 - Backup Completed

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

alarmprogram.sh has been set up to back up the logical logs as expected - this is good.  Every time Informix switches to a new log, then alarmprogram will back up any logs not yet backed up.  It's not quite the same as continuous log backups, which is its own thing, and what you have here is a very common configuration.

As long as the backup continue to be successful, then yes, they will happily wrap and you won't get stuck in the "blocked" situation because the next log is used and not backed up.

But, if the device or filesystem where you are backing up logs fills, or there's another error, then the logs will continue to fill, and you will eventually run into a problem, so monitoring that you don't have many logs in not backed up status ("U--") is a good idea.

You probably don't need to back up all your logs again, but not sure how you are doing that anyway.

HI Mike (and Art), thanks for the replies and information - really appreciate that.  Checking Alarmprogram looks like there is an entry BACKUPLOGS=Y as well as BACKUP_CMD="ontape -a -d". So the answer would be continuous backup is set and logs are being backed once used then, aye?  With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?

# PUBLIC SECTION : CONFIGURATION VARIABLES
#                                         
# ########################################

BACKUPLOGS=Y

Then at night we also run the ontape -s L 0 to create a Level 0 file (which we use for restores to test) as well as then a full backup of the logs again.  But this is not essential given we're running continuous log backups anyway?

many thanks!

Mark

ps. Can confirm we don't have LTAPEDEV set to /dev/null either. 

Hi Mark,

The "B" in the 3rd position means that the logical log has been backed up, which is good!

The log backup will be triggered by the script referenced in your onconfig file with the parameter ALARMPROGRAM.  It is called each time a logical log is filled or logical log is switched.  The other possibility is that LTAPEDEV is set to /dev/null (definitely not recommended) in which case the logical log will not be backed up, yet the flag will still show that it has.

The logical log will switch when full, but there are other operations that will trigger a log switch and may be part of your restart process or a manual switch (onmode -l) and may be part of your ontape backup process also, which is why you see some logical logs used but less than 100% used.  If you look at your Informix log file (MSGPATH), you will see a record there of each time a logical log is backed up and you may be able to see from that some event around when the particular log backup was performed.

03/30/23 08:30:49  Logical Log 5375644 Complete, timestamp: 0xd1531780.

03/30/23 08:30:49  Logical Log 5375644 - Backup Started

03/30/23 08:30:50  Logical Log 5375644 - Backup Completed

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

HI Mike, thanks for the further reply.  Ahh, good points about the disk space where the log backups reside.  Will check further.

We presently run a cron job that kicks off a L0 night backup script which takes about an hour and executes:

onmode -l
ontape -s -L 0

This process appears to clear out the disk space once complete. 

Cheers
mark

alarmprogram.sh has been set up to back up the logical logs as expected - this is good.  Every time Informix switches to a new log, then alarmprogram will back up any logs not yet backed up.  It's not quite the same as continuous log backups, which is its own thing, and what you have here is a very common configuration.

As long as the backup continue to be successful, then yes, they will happily wrap and you won't get stuck in the "blocked" situation because the next log is used and not backed up.

But, if the device or filesystem where you are backing up logs fills, or there's another error, then the logs will continue to fill, and you will eventually run into a problem, so monitoring that you don't have many logs in not backed up status ("U--") is a good idea.

You probably don't need to back up all your logs again, but not sure how you are doing that anyway.

HI Mike (and Art), thanks for the replies and information - really appreciate that.  Checking Alarmprogram looks like there is an entry BACKUPLOGS=Y as well as BACKUP_CMD="ontape -a -d". So the answer would be continuous backup is set and logs are being backed once used then, aye?  With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?

# PUBLIC SECTION : CONFIGURATION VARIABLES
#                                         
# ########################################

BACKUPLOGS=Y

Then at night we also run the ontape -s L 0 to create a Level 0 file (which we use for restores to test) as well as then a full backup of the logs again.  But this is not essential given we're running continuous log backups anyway?

many thanks!

Mark

ps. Can confirm we don't have LTAPEDEV set to /dev/null either. 

Hi Mark,

The "B" in the 3rd position means that the logical log has been backed up, which is good!

The log backup will be triggered by the script referenced in your onconfig file with the parameter ALARMPROGRAM.  It is called each time a logical log is filled or logical log is switched.  The other possibility is that LTAPEDEV is set to /dev/null (definitely not recommended) in which case the logical log will not be backed up, yet the flag will still show that it has.

The logical log will switch when full, but there are other operations that will trigger a log switch and may be part of your restart process or a manual switch (onmode -l) and may be part of your ontape backup process also, which is why you see some logical logs used but less than 100% used.  If you look at your Informix log file (MSGPATH), you will see a record there of each time a logical log is backed up and you may be able to see from that some event around when the particular log backup was performed.

03/30/23 08:30:49  Logical Log 5375644 Complete, timestamp: 0xd1531780.

03/30/23 08:30:49  Logical Log 5375644 - Backup Started

03/30/23 08:30:50  Logical Log 5375644 - Backup Completed

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

The onmode -l will switch to the next logical log, but it not the ontape will remove the log backups.  It's one reason for you having logical logs that are not 100% used.  There may be a "find" with an "-exec rm" or something similar to delete old logical logs, or maybe a simple "rm".  It may not be the best idea to zap ALL logical log backups though as if the next archive fails and then something bad happens to your system, you won't have them to roll forward from after restoring from an earlier archive.  

HI Mike, thanks for the further reply.  Ahh, good points about the disk space where the log backups reside.  Will check further.

We presently run a cron job that kicks off a L0 night backup script which takes about an hour and executes:

onmode -l
ontape -s -L 0

This process appears to clear out the disk space once complete. 

Cheers
mark

alarmprogram.sh has been set up to back up the logical logs as expected - this is good.  Every time Informix switches to a new log, then alarmprogram will back up any logs not yet backed up.  It's not quite the same as continuous log backups, which is its own thing, and what you have here is a very common configuration.

As long as the backup continue to be successful, then yes, they will happily wrap and you won't get stuck in the "blocked" situation because the next log is used and not backed up.

But, if the device or filesystem where you are backing up logs fills, or there's another error, then the logs will continue to fill, and you will eventually run into a problem, so monitoring that you don't have many logs in not backed up status ("U--") is a good idea.

You probably don't need to back up all your logs again, but not sure how you are doing that anyway.

HI Mike (and Art), thanks for the replies and information - really appreciate that.  Checking Alarmprogram looks like there is an entry BACKUPLOGS=Y as well as BACKUP_CMD="ontape -a -d". So the answer would be continuous backup is set and logs are being backed once used then, aye?  With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?

# PUBLIC SECTION : CONFIGURATION VARIABLES
#                                         
# ########################################

BACKUPLOGS=Y

Then at night we also run the ontape -s L 0 to create a Level 0 file (which we use for restores to test) as well as then a full backup of the logs again.  But this is not essential given we're running continuous log backups anyway?

many thanks!

Mark

ps. Can confirm we don't have LTAPEDEV set to /dev/null either. 

Hi Mark,

The "B" in the 3rd position means that the logical log has been backed up, which is good!

The log backup will be triggered by the script referenced in your onconfig file with the parameter ALARMPROGRAM.  It is called each time a logical log is filled or logical log is switched.  The other possibility is that LTAPEDEV is set to /dev/null (definitely not recommended) in which case the logical log will not be backed up, yet the flag will still show that it has.

The logical log will switch when full, but there are other operations that will trigger a log switch and may be part of your restart process or a manual switch (onmode -l) and may be part of your ontape backup process also, which is why you see some logical logs used but less than 100% used.  If you look at your Informix log file (MSGPATH), you will see a record there of each time a logical log is backed up and you may be able to see from that some event around when the particular log backup was performed.

03/30/23 08:30:49  Logical Log 5375644 Complete, timestamp: 0xd1531780.

03/30/23 08:30:49  Logical Log 5375644 - Backup Started

03/30/23 08:30:50  Logical Log 5375644 - Backup Completed

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

Thanks Mike. 

in alarmprogram I can see

# ########################################
#                                        
#     PRIVATE SECTION : EVENT HANDLERS      
#                                      
# ########################################

[some other stuff i removed]

BACKUP_CMD="ontape -a -d"
RM="rm -f"

but this would not be influencing what happens on the nightly ontape job? And I'm not sure the logical log files are being dropped or deleted.

clearly, there's a lot I don't understand, but i do wonder how come the used and backed up log files do not drop back to 0 used after each log is backed up after the alarmprogram executes the ontape -a -d?  it drops the space/size when it becomes the new 'current' file?

address          number   flags    uniqid   begin                size     used    %used
763e0f88         7        U-B----  118015   6:53               153600   153600   100.00
77d88970         8        U-B----  118016   6:153653           153600   153600   100.00
77d889d8         9        U-B----  118017   6:307253           153600   113524    73.91
77d88a40         10       U-B----  118018   6:460853           153600    73370    47.77
77d88aa8         11       U-B----  118019   6:614453           153600   153600   100.00
77d88b10         12       U-B----  118020   6:768053           153600   153600   100.00

many thanks!
mark

The onmode -l will switch to the next logical log, but it not the ontape will remove the log backups.  It's one reason for you having logical logs that are not 100% used.  There may be a "find" with an "-exec rm" or something similar to delete old logical logs, or maybe a simple "rm".  It may not be the best idea to zap ALL logical log backups though as if the next archive fails and then something bad happens to your system, you won't have them to roll forward from after restoring from an earlier archive.  

HI Mike, thanks for the further reply.  Ahh, good points about the disk space where the log backups reside.  Will check further.

We presently run a cron job that kicks off a L0 night backup script which takes about an hour and executes:

onmode -l
ontape -s -L 0

This process appears to clear out the disk space once complete. 

Cheers
mark

alarmprogram.sh has been set up to back up the logical logs as expected - this is good.  Every time Informix switches to a new log, then alarmprogram will back up any logs not yet backed up.  It's not quite the same as continuous log backups, which is its own thing, and what you have here is a very common configuration.

As long as the backup continue to be successful, then yes, they will happily wrap and you won't get stuck in the "blocked" situation because the next log is used and not backed up.

But, if the device or filesystem where you are backing up logs fills, or there's another error, then the logs will continue to fill, and you will eventually run into a problem, so monitoring that you don't have many logs in not backed up status ("U--") is a good idea.

You probably don't need to back up all your logs again, but not sure how you are doing that anyway.

HI Mike (and Art), thanks for the replies and information - really appreciate that.  Checking Alarmprogram looks like there is an entry BACKUPLOGS=Y as well as BACKUP_CMD="ontape -a -d". So the answer would be continuous backup is set and logs are being backed once used then, aye?  With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?

# PUBLIC SECTION : CONFIGURATION VARIABLES
#                                         
# ########################################

BACKUPLOGS=Y

Then at night we also run the ontape -s L 0 to create a Level 0 file (which we use for restores to test) as well as then a full backup of the logs again.  But this is not essential given we're running continuous log backups anyway?

many thanks!

Mark

ps. Can confirm we don't have LTAPEDEV set to /dev/null either. 

Hi Mark,

The "B" in the 3rd position means that the logical log has been backed up, which is good!

The log backup will be triggered by the script referenced in your onconfig file with the parameter ALARMPROGRAM.  It is called each time a logical log is filled or logical log is switched.  The other possibility is that LTAPEDEV is set to /dev/null (definitely not recommended) in which case the logical log will not be backed up, yet the flag will still show that it has.

The logical log will switch when full, but there are other operations that will trigger a log switch and may be part of your restart process or a manual switch (onmode -l) and may be part of your ontape backup process also, which is why you see some logical logs used but less than 100% used.  If you look at your Informix log file (MSGPATH), you will see a record there of each time a logical log is backed up and you may be able to see from that some event around when the particular log backup was performed.

03/30/23 08:30:49  Logical Log 5375644 Complete, timestamp: 0xd1531780.

03/30/23 08:30:49  Logical Log 5375644 - Backup Started

03/30/23 08:30:50  Logical Log 5375644 - Backup Completed

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

The log files get zero out when the current log wraps around to reuse each one.

Best practice is to keep the latest two level 0 archives and all of the logical log backups going back to that older archive you are keeping. So say you archive on Saturdays. You would currently have the archive files from the 18th and the 25th. And all of the log backup files that are newer than the archive from the 18th. When the archive completes on April 1 you can delete the archive from 3/18 and all of the log files older than the archive file from 3/25.

Thus protocol will allow you to recover even if the newest archive is corrupted by restoring the older one and rolling forward all of those logical logs.

Art

Thanks Mike. 

in alarmprogram I can see

# ########################################
#                                        
#     PRIVATE SECTION : EVENT HANDLERS      
#                                      
# ########################################

[some other stuff i removed]

BACKUP_CMD="ontape -a -d"
RM="rm -f"

but this would not be influencing what happens on the nightly ontape job? And I'm not sure the logical log files are being dropped or deleted.

clearly, there's a lot I don't understand, but i do wonder how come the used and backed up log files do not drop back to 0 used after each log is backed up after the alarmprogram executes the ontape -a -d?  it drops the space/size when it becomes the new 'current' file?

address          number   flags    uniqid   begin                size     used    %used
763e0f88         7        U-B----  118015   6:53               153600   153600   100.00
77d88970         8        U-B----  118016   6:153653           153600   153600   100.00
77d889d8         9        U-B----  118017   6:307253           153600   113524    73.91
77d88a40         10       U-B----  118018   6:460853           153600    73370    47.77
77d88aa8         11       U-B----  118019   6:614453           153600   153600   100.00
77d88b10         12       U-B----  118020   6:768053           153600   153600   100.00

many thanks!
mark

The onmode -l will switch to the next logical log, but it not the ontape will remove the log backups.  It's one reason for you having logical logs that are not 100% used.  There may be a "find" with an "-exec rm" or something similar to delete old logical logs, or maybe a simple "rm".  It may not be the best idea to zap ALL logical log backups though as if the next archive fails and then something bad happens to your system, you won't have them to roll forward from after restoring from an earlier archive.  

HI Mike, thanks for the further reply.  Ahh, good points about the disk space where the log backups reside.  Will check further.

We presently run a cron job that kicks off a L0 night backup script which takes about an hour and executes:

onmode -l
ontape -s -L 0

This process appears to clear out the disk space once complete. 

Cheers
mark

alarmprogram.sh has been set up to back up the logical logs as expected - this is good.  Every time Informix switches to a new log, then alarmprogram will back up any logs not yet backed up.  It's not quite the same as continuous log backups, which is its own thing, and what you have here is a very common configuration.

As long as the backup continue to be successful, then yes, they will happily wrap and you won't get stuck in the "blocked" situation because the next log is used and not backed up.

But, if the device or filesystem where you are backing up logs fills, or there's another error, then the logs will continue to fill, and you will eventually run into a problem, so monitoring that you don't have many logs in not backed up status ("U--") is a good idea.

You probably don't need to back up all your logs again, but not sure how you are doing that anyway.

HI Mike (and Art), thanks for the replies and information - really appreciate that.  Checking Alarmprogram looks like there is an entry BACKUPLOGS=Y as well as BACKUP_CMD="ontape -a -d". So the answer would be continuous backup is set and logs are being backed once used then, aye?  With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?

# PUBLIC SECTION : CONFIGURATION VARIABLES
#                                         
# ########################################

BACKUPLOGS=Y

Then at night we also run the ontape -s L 0 to create a Level 0 file (which we use for restores to test) as well as then a full backup of the logs again.  But this is not essential given we're running continuous log backups anyway?

many thanks!

Mark

ps. Can confirm we don't have LTAPEDEV set to /dev/null either. 

Hi Mark,

The "B" in the 3rd position means that the logical log has been backed up, which is good!

The log backup will be triggered by the script referenced in your onconfig file with the parameter ALARMPROGRAM.  It is called each time a logical log is filled or logical log is switched.  The other possibility is that LTAPEDEV is set to /dev/null (definitely not recommended) in which case the logical log will not be backed up, yet the flag will still show that it has.

The logical log will switch when full, but there are other operations that will trigger a log switch and may be part of your restart process or a manual switch (onmode -l) and may be part of your ontape backup process also, which is why you see some logical logs used but less than 100% used.  If you look at your Informix log file (MSGPATH), you will see a record there of each time a logical log is backed up and you may be able to see from that some event around when the particular log backup was performed.

03/30/23 08:30:49  Logical Log 5375644 Complete, timestamp: 0xd1531780.

03/30/23 08:30:49  Logical Log 5375644 - Backup Started

03/30/23 08:30:50  Logical Log 5375644 - Backup Completed

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

Thanks Mike. 

in alarmprogram I can see

# ########################################
#                                        
#     PRIVATE SECTION : EVENT HANDLERS      
#                                      
# ########################################

[some other stuff i removed]

BACKUP_CMD="ontape -a -d"
RM="rm -f"

but this would not be influencing what happens on the nightly ontape job? And I'm not sure the logical log files are being dropped or deleted.

clearly, there's a lot I don't understand, but i do wonder how come the used and backed up log files do not drop back to 0 used after each log is backed up after the alarmprogram executes the ontape -a -d?  it drops the space/size when it becomes the new 'current' file?

address          number   flags    uniqid   begin                size     used    %used
763e0f88         7        U-B----  118015   6:53               153600   153600   100.00
77d88970         8        U-B----  118016   6:153653           153600   153600   100.00
77d889d8         9        U-B----  118017   6:307253           153600   113524    73.91
77d88a40         10       U-B----  118018   6:460853           153600    73370    47.77
77d88aa8         11       U-B----  118019   6:614453           153600   153600   100.00
77d88b10         12       U-B----  118020   6:768053           153600   153600   100.00

many thanks!
mark

The onmode -l will switch to the next logical log, but it not the ontape will remove the log backups.  It's one reason for you having logical logs that are not 100% used.  There may be a "find" with an "-exec rm" or something similar to delete old logical logs, or maybe a simple "rm".  It may not be the best idea to zap ALL logical log backups though as if the next archive fails and then something bad happens to your system, you won't have them to roll forward from after restoring from an earlier archive.  

HI Mike, thanks for the further reply.  Ahh, good points about the disk space where the log backups reside.  Will check further.

We presently run a cron job that kicks off a L0 night backup script which takes about an hour and executes:

onmode -l
ontape -s -L 0

This process appears to clear out the disk space once complete. 

Cheers
mark

alarmprogram.sh has been set up to back up the logical logs as expected - this is good.  Every time Informix switches to a new log, then alarmprogram will back up any logs not yet backed up.  It's not quite the same as continuous log backups, which is its own thing, and what you have here is a very common configuration.

As long as the backup continue to be successful, then yes, they will happily wrap and you won't get stuck in the "blocked" situation because the next log is used and not backed up.

But, if the device or filesystem where you are backing up logs fills, or there's another error, then the logs will continue to fill, and you will eventually run into a problem, so monitoring that you don't have many logs in not backed up status ("U--") is a good idea.

You probably don't need to back up all your logs again, but not sure how you are doing that anyway.

HI Mike (and Art), thanks for the replies and information - really appreciate that.  Checking Alarmprogram looks like there is an entry BACKUPLOGS=Y as well as BACKUP_CMD="ontape -a -d". So the answer would be continuous backup is set and logs are being backed once used then, aye?  With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?

# PUBLIC SECTION : CONFIGURATION VARIABLES
#                                         
# ########################################

BACKUPLOGS=Y

Then at night we also run the ontape -s L 0 to create a Level 0 file (which we use for restores to test) as well as then a full backup of the logs again.  But this is not essential given we're running continuous log backups anyway?

many thanks!

Mark

ps. Can confirm we don't have LTAPEDEV set to /dev/null either. 

Hi Mark,

The "B" in the 3rd position means that the logical log has been backed up, which is good!

The log backup will be triggered by the script referenced in your onconfig file with the parameter ALARMPROGRAM.  It is called each time a logical log is filled or logical log is switched.  The other possibility is that LTAPEDEV is set to /dev/null (definitely not recommended) in which case the logical log will not be backed up, yet the flag will still show that it has.

The logical log will switch when full, but there are other operations that will trigger a log switch and may be part of your restart process or a manual switch (onmode -l) and may be part of your ontape backup process also, which is why you see some logical logs used but less than 100% used.  If you look at your Informix log file (MSGPATH), you will see a record there of each time a logical log is backed up and you may be able to see from that some event around when the particular log backup was performed.

03/30/23 08:30:49  Logical Log 5375644 Complete, timestamp: 0xd1531780.

03/30/23 08:30:49  Logical Log 5375644 - Backup Started

03/30/23 08:30:50  Logical Log 5375644 - Backup Completed

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

The alarmprogram is just a shell script, so it can be customized to do all sorts of things.  However, it would be unusual to have alarmprogram do anything like remove the logical log backups after a full backup.  Unusual doesn't mean it can't be done though! 

I think that I understand your question about why doesn't the logical log show 0 used once it has been backed up...  Think of it like this...the logical logs are permanent "objects" or "things" saved in your database instance...they are not just stuff saved off when backed up.  In your last post, I can see that they live in chunk #6 (the first number in the "begin" field).  Don't confuse what you see in "onstat -l" as only a record of what's been saved off to disk with backups.  They are used for various recovery operations and replication and are a very important and necessary part of the Informix engine.  So, even once they have been backed up, they are retained, and the data contained within them is retained, and that's why they still show as used.  It's not until they wrap that the next one will be emptied and used to record activity going forward.  Because they play such an important part in recovery, is why Informix will not overwrite a logical log unless it has been backed up because to do so may mean not being able to recover fully after a failure.  I hope that makes sense!

Thanks Mike. 

in alarmprogram I can see

# ########################################
#                                        
#     PRIVATE SECTION : EVENT HANDLERS      
#                                      
# ########################################

[some other stuff i removed]

BACKUP_CMD="ontape -a -d"
RM="rm -f"

but this would not be influencing what happens on the nightly ontape job? And I'm not sure the logical log files are being dropped or deleted.

clearly, there's a lot I don't understand, but i do wonder how come the used and backed up log files do not drop back to 0 used after each log is backed up after the alarmprogram executes the ontape -a -d?  it drops the space/size when it becomes the new 'current' file?

address          number   flags    uniqid   begin                size     used    %used
763e0f88         7        U-B----  118015   6:53               153600   153600   100.00
77d88970         8        U-B----  118016   6:153653           153600   153600   100.00
77d889d8         9        U-B----  118017   6:307253           153600   113524    73.91
77d88a40         10       U-B----  118018   6:460853           153600    73370    47.77
77d88aa8         11       U-B----  118019   6:614453           153600   153600   100.00
77d88b10         12       U-B----  118020   6:768053           153600   153600   100.00

many thanks!
mark

The onmode -l will switch to the next logical log, but it not the ontape will remove the log backups.  It's one reason for you having logical logs that are not 100% used.  There may be a "find" with an "-exec rm" or something similar to delete old logical logs, or maybe a simple "rm".  It may not be the best idea to zap ALL logical log backups though as if the next archive fails and then something bad happens to your system, you won't have them to roll forward from after restoring from an earlier archive.  

HI Mike, thanks for the further reply.  Ahh, good points about the disk space where the log backups reside.  Will check further.

We presently run a cron job that kicks off a L0 night backup script which takes about an hour and executes:

onmode -l
ontape -s -L 0

This process appears to clear out the disk space once complete. 

Cheers
mark

alarmprogram.sh has been set up to back up the logical logs as expected - this is good.  Every time Informix switches to a new log, then alarmprogram will back up any logs not yet backed up.  It's not quite the same as continuous log backups, which is its own thing, and what you have here is a very common configuration.

As long as the backup continue to be successful, then yes, they will happily wrap and you won't get stuck in the "blocked" situation because the next log is used and not backed up.

But, if the device or filesystem where you are backing up logs fills, or there's another error, then the logs will continue to fill, and you will eventually run into a problem, so monitoring that you don't have many logs in not backed up status ("U--") is a good idea.

You probably don't need to back up all your logs again, but not sure how you are doing that anyway.

HI Mike (and Art), thanks for the replies and information - really appreciate that.  Checking Alarmprogram looks like there is an entry BACKUPLOGS=Y as well as BACKUP_CMD="ontape -a -d". So the answer would be continuous backup is set and logs are being backed once used then, aye?  With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?

# PUBLIC SECTION : CONFIGURATION VARIABLES
#                                         
# ########################################

BACKUPLOGS=Y

Then at night we also run the ontape -s L 0 to create a Level 0 file (which we use for restores to test) as well as then a full backup of the logs again.  But this is not essential given we're running continuous log backups anyway?

many thanks!

Mark

ps. Can confirm we don't have LTAPEDEV set to /dev/null either. 

Hi Mark,

The "B" in the 3rd position means that the logical log has been backed up, which is good!

The log backup will be triggered by the script referenced in your onconfig file with the parameter ALARMPROGRAM.  It is called each time a logical log is filled or logical log is switched.  The other possibility is that LTAPEDEV is set to /dev/null (definitely not recommended) in which case the logical log will not be backed up, yet the flag will still show that it has.

The logical log will switch when full, but there are other operations that will trigger a log switch and may be part of your restart process or a manual switch (onmode -l) and may be part of your ontape backup process also, which is why you see some logical logs used but less than 100% used.  If you look at your Informix log file (MSGPATH), you will see a record there of each time a logical log is backed up and you may be able to see from that some event around when the particular log backup was performed.

03/30/23 08:30:49  Logical Log 5375644 Complete, timestamp: 0xd1531780.

03/30/23 08:30:49  Logical Log 5375644 - Backup Started

03/30/23 08:30:50  Logical Log 5375644 - Backup Completed

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

Thanks for your reply too Mike, as per my reply to Art, I think the penny has started to drop for me now, i.e. the Logical logs won't change until they wrap back to current, hence the space used in the logs remains the same until they become current. I need to dig a bit more around the backup files that are generated, as opposed to the actual logical log file - the backup files are where the space changes will happen as the backup files are generated during the day and night, and then moved off to alternate storage.

Really appreciate the help and guidance from you both.

Cheers
Mark

The alarmprogram is just a shell script, so it can be customized to do all sorts of things.  However, it would be unusual to have alarmprogram do anything like remove the logical log backups after a full backup.  Unusual doesn't mean it can't be done though! 

I think that I understand your question about why doesn't the logical log show 0 used once it has been backed up...  Think of it like this...the logical logs are permanent "objects" or "things" saved in your database instance...they are not just stuff saved off when backed up.  In your last post, I can see that they live in chunk #6 (the first number in the "begin" field).  Don't confuse what you see in "onstat -l" as only a record of what's been saved off to disk with backups.  They are used for various recovery operations and replication and are a very important and necessary part of the Informix engine.  So, even once they have been backed up, they are retained, and the data contained within them is retained, and that's why they still show as used.  It's not until they wrap that the next one will be emptied and used to record activity going forward.  Because they play such an important part in recovery, is why Informix will not overwrite a logical log unless it has been backed up because to do so may mean not being able to recover fully after a failure.  I hope that makes sense!

Thanks Mike. 

in alarmprogram I can see

# ########################################
#                                        
#     PRIVATE SECTION : EVENT HANDLERS      
#                                      
# ########################################

[some other stuff i removed]

BACKUP_CMD="ontape -a -d"
RM="rm -f"

but this would not be influencing what happens on the nightly ontape job? And I'm not sure the logical log files are being dropped or deleted.

clearly, there's a lot I don't understand, but i do wonder how come the used and backed up log files do not drop back to 0 used after each log is backed up after the alarmprogram executes the ontape -a -d?  it drops the space/size when it becomes the new 'current' file?

address          number   flags    uniqid   begin                size     used    %used
763e0f88         7        U-B----  118015   6:53               153600   153600   100.00
77d88970         8        U-B----  118016   6:153653           153600   153600   100.00
77d889d8         9        U-B----  118017   6:307253           153600   113524    73.91
77d88a40         10       U-B----  118018   6:460853           153600    73370    47.77
77d88aa8         11       U-B----  118019   6:614453           153600   153600   100.00
77d88b10         12       U-B----  118020   6:768053           153600   153600   100.00

many thanks!
mark

The onmode -l will switch to the next logical log, but it not the ontape will remove the log backups.  It's one reason for you having logical logs that are not 100% used.  There may be a "find" with an "-exec rm" or something similar to delete old logical logs, or maybe a simple "rm".  It may not be the best idea to zap ALL logical log backups though as if the next archive fails and then something bad happens to your system, you won't have them to roll forward from after restoring from an earlier archive.  

HI Mike, thanks for the further reply.  Ahh, good points about the disk space where the log backups reside.  Will check further.

We presently run a cron job that kicks off a L0 night backup script which takes about an hour and executes:

onmode -l
ontape -s -L 0

This process appears to clear out the disk space once complete. 

Cheers
mark

alarmprogram.sh has been set up to back up the logical logs as expected - this is good.  Every time Informix switches to a new log, then alarmprogram will back up any logs not yet backed up.  It's not quite the same as continuous log backups, which is its own thing, and what you have here is a very common configuration.

As long as the backup continue to be successful, then yes, they will happily wrap and you won't get stuck in the "blocked" situation because the next log is used and not backed up.

But, if the device or filesystem where you are backing up logs fills, or there's another error, then the logs will continue to fill, and you will eventually run into a problem, so monitoring that you don't have many logs in not backed up status ("U--") is a good idea.

You probably don't need to back up all your logs again, but not sure how you are doing that anyway.

HI Mike (and Art), thanks for the replies and information - really appreciate that.  Checking Alarmprogram looks like there is an entry BACKUPLOGS=Y as well as BACKUP_CMD="ontape -a -d". So the answer would be continuous backup is set and logs are being backed once used then, aye?  With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?

# PUBLIC SECTION : CONFIGURATION VARIABLES
#                                         
# ########################################

BACKUPLOGS=Y

Then at night we also run the ontape -s L 0 to create a Level 0 file (which we use for restores to test) as well as then a full backup of the logs again.  But this is not essential given we're running continuous log backups anyway?

many thanks!

Mark

ps. Can confirm we don't have LTAPEDEV set to /dev/null either. 

Hi Mark,

The "B" in the 3rd position means that the logical log has been backed up, which is good!

The log backup will be triggered by the script referenced in your onconfig file with the parameter ALARMPROGRAM.  It is called each time a logical log is filled or logical log is switched.  The other possibility is that LTAPEDEV is set to /dev/null (definitely not recommended) in which case the logical log will not be backed up, yet the flag will still show that it has.

The logical log will switch when full, but there are other operations that will trigger a log switch and may be part of your restart process or a manual switch (onmode -l) and may be part of your ontape backup process also, which is why you see some logical logs used but less than 100% used.  If you look at your Informix log file (MSGPATH), you will see a record there of each time a logical log is backed up and you may be able to see from that some event around when the particular log backup was performed.

03/30/23 08:30:49  Logical Log 5375644 Complete, timestamp: 0xd1531780.

03/30/23 08:30:49  Logical Log 5375644 - Backup Started

03/30/23 08:30:50  Logical Log 5375644 - Backup Completed

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

The alarmprogram is just a shell script, so it can be customized to do all sorts of things.  However, it would be unusual to have alarmprogram do anything like remove the logical log backups after a full backup.  Unusual doesn't mean it can't be done though! 

I think that I understand your question about why doesn't the logical log show 0 used once it has been backed up...  Think of it like this...the logical logs are permanent "objects" or "things" saved in your database instance...they are not just stuff saved off when backed up.  In your last post, I can see that they live in chunk #6 (the first number in the "begin" field).  Don't confuse what you see in "onstat -l" as only a record of what's been saved off to disk with backups.  They are used for various recovery operations and replication and are a very important and necessary part of the Informix engine.  So, even once they have been backed up, they are retained, and the data contained within them is retained, and that's why they still show as used.  It's not until they wrap that the next one will be emptied and used to record activity going forward.  Because they play such an important part in recovery, is why Informix will not overwrite a logical log unless it has been backed up because to do so may mean not being able to recover fully after a failure.  I hope that makes sense!

Thanks Mike. 

in alarmprogram I can see

# ########################################
#                                        
#     PRIVATE SECTION : EVENT HANDLERS      
#                                      
# ########################################

[some other stuff i removed]

BACKUP_CMD="ontape -a -d"
RM="rm -f"

but this would not be influencing what happens on the nightly ontape job? And I'm not sure the logical log files are being dropped or deleted.

clearly, there's a lot I don't understand, but i do wonder how come the used and backed up log files do not drop back to 0 used after each log is backed up after the alarmprogram executes the ontape -a -d?  it drops the space/size when it becomes the new 'current' file?

address          number   flags    uniqid   begin                size     used    %used
763e0f88         7        U-B----  118015   6:53               153600   153600   100.00
77d88970         8        U-B----  118016   6:153653           153600   153600   100.00
77d889d8         9        U-B----  118017   6:307253           153600   113524    73.91
77d88a40         10       U-B----  118018   6:460853           153600    73370    47.77
77d88aa8         11       U-B----  118019   6:614453           153600   153600   100.00
77d88b10         12       U-B----  118020   6:768053           153600   153600   100.00

many thanks!
mark

The onmode -l will switch to the next logical log, but it not the ontape will remove the log backups.  It's one reason for you having logical logs that are not 100% used.  There may be a "find" with an "-exec rm" or something similar to delete old logical logs, or maybe a simple "rm".  It may not be the best idea to zap ALL logical log backups though as if the next archive fails and then something bad happens to your system, you won't have them to roll forward from after restoring from an earlier archive.  

HI Mike, thanks for the further reply.  Ahh, good points about the disk space where the log backups reside.  Will check further.

We presently run a cron job that kicks off a L0 night backup script which takes about an hour and executes:

onmode -l
ontape -s -L 0

This process appears to clear out the disk space once complete. 

Cheers
mark

alarmprogram.sh has been set up to back up the logical logs as expected - this is good.  Every time Informix switches to a new log, then alarmprogram will back up any logs not yet backed up.  It's not quite the same as continuous log backups, which is its own thing, and what you have here is a very common configuration.

As long as the backup continue to be successful, then yes, they will happily wrap and you won't get stuck in the "blocked" situation because the next log is used and not backed up.

But, if the device or filesystem where you are backing up logs fills, or there's another error, then the logs will continue to fill, and you will eventually run into a problem, so monitoring that you don't have many logs in not backed up status ("U--") is a good idea.

You probably don't need to back up all your logs again, but not sure how you are doing that anyway.

HI Mike (and Art), thanks for the replies and information - really appreciate that.  Checking Alarmprogram looks like there is an entry BACKUPLOGS=Y as well as BACKUP_CMD="ontape -a -d". So the answer would be continuous backup is set and logs are being backed once used then, aye?  With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?

# PUBLIC SECTION : CONFIGURATION VARIABLES
#                                         
# ########################################

BACKUPLOGS=Y

Then at night we also run the ontape -s L 0 to create a Level 0 file (which we use for restores to test) as well as then a full backup of the logs again.  But this is not essential given we're running continuous log backups anyway?

many thanks!

Mark

ps. Can confirm we don't have LTAPEDEV set to /dev/null either. 

Hi Mark,

The "B" in the 3rd position means that the logical log has been backed up, which is good!

The log backup will be triggered by the script referenced in your onconfig file with the parameter ALARMPROGRAM.  It is called each time a logical log is filled or logical log is switched.  The other possibility is that LTAPEDEV is set to /dev/null (definitely not recommended) in which case the logical log will not be backed up, yet the flag will still show that it has.

The logical log will switch when full, but there are other operations that will trigger a log switch and may be part of your restart process or a manual switch (onmode -l) and may be part of your ontape backup process also, which is why you see some logical logs used but less than 100% used.  If you look at your Informix log file (MSGPATH), you will see a record there of each time a logical log is backed up and you may be able to see from that some event around when the particular log backup was performed.

03/30/23 08:30:49  Logical Log 5375644 Complete, timestamp: 0xd1531780.

03/30/23 08:30:49  Logical Log 5375644 - Backup Started

03/30/23 08:30:50  Logical Log 5375644 - Backup Completed

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

HI Mike (and Art), thanks for the replies and information - really appreciate that.  Checking Alarmprogram looks like there is an entry BACKUPLOGS=Y as well as BACKUP_CMD="ontape -a -d". So the answer would be continuous backup is set and logs are being backed once used then, aye?  With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?

# PUBLIC SECTION : CONFIGURATION VARIABLES
#                                         
# ########################################

BACKUPLOGS=Y

Then at night we also run the ontape -s L 0 to create a Level 0 file (which we use for restores to test) as well as then a full backup of the logs again.  But this is not essential given we're running continuous log backups anyway?

many thanks!

Mark

ps. Can confirm we don't have LTAPEDEV set to /dev/null either. 

Hi Mark,

The "B" in the 3rd position means that the logical log has been backed up, which is good!

The log backup will be triggered by the script referenced in your onconfig file with the parameter ALARMPROGRAM.  It is called each time a logical log is filled or logical log is switched.  The other possibility is that LTAPEDEV is set to /dev/null (definitely not recommended) in which case the logical log will not be backed up, yet the flag will still show that it has.

The logical log will switch when full, but there are other operations that will trigger a log switch and may be part of your restart process or a manual switch (onmode -l) and may be part of your ontape backup process also, which is why you see some logical logs used but less than 100% used.  If you look at your Informix log file (MSGPATH), you will see a record there of each time a logical log is backed up and you may be able to see from that some event around when the particular log backup was performed.

03/30/23 08:30:49  Logical Log 5375644 Complete, timestamp: 0xd1531780.

03/30/23 08:30:49  Logical Log 5375644 - Backup Started

03/30/23 08:30:50  Logical Log 5375644 - Backup Completed

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

Hi Mark,

One statement in your post above has not been addressed - "With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?"

The answer is a qualified "Yes".  The qualification has to do with what's called a Long Transaction.  A long transaction is not based on time, or even by how many updates it actually performs, but on the percentage of available logical logs that a given transaction spans.  This is controlled by the onconfig parameters LTXHWM and LTXEHWM.  If a transaction takes up too much of the available logical logs, the system will start to worry, not being certain that the transaction will finish before it fills up the logical logs.  As I said earlier, the transaction in question may not be doing that much update activity.  It may be that someone is in dbaccess and did a BEGIN WORK, and updated a single row, but did not do a COMMIT WORK, but got distracted, went to lunch, got called to a meeting, whatever.  That transaction is still open, and the logical log in which it started (as well as any later logical logs) cannot be released for reuse until that transaction is closed by either a COMMIT or a ROLLBACK WORK.

For example, say your system has 20 logical logs, and LTXHWM is set to 50, and LTXEHWM is set to 60.  If the current logical log number is 445 at the time that a transaction starts, that means you have 20 logical logs available (#445 - 464).  If that transaction has not done a COMMIT or ROLLBACK prior to the logical logs reaching the 50% mark (#455), the engine will force it to do a ROLLBACK, and you will see a message in your online.log file saying "Long transaction aborted".  The second parameter (LTXEHWM) is a bit of a failsafe.  As the system-initiated ROLLBACK is in progress, it is writing additional records to the logical logs.  And if you've got multiple users on the system, they may be running other transactions, which also are writing to the logical logs.  When the system sees that you have reached 60% full (12 logs into your 20 available because LTXEHWM is set to 60, so at logical log 457), the system will suspend all of the other update activity so that the ROLLBACK has exclusive use of the remaining logical log space, hoping that it can complete the ROLLBACK before it runs out of space.

In the old days, you could fill up the logical logs and the system would just hang, as there was no more space to write in the logical logs.  Worse, if you tried to add a new dbspace or chunk, or simply add a new logical log in an existing dbspace, the system needed to write to the logical log to record the fact that you had added this, but since there was no space, it could not do so.  They have added onconfig parameters (DYNAMIC_LOGS, AUTO_LLOGS, possibly others) to help you avoid this situation, but since you are new, you'll need to review those to confirm that they are in fact in place.

HI Mike (and Art), thanks for the replies and information - really appreciate that.  Checking Alarmprogram looks like there is an entry BACKUPLOGS=Y as well as BACKUP_CMD="ontape -a -d". So the answer would be continuous backup is set and logs are being backed once used then, aye?  With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?

# PUBLIC SECTION : CONFIGURATION VARIABLES
#                                         
# ########################################

BACKUPLOGS=Y

Then at night we also run the ontape -s L 0 to create a Level 0 file (which we use for restores to test) as well as then a full backup of the logs again.  But this is not essential given we're running continuous log backups anyway?

many thanks!

Mark

ps. Can confirm we don't have LTAPEDEV set to /dev/null either. 

Hi Mark,

The "B" in the 3rd position means that the logical log has been backed up, which is good!

The log backup will be triggered by the script referenced in your onconfig file with the parameter ALARMPROGRAM.  It is called each time a logical log is filled or logical log is switched.  The other possibility is that LTAPEDEV is set to /dev/null (definitely not recommended) in which case the logical log will not be backed up, yet the flag will still show that it has.

The logical log will switch when full, but there are other operations that will trigger a log switch and may be part of your restart process or a manual switch (onmode -l) and may be part of your ontape backup process also, which is why you see some logical logs used but less than 100% used.  If you look at your Informix log file (MSGPATH), you will see a record there of each time a logical log is backed up and you may be able to see from that some event around when the particular log backup was performed.

03/30/23 08:30:49  Logical Log 5375644 Complete, timestamp: 0xd1531780.

03/30/23 08:30:49  Logical Log 5375644 - Backup Started

03/30/23 08:30:50  Logical Log 5375644 - Backup Completed

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

Hi Mark, thanks for the further info and reply.  That long transaction rollback thing is hard to get my head around but the explanation helped clarify.  And I certainly take your point on qualifying the "we'll never run out of disk space" comment. I've got a task to make sure we have more robust monitoring on processes and disk space etc. so that's scenario is a good thing for us to be aware of.

In terms of settings we have:
DYNAMIC_LOGS 2

LTXHWM 70
LTXEHWM 80

I'm not sure if 70 / 80 are quite high values though as could burn a lot of spare disk before reaching them?

Cheers
Mark

Hi Mark,

One statement in your post above has not been addressed - "With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?"

The answer is a qualified "Yes".  The qualification has to do with what's called a Long Transaction.  A long transaction is not based on time, or even by how many updates it actually performs, but on the percentage of available logical logs that a given transaction spans.  This is controlled by the onconfig parameters LTXHWM and LTXEHWM.  If a transaction takes up too much of the available logical logs, the system will start to worry, not being certain that the transaction will finish before it fills up the logical logs.  As I said earlier, the transaction in question may not be doing that much update activity.  It may be that someone is in dbaccess and did a BEGIN WORK, and updated a single row, but did not do a COMMIT WORK, but got distracted, went to lunch, got called to a meeting, whatever.  That transaction is still open, and the logical log in which it started (as well as any later logical logs) cannot be released for reuse until that transaction is closed by either a COMMIT or a ROLLBACK WORK.

For example, say your system has 20 logical logs, and LTXHWM is set to 50, and LTXEHWM is set to 60.  If the current logical log number is 445 at the time that a transaction starts, that means you have 20 logical logs available (#445 - 464).  If that transaction has not done a COMMIT or ROLLBACK prior to the logical logs reaching the 50% mark (#455), the engine will force it to do a ROLLBACK, and you will see a message in your online.log file saying "Long transaction aborted".  The second parameter (LTXEHWM) is a bit of a failsafe.  As the system-initiated ROLLBACK is in progress, it is writing additional records to the logical logs.  And if you've got multiple users on the system, they may be running other transactions, which also are writing to the logical logs.  When the system sees that you have reached 60% full (12 logs into your 20 available because LTXEHWM is set to 60, so at logical log 457), the system will suspend all of the other update activity so that the ROLLBACK has exclusive use of the remaining logical log space, hoping that it can complete the ROLLBACK before it runs out of space.

In the old days, you could fill up the logical logs and the system would just hang, as there was no more space to write in the logical logs.  Worse, if you tried to add a new dbspace or chunk, or simply add a new logical log in an existing dbspace, the system needed to write to the logical log to record the fact that you had added this, but since there was no space, it could not do so.  They have added onconfig parameters (DYNAMIC_LOGS, AUTO_LLOGS, possibly others) to help you avoid this situation, but since you are new, you'll need to review those to confirm that they are in fact in place.

HI Mike (and Art), thanks for the replies and information - really appreciate that.  Checking Alarmprogram looks like there is an entry BACKUPLOGS=Y as well as BACKUP_CMD="ontape -a -d". So the answer would be continuous backup is set and logs are being backed once used then, aye?  With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?

# PUBLIC SECTION : CONFIGURATION VARIABLES
#                                         
# ########################################

BACKUPLOGS=Y

Then at night we also run the ontape -s L 0 to create a Level 0 file (which we use for restores to test) as well as then a full backup of the logs again.  But this is not essential given we're running continuous log backups anyway?

many thanks!

Mark

ps. Can confirm we don't have LTAPEDEV set to /dev/null either. 

Hi Mark,

The "B" in the 3rd position means that the logical log has been backed up, which is good!

The log backup will be triggered by the script referenced in your onconfig file with the parameter ALARMPROGRAM.  It is called each time a logical log is filled or logical log is switched.  The other possibility is that LTAPEDEV is set to /dev/null (definitely not recommended) in which case the logical log will not be backed up, yet the flag will still show that it has.

The logical log will switch when full, but there are other operations that will trigger a log switch and may be part of your restart process or a manual switch (onmode -l) and may be part of your ontape backup process also, which is why you see some logical logs used but less than 100% used.  If you look at your Informix log file (MSGPATH), you will see a record there of each time a logical log is backed up and you may be able to see from that some event around when the particular log backup was performed.

03/30/23 08:30:49  Logical Log 5375644 Complete, timestamp: 0xd1531780.

03/30/23 08:30:49  Logical Log 5375644 - Backup Started

03/30/23 08:30:50  Logical Log 5375644 - Backup Completed

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

Mark,

The DYNAMIC_LOGS value of 2 is good, but make certain that you've got plenty of space left in your rootdbs, as that's where the extra logical logs will be created.  The LTXHWM/LTXEHWM values of 70/80 are (I believe) the defaults.  So long as your application is doing small transactions, it probably is OK, but that's something you have to check.  Loading large amounts of data without intermediate COMMITs might require tweaking those default values.  Even with tweaking, you may have situations where you don't have enough logical log space.  In one of my test instances, I don't have enough space to do a dbimport of a particular database because of one table.  I don't have any more disk allocated to that instance, so I can't add logical logs.  For that one, I just import the database as an unlogged database, then use ondblog to change the database to logged and run a level 0 archive.  I also could probably find something in Art's utilities to do the import with periodic commits, but I've not looked into it yet.

I wanted to expand on one point from my previous post.  The LTXHWM/LTXEHWM values are percentages of logical logs, not of the logical log pages.  The engine does not care that the start of a transaction is near the very end of a certain logical log.  The entire logical log is prevented from being reused so long as there is an open transaction inside that logical log.  So whether the BEGIN WORK record was the third item written to a log file or next-to-last thing written, it's the same effect - that log file cannot be reused until the transaction is committed or rolled back.

With that in mind, it is better to have a lot of smaller logical logs than a few large ones.  If you were to allocate 1 GB of space to logical logs, it would be better to have one hundred 10 MB logical logs than to have ten 100 MB logs.

Hi Mark, thanks for the further info and reply.  That long transaction rollback thing is hard to get my head around but the explanation helped clarify.  And I certainly take your point on qualifying the "we'll never run out of disk space" comment. I've got a task to make sure we have more robust monitoring on processes and disk space etc. so that's scenario is a good thing for us to be aware of.

In terms of settings we have:
DYNAMIC_LOGS 2

LTXHWM 70
LTXEHWM 80

I'm not sure if 70 / 80 are quite high values though as could burn a lot of spare disk before reaching them?

Cheers
Mark

Hi Mark,

One statement in your post above has not been addressed - "With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?"

The answer is a qualified "Yes".  The qualification has to do with what's called a Long Transaction.  A long transaction is not based on time, or even by how many updates it actually performs, but on the percentage of available logical logs that a given transaction spans.  This is controlled by the onconfig parameters LTXHWM and LTXEHWM.  If a transaction takes up too much of the available logical logs, the system will start to worry, not being certain that the transaction will finish before it fills up the logical logs.  As I said earlier, the transaction in question may not be doing that much update activity.  It may be that someone is in dbaccess and did a BEGIN WORK, and updated a single row, but did not do a COMMIT WORK, but got distracted, went to lunch, got called to a meeting, whatever.  That transaction is still open, and the logical log in which it started (as well as any later logical logs) cannot be released for reuse until that transaction is closed by either a COMMIT or a ROLLBACK WORK.

For example, say your system has 20 logical logs, and LTXHWM is set to 50, and LTXEHWM is set to 60.  If the current logical log number is 445 at the time that a transaction starts, that means you have 20 logical logs available (#445 - 464).  If that transaction has not done a COMMIT or ROLLBACK prior to the logical logs reaching the 50% mark (#455), the engine will force it to do a ROLLBACK, and you will see a message in your online.log file saying "Long transaction aborted".  The second parameter (LTXEHWM) is a bit of a failsafe.  As the system-initiated ROLLBACK is in progress, it is writing additional records to the logical logs.  And if you've got multiple users on the system, they may be running other transactions, which also are writing to the logical logs.  When the system sees that you have reached 60% full (12 logs into your 20 available because LTXEHWM is set to 60, so at logical log 457), the system will suspend all of the other update activity so that the ROLLBACK has exclusive use of the remaining logical log space, hoping that it can complete the ROLLBACK before it runs out of space.

In the old days, you could fill up the logical logs and the system would just hang, as there was no more space to write in the logical logs.  Worse, if you tried to add a new dbspace or chunk, or simply add a new logical log in an existing dbspace, the system needed to write to the logical log to record the fact that you had added this, but since there was no space, it could not do so.  They have added onconfig parameters (DYNAMIC_LOGS, AUTO_LLOGS, possibly others) to help you avoid this situation, but since you are new, you'll need to review those to confirm that they are in fact in place.

HI Mike (and Art), thanks for the replies and information - really appreciate that.  Checking Alarmprogram looks like there is an entry BACKUPLOGS=Y as well as BACKUP_CMD="ontape -a -d". So the answer would be continuous backup is set and logs are being backed once used then, aye?  With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?

# PUBLIC SECTION : CONFIGURATION VARIABLES
#                                         
# ########################################

BACKUPLOGS=Y

Then at night we also run the ontape -s L 0 to create a Level 0 file (which we use for restores to test) as well as then a full backup of the logs again.  But this is not essential given we're running continuous log backups anyway?

many thanks!

Mark

ps. Can confirm we don't have LTAPEDEV set to /dev/null either. 

Hi Mark,

The "B" in the 3rd position means that the logical log has been backed up, which is good!

The log backup will be triggered by the script referenced in your onconfig file with the parameter ALARMPROGRAM.  It is called each time a logical log is filled or logical log is switched.  The other possibility is that LTAPEDEV is set to /dev/null (definitely not recommended) in which case the logical log will not be backed up, yet the flag will still show that it has.

The logical log will switch when full, but there are other operations that will trigger a log switch and may be part of your restart process or a manual switch (onmode -l) and may be part of your ontape backup process also, which is why you see some logical logs used but less than 100% used.  If you look at your Informix log file (MSGPATH), you will see a record there of each time a logical log is backed up and you may be able to see from that some event around when the particular log backup was performed.

03/30/23 08:30:49  Logical Log 5375644 Complete, timestamp: 0xd1531780.

03/30/23 08:30:49  Logical Log 5375644 - Backup Started

03/30/23 08:30:50  Logical Log 5375644 - Backup Completed

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

Hi Mark, thanks for the reply and further info.  Will review that area in a bit more detail in the future as it raises some interesting points.

One other thing.  I noted in the manual there is an option for a quiescent backup, but the manual is lite in terms of how this gets set etc?  Any useful references for understanding this better?

Cheers
Mark

Mark,

The DYNAMIC_LOGS value of 2 is good, but make certain that you've got plenty of space left in your rootdbs, as that's where the extra logical logs will be created.  The LTXHWM/LTXEHWM values of 70/80 are (I believe) the defaults.  So long as your application is doing small transactions, it probably is OK, but that's something you have to check.  Loading large amounts of data without intermediate COMMITs might require tweaking those default values.  Even with tweaking, you may have situations where you don't have enough logical log space.  In one of my test instances, I don't have enough space to do a dbimport of a particular database because of one table.  I don't have any more disk allocated to that instance, so I can't add logical logs.  For that one, I just import the database as an unlogged database, then use ondblog to change the database to logged and run a level 0 archive.  I also could probably find something in Art's utilities to do the import with periodic commits, but I've not looked into it yet.

I wanted to expand on one point from my previous post.  The LTXHWM/LTXEHWM values are percentages of logical logs, not of the logical log pages.  The engine does not care that the start of a transaction is near the very end of a certain logical log.  The entire logical log is prevented from being reused so long as there is an open transaction inside that logical log.  So whether the BEGIN WORK record was the third item written to a log file or next-to-last thing written, it's the same effect - that log file cannot be reused until the transaction is committed or rolled back.

With that in mind, it is better to have a lot of smaller logical logs than a few large ones.  If you were to allocate 1 GB of space to logical logs, it would be better to have one hundred 10 MB logical logs than to have ten 100 MB logs.

Hi Mark, thanks for the further info and reply.  That long transaction rollback thing is hard to get my head around but the explanation helped clarify.  And I certainly take your point on qualifying the "we'll never run out of disk space" comment. I've got a task to make sure we have more robust monitoring on processes and disk space etc. so that's scenario is a good thing for us to be aware of.

In terms of settings we have:
DYNAMIC_LOGS 2

LTXHWM 70
LTXEHWM 80

I'm not sure if 70 / 80 are quite high values though as could burn a lot of spare disk before reaching them?

Cheers
Mark

Hi Mark,

One statement in your post above has not been addressed - "With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?"

The answer is a qualified "Yes".  The qualification has to do with what's called a Long Transaction.  A long transaction is not based on time, or even by how many updates it actually performs, but on the percentage of available logical logs that a given transaction spans.  This is controlled by the onconfig parameters LTXHWM and LTXEHWM.  If a transaction takes up too much of the available logical logs, the system will start to worry, not being certain that the transaction will finish before it fills up the logical logs.  As I said earlier, the transaction in question may not be doing that much update activity.  It may be that someone is in dbaccess and did a BEGIN WORK, and updated a single row, but did not do a COMMIT WORK, but got distracted, went to lunch, got called to a meeting, whatever.  That transaction is still open, and the logical log in which it started (as well as any later logical logs) cannot be released for reuse until that transaction is closed by either a COMMIT or a ROLLBACK WORK.

For example, say your system has 20 logical logs, and LTXHWM is set to 50, and LTXEHWM is set to 60.  If the current logical log number is 445 at the time that a transaction starts, that means you have 20 logical logs available (#445 - 464).  If that transaction has not done a COMMIT or ROLLBACK prior to the logical logs reaching the 50% mark (#455), the engine will force it to do a ROLLBACK, and you will see a message in your online.log file saying "Long transaction aborted".  The second parameter (LTXEHWM) is a bit of a failsafe.  As the system-initiated ROLLBACK is in progress, it is writing additional records to the logical logs.  And if you've got multiple users on the system, they may be running other transactions, which also are writing to the logical logs.  When the system sees that you have reached 60% full (12 logs into your 20 available because LTXEHWM is set to 60, so at logical log 457), the system will suspend all of the other update activity so that the ROLLBACK has exclusive use of the remaining logical log space, hoping that it can complete the ROLLBACK before it runs out of space.

In the old days, you could fill up the logical logs and the system would just hang, as there was no more space to write in the logical logs.  Worse, if you tried to add a new dbspace or chunk, or simply add a new logical log in an existing dbspace, the system needed to write to the logical log to record the fact that you had added this, but since there was no space, it could not do so.  They have added onconfig parameters (DYNAMIC_LOGS, AUTO_LLOGS, possibly others) to help you avoid this situation, but since you are new, you'll need to review those to confirm that they are in fact in place.

HI Mike (and Art), thanks for the replies and information - really appreciate that.  Checking Alarmprogram looks like there is an entry BACKUPLOGS=Y as well as BACKUP_CMD="ontape -a -d". So the answer would be continuous backup is set and logs are being backed once used then, aye?  With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?

# PUBLIC SECTION : CONFIGURATION VARIABLES
#                                         
# ########################################

BACKUPLOGS=Y

Then at night we also run the ontape -s L 0 to create a Level 0 file (which we use for restores to test) as well as then a full backup of the logs again.  But this is not essential given we're running continuous log backups anyway?

many thanks!

Mark

ps. Can confirm we don't have LTAPEDEV set to /dev/null either. 

Hi Mark,

The "B" in the 3rd position means that the logical log has been backed up, which is good!

The log backup will be triggered by the script referenced in your onconfig file with the parameter ALARMPROGRAM.  It is called each time a logical log is filled or logical log is switched.  The other possibility is that LTAPEDEV is set to /dev/null (definitely not recommended) in which case the logical log will not be backed up, yet the flag will still show that it has.

The logical log will switch when full, but there are other operations that will trigger a log switch and may be part of your restart process or a manual switch (onmode -l) and may be part of your ontape backup process also, which is why you see some logical logs used but less than 100% used.  If you look at your Informix log file (MSGPATH), you will see a record there of each time a logical log is backed up and you may be able to see from that some event around when the particular log backup was performed.

03/30/23 08:30:49  Logical Log 5375644 Complete, timestamp: 0xd1531780.

03/30/23 08:30:49  Logical Log 5375644 - Backup Started

03/30/23 08:30:50  Logical Log 5375644 - Backup Completed

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

I don't know the specific passage in the manual that mentions quiescent backups, but you can place the instance in quiescent mode via the 'onmode -s' command.  The '-s' leaves existing sessions connected and allows them to continue until the user ends the session.  If you're impatient, you can use the 'onmode -u' to kill all attached sessions and switch to quiescent mode.

Hi Mark, thanks for the reply and further info.  Will review that area in a bit more detail in the future as it raises some interesting points.

One other thing.  I noted in the manual there is an option for a quiescent backup, but the manual is lite in terms of how this gets set etc?  Any useful references for understanding this better?

Cheers
Mark

Mark,

The DYNAMIC_LOGS value of 2 is good, but make certain that you've got plenty of space left in your rootdbs, as that's where the extra logical logs will be created.  The LTXHWM/LTXEHWM values of 70/80 are (I believe) the defaults.  So long as your application is doing small transactions, it probably is OK, but that's something you have to check.  Loading large amounts of data without intermediate COMMITs might require tweaking those default values.  Even with tweaking, you may have situations where you don't have enough logical log space.  In one of my test instances, I don't have enough space to do a dbimport of a particular database because of one table.  I don't have any more disk allocated to that instance, so I can't add logical logs.  For that one, I just import the database as an unlogged database, then use ondblog to change the database to logged and run a level 0 archive.  I also could probably find something in Art's utilities to do the import with periodic commits, but I've not looked into it yet.

I wanted to expand on one point from my previous post.  The LTXHWM/LTXEHWM values are percentages of logical logs, not of the logical log pages.  The engine does not care that the start of a transaction is near the very end of a certain logical log.  The entire logical log is prevented from being reused so long as there is an open transaction inside that logical log.  So whether the BEGIN WORK record was the third item written to a log file or next-to-last thing written, it's the same effect - that log file cannot be reused until the transaction is committed or rolled back.

With that in mind, it is better to have a lot of smaller logical logs than a few large ones.  If you were to allocate 1 GB of space to logical logs, it would be better to have one hundred 10 MB logical logs than to have ten 100 MB logs.

Hi Mark, thanks for the further info and reply.  That long transaction rollback thing is hard to get my head around but the explanation helped clarify.  And I certainly take your point on qualifying the "we'll never run out of disk space" comment. I've got a task to make sure we have more robust monitoring on processes and disk space etc. so that's scenario is a good thing for us to be aware of.

In terms of settings we have:
DYNAMIC_LOGS 2

LTXHWM 70
LTXEHWM 80

I'm not sure if 70 / 80 are quite high values though as could burn a lot of spare disk before reaching them?

Cheers
Mark

Hi Mark,

One statement in your post above has not been addressed - "With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?"

The answer is a qualified "Yes".  The qualification has to do with what's called a Long Transaction.  A long transaction is not based on time, or even by how many updates it actually performs, but on the percentage of available logical logs that a given transaction spans.  This is controlled by the onconfig parameters LTXHWM and LTXEHWM.  If a transaction takes up too much of the available logical logs, the system will start to worry, not being certain that the transaction will finish before it fills up the logical logs.  As I said earlier, the transaction in question may not be doing that much update activity.  It may be that someone is in dbaccess and did a BEGIN WORK, and updated a single row, but did not do a COMMIT WORK, but got distracted, went to lunch, got called to a meeting, whatever.  That transaction is still open, and the logical log in which it started (as well as any later logical logs) cannot be released for reuse until that transaction is closed by either a COMMIT or a ROLLBACK WORK.

For example, say your system has 20 logical logs, and LTXHWM is set to 50, and LTXEHWM is set to 60.  If the current logical log number is 445 at the time that a transaction starts, that means you have 20 logical logs available (#445 - 464).  If that transaction has not done a COMMIT or ROLLBACK prior to the logical logs reaching the 50% mark (#455), the engine will force it to do a ROLLBACK, and you will see a message in your online.log file saying "Long transaction aborted".  The second parameter (LTXEHWM) is a bit of a failsafe.  As the system-initiated ROLLBACK is in progress, it is writing additional records to the logical logs.  And if you've got multiple users on the system, they may be running other transactions, which also are writing to the logical logs.  When the system sees that you have reached 60% full (12 logs into your 20 available because LTXEHWM is set to 60, so at logical log 457), the system will suspend all of the other update activity so that the ROLLBACK has exclusive use of the remaining logical log space, hoping that it can complete the ROLLBACK before it runs out of space.

In the old days, you could fill up the logical logs and the system would just hang, as there was no more space to write in the logical logs.  Worse, if you tried to add a new dbspace or chunk, or simply add a new logical log in an existing dbspace, the system needed to write to the logical log to record the fact that you had added this, but since there was no space, it could not do so.  They have added onconfig parameters (DYNAMIC_LOGS, AUTO_LLOGS, possibly others) to help you avoid this situation, but since you are new, you'll need to review those to confirm that they are in fact in place.

HI Mike (and Art), thanks for the replies and information - really appreciate that.  Checking Alarmprogram looks like there is an entry BACKUPLOGS=Y as well as BACKUP_CMD="ontape -a -d". So the answer would be continuous backup is set and logs are being backed once used then, aye?  With this setting would it be safe to also assume that we would not run out of log space etc. as backups are continuous and would just continue to cycle around in sequence?

# PUBLIC SECTION : CONFIGURATION VARIABLES
#                                         
# ########################################

BACKUPLOGS=Y

Then at night we also run the ontape -s L 0 to create a Level 0 file (which we use for restores to test) as well as then a full backup of the logs again.  But this is not essential given we're running continuous log backups anyway?

many thanks!

Mark

ps. Can confirm we don't have LTAPEDEV set to /dev/null either. 

Hi Mark,

The "B" in the 3rd position means that the logical log has been backed up, which is good!

The log backup will be triggered by the script referenced in your onconfig file with the parameter ALARMPROGRAM.  It is called each time a logical log is filled or logical log is switched.  The other possibility is that LTAPEDEV is set to /dev/null (definitely not recommended) in which case the logical log will not be backed up, yet the flag will still show that it has.

The logical log will switch when full, but there are other operations that will trigger a log switch and may be part of your restart process or a manual switch (onmode -l) and may be part of your ontape backup process also, which is why you see some logical logs used but less than 100% used.  If you look at your Informix log file (MSGPATH), you will see a record there of each time a logical log is backed up and you may be able to see from that some event around when the particular log backup was performed.

03/30/23 08:30:49  Logical Log 5375644 Complete, timestamp: 0xd1531780.

03/30/23 08:30:49  Logical Log 5375644 - Backup Started

03/30/23 08:30:50  Logical Log 5375644 - Backup Completed

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------

Mark:

There are five ways in which the logical logs will be marked with the "B" indicating that the log has been backed up and is available to be reused when the "current" log comes around again:

1. A manual backup using ontape -a or onbar -b -l
2. A continuous backup of logs as they fill by using ontape -c or onbar -b -l -C
3. Backup of each log as it fills using the ALARMPROGRAM
4. During a full system archive if you respond "Y" to the prompt asking whether to back up the logs
5. If the ONCONFIG paramter LTAPEDEV is set to /dev/null logs are discarded when they fill and are marked as backed up (B) without actually having saved their contents at all.

Art

Hi, total newbie here but needing some guidance re: logical logs and what the onstat flags mean, so would appreciate anything to help improve our understanding. Been reading up on what we can, but informix DBA knowledge is a bit thin in this neck of the woods, hence the forum question.

below is a snip from onstat -l.   Informix is 12.10 FC13

My understanding is current Logical Log in use is 47, but was expecting logs immediately prior (46, 45 etc) to on be flagged as U------ (used, ready for backup).  With the status U-B----, does that means they're being backed up as they are used, or am I not understanding this correctly?  if they are backing up as used would that be a setting in Alarmprogram.sh then?  But then we also have a nightly ontape -s L 0 which clears the logging disk space before that slowly builds again during the day.  In the full Log list there appear to be no logs with just a U or F flag (maybe immediately after the backup there is, but that's in the wee small hours)

And secondary to that, why are logs 41, 46, and 50 only partially filled?

Thanks in advance and appreciate your collective patience with the 101 questions.

Cheers

Mark

------------------------------
Mark Clayton
------------------------------