A recent discussion on LinkedIn discouraged IPLing frequently. And they have a good point as each IPL resets your query optimizer and throws out all temporary indexes. The catch was basically you should reduce IPLs for PTF's, etc.

Here's the catch though: How often should you put on the PTF's? I think we're better than most shops because we put on the latest cume and groups on a quarterly basis. And there's always those shops that want to be some lengthy amount of time behind. But how do you explain to your stakeholders that you were notified that you were potentially exposed because you don't have this Java group ptf on, or you don't have this level of firmware on, or this Hiper group on? Do you tell them:

- "I know of the risk that we could lose a bunch of confidential information but what if there's a flawed PTF and it causes issues?"

- "I know that IBM found an issue with the current level we're at and issued a fixing PTF but what if that fixing PTF causes issues?"

- "We've been on this level for two years now and nothing has happened so why fix what isn't broken?"

We're finding out that newer certification levels are requiring you to put patches on much more frequently than most people do. And, frankly, it takes less disruptive time to patch an IBM Power system than it does a Windows Server.

If there's an incident, and management finds out that you were notified by IBM notifications and you didn't put on the patch, are you going under the bus?

------------------------------
Robert Berendt IBMChampion
------------------------------

We have an ongoing discussion about the frequency of ptf upgrades.  What is best practice?  2 times a year is a normal frequency.  Http ptfs come every 3 months according to Tim Rowe.  But it does not go beyond the database in the IBMi.  What about security ptfs and other improvements and fixes?

------------------------------
Erik Aasland
IBMi administrator
Fremtind Insurance
------------------------------

Is frequency still a consideration?
Is frequency combined with notifications the way one should be going?  For example, if you got one of the following notifications recently, and you just did one of your 'twice a year' updates do you wait 6 months?
https://www.ibm.com/support/pages/node/6997499?myns=swgother&mynp=OCSWG60&mynp=OCSS9QQS&mynp=OCSSTS2D&mynp=OCSSB23CE&mynp=OCSSC5L9&mync=E&cm_sp=swgother-_-OCSWG60-OCSS9QQS-OCSSTS2D-OCSSB23CE-OCSSC5L9-_-E
https://www.ibm.com/support/pages/node/6993021?mhsrc=ibmsearch_a&mhq=CVE%202023-30438

------------------------------
Robert Berendt IBMChampion
------------------------------

Original Message:
Sent: Tue May 23, 2023 02:59 PM
From: Erik Aasland
Subject: How often to IPL?

We have an ongoing discussion about the frequency of ptf upgrades.  What is best practice?  2 times a year is a normal frequency.  Http ptfs come every 3 months according to Tim Rowe.  But it does not go beyond the database in the IBMi.  What about security ptfs and other improvements and fixes?

------------------------------
Erik Aasland
IBMi administrator
Fremtind Insurance
------------------------------

Those were the answers I was looking for. In my opinion, which you also share, you cannot have an upgrade frequency x number of times a year, but rather look at criticality and cost benefit. New ptfs can also mean new functionality that can make an improvement.

Do you have other points of view regarding ptf updates?

For the temporary indexes it's like Birgitta says "convert to permanent indexes" it's a good idea. When startup after IPL you need more cpu to create all the temporary indexes again and it will take som time.

------------------------------
Erik Aasland
IBMi administrator
Fremtind Insurance
------------------------------

Original Message:
Sent: Tue May 23, 2023 03:31 PM
From: Robert Berendt
Subject: How often to IPL?

Is frequency still a consideration?
Is frequency combined with notifications the way one should be going?  For example, if you got one of the following notifications recently, and you just did one of your 'twice a year' updates do you wait 6 months?
https://www.ibm.com/support/pages/node/6997499?myns=swgother&mynp=OCSWG60&mynp=OCSS9QQS&mynp=OCSSTS2D&mynp=OCSSB23CE&mynp=OCSSC5L9&mync=E&cm_sp=swgother-_-OCSWG60-OCSS9QQS-OCSSTS2D-OCSSB23CE-OCSSC5L9-_-E
https://www.ibm.com/support/pages/node/6993021?mhsrc=ibmsearch_a&mhq=CVE%202023-30438

------------------------------
Robert Berendt IBMChampion
------------------------------


Original Message:
Sent: Tue May 23, 2023 02:59 PM
From: Erik Aasland
Subject: How often to IPL?

We have an ongoing discussion about the frequency of ptf upgrades.  What is best practice?  2 times a year is a normal frequency.  Http ptfs come every 3 months according to Tim Rowe.  But it does not go beyond the database in the IBMi.  What about security ptfs and other improvements and fixes?

------------------------------
Erik Aasland
IBMi administrator
Fremtind Insurance

If the SQL Plan Cache is the only reason to avoid IPLs ... I'd suggest to analyze your SQL data and convert the MTIs (Maintained Temporary Indexes) into permanent indexes.
... but IMHO it is enough to IPL after PTF fixes. 

------------------------------
Birgitta Hauser
Database and Software Engineer
Selfemployed - Modernization-Education-Consulting on IBM i
Kaufering
+49 170 5269964
------------------------------

>>>>  If there's an incident, and management finds out that you were notified by IBM notifications and you didn't put on the patch, are you going under the bus? <<<<

Practically no customers in my geography ever subscribe to such IBM Notifications and never have to deal with this aspect of the matter :-)   Ignorance is bliss or not. 

In my experience, customers who have DR systems and the switchover process is not complex are in the best position to do regular PTF application followed by IPL but I would not encourage them to do it more than once a quarter, except for any PTF notification that causes them deep concern (most of my customers in my part of the world are not too savvy in this respect.).  Customers who practically have 24/7 large workload (banks, service companies and web stores) and have complex DR switchover process are the ones that have been reluctant to IPL at all, even after a decade :-) 

I also notice that the type of customer who suffers severely after each IPL is one who runs 100% (or high proportion of) SQL workload (such as the old SAP ECC, J.D. Edwards EnterpriseOne, Infor M3, etc) that is large and NEVER creates any useful indexes for their workload.  They almost always suffer degraded overall SQL workload performance for a few to several hours because all autonomic indexes are deleted at IPL and the system takes a bit of time to recreate them after SQL  workload resumes.  The degree of this issue seems to be less severe in servers that no longer use any spinning disk.   

------------------------------
Education is not the learning of facts but the training of the mind to think. -- Albert Einstein.
------------------------------
Satid S.
------------------------------

That "ignorance is bliss" is a good point.  Is it legally better NOT to subscribe to notifications if you are highly reluctant to adjust your fix/IPL schedule?

------------------------------
Robert Berendt IBMChampion
------------------------------

Original Message:
Sent: Wed May 24, 2023 03:19 AM
From: Satid Singkorapoom
Subject: How often to IPL?

>>>>  If there's an incident, and management finds out that you were notified by IBM notifications and you didn't put on the patch, are you going under the bus? <<<<

Practically no customers in my geography ever subscribe to such IBM Notifications and never have to deal with this aspect of the matter :-)   Ignorance is bliss or not. 

In my experience, customers who have DR systems and the switchover process is not complex are in the best position to do regular PTF application followed by IPL but I would not encourage them to do it more than once a quarter, except for any PTF notification that causes them deep concern (most of my customers in my part of the world are not too savvy in this respect.).  Customers who practically have 24/7 large workload (banks, service companies and web stores) and have complex DR switchover process are the ones that have been reluctant to IPL at all, even after a decade :-) 

I also notice that the type of customer who suffers severely after each IPL is one who runs 100% (or high proportion of) SQL workload (such as the old SAP ECC, J.D. Edwards EnterpriseOne, Infor M3, etc) that is large and NEVER creates any useful indexes for their workload.  They almost always suffer degraded overall SQL workload performance for a few to several hours because all autonomic indexes are deleted at IPL and the system takes a bit of time to recreate them after SQL  workload resumes.  The degree of this issue seems to be less severe in servers that no longer use any spinning disk.   

------------------------------
Education is not the learning of facts but the training of the mind to think. -- Albert Einstein.
------------------------------
Satid S.
------------------------------

Obvious answer: I think nowadays depends only by the company policy and by the uptime needs.

In my case, I IPL the production once a year just to keep the OS up to date with PTF/TR/Firmware. First the slave (same hardware) machine to limit the risks, then the main (I run a powerha setup).
If security demands it (only in really severe cases, with proper assessment, when a critical vulnerability impact a software components really used by the company), one can plan for the patch accordingly if it requires IPL.
Frankly, with the new nvme machines (native OS, no VIOS), a full refresh, applying all ptf groups after a year of accumulation + IPL reboot, was about 20 minutes circa.

It is still an half hour window, so it must be planned accordingly for environments requiring 24/7.

DB2Mirror could be a solution to keep an active-active with even heterogeneous PTF levels between machines, let's see how the solution mature after gaining some time on the field.

------------------------------
--ft
------------------------------

Thanks for the reply.
<snip>
Frankly, with the new nvme machines (native OS, no VIOS), a full refresh, applying all ptf groups after a year of accumulation + IPL reboot, was about 20 minutes circa.
</snip>
I have VIOS and NVMe on SAN on Power 10 and I agree with you.  I can upgrade the OS and put on the latest cume and groups in less time than it took to put on this last Windows patch.

------------------------------
Robert Berendt IBMChampion
------------------------------

Original Message:
Sent: Wed May 24, 2023 04:11 AM
From: ace ace
Subject: How often to IPL?

Obvious answer: I think nowadays depends only by the company policy and by the uptime needs.

In my case, I IPL the production once a year just to keep the OS up to date with PTF/TR/Firmware. First the slave (same hardware) machine to limit the risks, then the main (I run a powerha setup).
If security demands it (only in really severe cases, with proper assessment, when a critical vulnerability impact a software components really used by the company), one can plan for the patch accordingly if it requires IPL.
Frankly, with the new nvme machines (native OS, no VIOS), a full refresh, applying all ptf groups after a year of accumulation + IPL reboot, was about 20 minutes circa.

It is still an half hour window, so it must be planned accordingly for environments requiring 24/7.

DB2Mirror could be a solution to keep an active-active with even heterogeneous PTF levels between machines, let's see how the solution mature after gaining some time on the field.

------------------------------
--ft
------------------------------

Oddly enough I got a reply on another forum that there is someone who still has an issue with "% temp addresses" on DSPSYSSTS.  I haven't had an issue with that since CISC days.  He has to IPL because of that.  It hasn't been a big enough issue to pursue more vigorously.
Remember, "% temp addresses" is NOT temporary space.  Temporary space is something you can research with some of the newer IBM i Services.

------------------------------
Robert Berendt IBMChampion
------------------------------

Hi Rob, that was me. :)  And yes indeed after ~6 months LPAR uptime I see 53% temp addresses used.  Ideally I'd like to be applying PTFs (and IPLing) 4x per year, and I'd settle for 2x per year.  In reality I get 1 outage per year for my main LPAR, which is where temp addresses become a concern.

We also have an issue with very high "native full opens".

Unfortunately it feels like I'm screaming into the void when I try to raise either of these concerns.

------------------------------
Steven Riedmueller Certified IBM i Admin - Speaker, Mentor, and Advocate
------------------------------

Original Message:
Sent: Wed May 24, 2023 07:43 AM
From: Robert Berendt
Subject: How often to IPL?

Oddly enough I got a reply on another forum that there is someone who still has an issue with "% temp addresses" on DSPSYSSTS.  I haven't had an issue with that since CISC days.  He has to IPL because of that.  It hasn't been a big enough issue to pursue more vigorously.
Remember, "% temp addresses" is NOT temporary space.  Temporary space is something you can research with some of the newer IBM i Services.

------------------------------
Robert Berendt IBMChampion
------------------------------

Kent Milligan has just made a DB2 for i blog post sharing his insight on IPL, DB2 Plan Cache and autonomic index (Maintained temp index - MTI) with a focus on SQL workload performance after an IPL.   One useful information is on the new MTI_INFO service that helps us be aware of all existing autonomic indexes to help us make a decision to create them (not necessarily all) as the normal indexes to prevent what he calls "Monday Blues" :-)    Read it here : https://db2ibmi.blogspot.com/2023/06/the-amazing-disappearing-plan-cache.html 

------------------------------
Education is not the learning of facts but the training of the mind to think. -- Albert Einstein.
------------------------------
Satid S.
------------------------------

Original Message:
Sent: Wed May 24, 2023 09:12 AM
From: Steven Riedmueller
Subject: How often to IPL?

Hi Rob, that was me. :)  And yes indeed after ~6 months LPAR uptime I see 53% temp addresses used.  Ideally I'd like to be applying PTFs (and IPLing) 4x per year, and I'd settle for 2x per year.  In reality I get 1 outage per year for my main LPAR, which is where temp addresses become a concern.

We also have an issue with very high "native full opens".

Unfortunately it feels like I'm screaming into the void when I try to raise either of these concerns.

------------------------------
Steven Riedmueller Certified IBM i Admin - Speaker, Mentor, and Advocate
------------------------------


Original Message:
Sent: Wed May 24, 2023 07:43 AM
From: Robert Berendt
Subject: How often to IPL?

Oddly enough I got a reply on another forum that there is someone who still has an issue with "% temp addresses" on DSPSYSSTS.  I haven't had an issue with that since CISC days.  He has to IPL because of that.  It hasn't been a big enough issue to pursue more vigorously.
Remember, "% temp addresses" is NOT temporary space.  Temporary space is something you can research with some of the newer IBM i Services.

------------------------------
Robert Berendt IBMChampion

Thanks Satid.  In case it might be useful to someone who finds this thread, I came up with a query to estimate the date when the system would reach 100% temp address utilization.  This is based on the last IPL date (taken as the timestamp of QCTL subsystem starting up) and the current temp address percentage.

--Description: based on the start date of QCTL subsystem and the current temp address percentage, estimate the date when temp addresses would reach 100% causing a crash of the LPAR
SELECT CAST((A.TEMPORARY_ADDRESS_RATE)/(TIMESTAMPDIFF(16, CHAR(CURRENT TIMESTAMP - B.JOB_ACTIVE_TIME))) AS DEC(8,6)) AS TEMP_ADDRESS_PERCENT_PER_DAY, 
       (100-A.TEMPORARY_ADDRESS_RATE) AS TEMP_ADDRESS_PERCENT_REMAINING,
       (100-A.TEMPORARY_ADDRESS_RATE) / ((A.TEMPORARY_ADDRESS_RATE)/(TIMESTAMPDIFF(16, CHAR(CURRENT TIMESTAMP - B.JOB_ACTIVE_TIME)))) AS DAYS_UNTIL_100PERCENT,
       CURRENT DATE + ((100-A.TEMPORARY_ADDRESS_RATE) / ((A.TEMPORARY_ADDRESS_RATE)/(TIMESTAMPDIFF(16, CHAR(CURRENT TIMESTAMP - B.JOB_ACTIVE_TIME))))) DAYS AS PROJECTED_CRASH_DATE
    FROM QSYS2.SYSTEM_STATUS_INFO_BASIC A
         CROSS JOIN
             TABLE ( QSYS2.ACTIVE_JOB_INFO( JOB_NAME_FILTER => 'QCTL', DETAILED_INFO => 'WORK') ) B
    WHERE JOB_TYPE = 'SBS';

------------------------------
Steven Riedmueller Certified IBM i Admin - Speaker, Mentor, and Advocate
------------------------------

Original Message:
Sent: Sat June 24, 2023 10:32 PM
From: Satid Singkorapoom
Subject: How often to IPL?

Kent Milligan has just made a DB2 for i blog post sharing his insight on IPL, DB2 Plan Cache and autonomic index (Maintained temp index - MTI) with a focus on SQL workload performance after an IPL.   One useful information is on the new MTI_INFO service that helps us be aware of all existing autonomic indexes to help us make a decision to create them (not necessarily all) as the normal indexes to prevent what he calls "Monday Blues" :-)    Read it here : https://db2ibmi.blogspot.com/2023/06/the-amazing-disappearing-plan-cache.html 

------------------------------
Education is not the learning of facts but the training of the mind to think. -- Albert Einstein.
------------------------------
Satid S.
------------------------------


Original Message:
Sent: Wed May 24, 2023 09:12 AM
From: Steven Riedmueller
Subject: How often to IPL?

Hi Rob, that was me. :)  And yes indeed after ~6 months LPAR uptime I see 53% temp addresses used.  Ideally I'd like to be applying PTFs (and IPLing) 4x per year, and I'd settle for 2x per year.  In reality I get 1 outage per year for my main LPAR, which is where temp addresses become a concern.

We also have an issue with very high "native full opens".

Unfortunately it feels like I'm screaming into the void when I try to raise either of these concerns.

------------------------------
Steven Riedmueller Certified IBM i Admin - Speaker, Mentor, and Advocate


Original Message:
Sent: Wed May 24, 2023 07:43 AM
From: Robert Berendt
Subject: How often to IPL?

Oddly enough I got a reply on another forum that there is someone who still has an issue with "% temp addresses" on DSPSYSSTS.  I haven't had an issue with that since CISC days.  He has to IPL because of that.  It hasn't been a big enough issue to pursue more vigorously.
Remember, "% temp addresses" is NOT temporary space.  Temporary space is something you can research with some of the newer IBM i Services.

------------------------------
Robert Berendt IBMChampion

Hello, like other people said, it depends on the customer environment,  you have to make an assesment to review all about the index strategy and how often your data  change. Other thing  to consider  is to have a second  site with any HA solution, so you  can apply the PTFs in the primary machine meanwhile the users work in the HA Machine.

------------------------------
Juan Martin Araya Carrasco
------------------------------