What is a SYSVAR-enabled environment in OMEGAMON?
SYSVAR environments are the runtime environments which do not fully resolve configuration parameter values during generation. This gives you the ability to plug in system dependent symbols in your runtime environments. In addition to PARMLIB-defined symbols, you also have the ability to define your own symbols which we refer to user-defined variables, more on this topic can be found in the doc. The scope where variables stay unresolved is limited to RKANPARU, RKANSAMU and RKANCMDU (plus RKD2PAR, RKD2PRF and RKD2SAM in OMEGAMON for Db2 agent) libraries. SYSVAR environments are available both in PARMGEN and Configuration Manager. However, maintenance scenarios between these tools differ. In Configuration Manager, the only maintenance scenario is GENERATE, but you can also control it using OPTIONs. This post will focus on Configuration Manager and the generic ideas of flexible RTEs.
Note: parts of the PARMGEN doc regarding variables still apply to Configuration Manager, as it creates the same RTE artifacts. I may also use a word "symbol" as a synonym to "variable", as "symbolic" is a nice adjective defining the behavior introduced by a variable. For the simplification, I will also refer to three main RK* SYSVAR-enabled libraries as an "RKAN*U triad", as they are present in most agents. OMEGAMON for Db2 adds more.
SYSVAR setting can be toggled by using the parameter RTE_SYSV_SYSVAR_FLAG. The default is N. In the initial RTEDEF created by Configuration Manager CREATE action, this parameter is hidden, so to enable the setting you would need to add it to RTEDEF(rte_name) member. RTE_SYSV_SYSVAR_FLAG is scoped over the entire RTE, so it would be enabled for all agents configured, but the RTEDEF can have a mixture of SYSVAR and non-SYSVAR environments.
With SYSVAR setting enabled, GENERATE will not resolve parameters which use a standard &SYMBOL. notation, and other parameters will operate normally. Your parameter can either use a symbol partially, or the value can be fully symbolic. In this example, both user (&SYSIPHOSTNAME.) and system (&SYSNAME.) variables are used:
RTE_TCP_HOST "&SYSIPHOSTNAME." * The whole parameter value is a system variable
RTE_TEMS_NAME_NODEID "&SYSNAME.:TEMS" * System variable is used in the value partially
Not all parameters were created equal
If you'd like to use a symbol in the parameter, it is important to choose the correct parameter type:
- Some parameters are practically directly copied into members of RKAN*U libraries, only to be read by the agents. They do not necessarily impact how those members are generated, they are not used to derive other parameter values.
- Other parameters control how the RTE is generated. These are the parameters that Configuration Manager checks against various rules, and builds the output accordingly.
Guess which ones are not really eligible for SYSVARS use?
If your goal is to have a dynamic RTE that responds to the reconfiguration of the LPAR, you should only enable variables in the parameters that fall under option 1. It may be tricky to know which is which, and the knowledge will come with experimentation or experience. There are thousands of parameters after all. However, at some point in the past, PARMGEN shipped and maintained configuration templates, and one of them included the best practice parameters for system variable enablement. Most of those parameters listed in the template were related to network settings for the TEMS. But there can be others.
One example of the variable which is ineligible for SYSVARS use is a set of Kpp_AGT_COMM_PROTOCOLn parameters. Configuration process must always know about the value in its resolved form to process the product templates correctly, which is not possible with SYSVARS. You can still have SYSVARS on that environment enabled, but you must keep this parameter value hard coded.
SYSVARS are resolved at agent startup
The only official SYSVARS use case is that OMEGAMON agents should be able to run with different configuration parameters without re-running all of the reconfiguration steps.
When SYSVARS are enabled, there are three data sets processed by KCIPARSE as the first step of agent's JCL: RKANCMDU, RKANPARU and RKANSAMU. The following DD names are allocated and processed:
USERCMDU → KCIPARSE → TMPCMDU
USERPARU → KCIPARSE → TMPPARU
USERSAMU → KCIPARSE → TMPSAMU
If you configure OMEGAMON for Db2, it would additionally process RKD2PAR, RKD2PRF and RKD2SAM:
USERD2PU → KCIPARSE → TMPD2PU
USERD2FU → KCIPARSE → TMPD2FU
USERD2SU → KCIPARSE → TMPD2SU
These data sets with variables fully resolved are allocated to the started tasks as temporary data sets. When the agent stops, temp data sets are deleted, and upon the next startup, must be resolved again.
For backwards compatibility with the KCIPARSE variable resolution step, Configuration Manager also creates a USERJCL data set, so KCIPARSE attempts to resolve values found in USERJCL first.
User-defined variables are completely optional
In Configuration Manager, RTEDEF VAR$GLOB and VAR$lpar members are reserved for user-created symbols only. You do not need to add PARMLIB-defined symbols here, only the ones that may be too specific to be used outside of OMEGAMON. As a bonus, the name is not limited to 8 characters. In addition to VAR$* members, you might have also noticed that DISCOVER action fetches system symbols from the PARMLIB definitions and creates a SYS@lpar member. This member is also optional, and is most often used for remote environments. If it was created as part of the discovery, but you find you do not need it, you can delete it from RTEDEF. It is important to understand that a system symbol will be resolved at the agent startup even if it's not found in USERJCL - it is sufficient to have it defined in PARMLIB.
In some cases, you might not need user-defined symbols at all, if PARMLIB-defined symbols have it all covered. We recommend that you review your environment, as you might find it can be simplified.
Risks associated with using system variables
In Configuration Manager, system and user symbols that have been defined in RTEDEF, will have to be synced with USERJCL. KCIPARSE will not read RTEDEF directly. Currently, the only way to update the USERJCL is by rerunning GENERATE (OPTION QUICKCONFIG can be used). If this is not feasible on your environments, then you can copy the updated variables from RTEDEF manually. We are looking at how this can be improved.
In addition to the issue mentioned, the following risks are also present:
- You will not be able to fully validate the parameter value, so there might be errors during agent startup.
- You might run into an unresolved parameter - in the remote configuration scenarios, you run Configuration Manager jobs on a configuration LPAR, and you do not have the full picture of system variables on your remote system. System symbols might have changed since you last ran GENERATE.
- If you changed a value of a variable, and the agent is already running, you need to have the temporary data sets rebuilt - that means you must recycle the started task.
- All symbols are resolved during agent startup in the additional step. It may not be much, but it still impacts the speed of agent startup. The extent of it depends on the size of your environments.
- It is more difficult to diagnose issues with the agents due to this additional abstraction layer.
Possible layers of abstraction
Let's briefly ignore the current implementation and think about which layers of RTE creation could be abstracted. Such layers could be: configuration layer, shared SYSVAR-enabled RKAN*U libraries, shared procs, and procs that respond to environment changes. Currently SYSVAR-enabled environments implement the "procs that respond to environment changes" abstraction layer.
Without having SYSVARS enabled, but using system symbols as parameter values, you could achieve the "configuration layer" abstraction. With SYSVARS disabled, you could also achieve the shared proc abstraction. While the former should be solved by defining LPAR-level members in your RTEDEF configuration, and not by using the variables, the latter is actually possible today.
Creating a shared agent started task without enabling SYSVARS in Configuration Manager
Your requirement could only be "having shared started tasks" - i.e., you can only have one started task JCL per SYSPLEX. With some manual modifications, you can achieve this shared agent STC functionality, and your environments do not have to be SYSVAR-enabled! You still need to have at least one suitable PARMLIB symbol which you will use to identify the LPAR for the started task. It may be as simple as a &SYSNAME. In this case, the RTE_NAME must also be equal to a &SYSNAME.
Generally, a generated proc will always refer to its own set of RTE data sets, including the RKAN*U triad. So the proc parameter then must have a variable to differentiate between the systems. In this example, manually edited PROC parameters were: SYS, USERCMDU, USERPARU and USERSAMU.
//TEMSHUB PROC RGN=0M,TIM=1440,MEMLIM=NOLIMIT,
// SYS=&SYSNAME.,
// RHILEV=OMEG.DEMO,
// BASEHLEV=OMEG.DEMO.SMPE.T,
// USERCMDU=OMEG.DEMO.&SYSNAME..RKANCMDU,
// USERPARU=OMEG.DEMO.&SYSNAME..RKANPARU,
// USERSAMU=OMEG.DEMO.&SYSNAME..RKANSAMU,
// SOUT=X, LOG OUTPUT CLASS
// DOUT=X, DEBUGGING OUTPUT CLASS
// RVHILEV=OMEG.DEMO,
// STARTUP=KDSSYSIN
In addition to this proc change, VTAM definitions can use &SYSCLONE. in the APPLID fields.
There is no value that SYSVARS add in this example.
A shared RKAN*U triad
An interesting use case could be the ability to create a single triad of RKAN*U data sets, and refer to those in agent procs. Currently, RKAN*U data sets are created for each RTE separately. What if you used the same set? You may have so many systems to monitor, that your RKAN*U data sets are virtually identical. In that case, SYSVAR-enabled environment would not be a bad idea. To create such a setup, some manual edits of agent STCs and renaming of RKAN*U data sets are also required.
What is your system variable use case?
Do you want to have a completely predictable GENERATE, and have every single variable controlled through RTEDEF? Or maybe you only need to reuse the generated procs? In that case, SYSVARS should be disabled.
If your environments are large, difficult to maintain otherwise, then you may benefit from SYSVARS.
If you have any additional thoughts or need advice regarding the simplification of the RTEDEF data set, please contact Configuration Manager support.
Post-configuration of started tasks
The methods of editing procs mentioned here are classified under post-configuration actions. While we ask you to never edit RK* data sets manually, it is OK to edit generated procs as required. Configuration Manager generates procs from the initial templates, removes comments, and copies them to a data set specified by the parameter GBL_DSN_SYS1_PROCLIB, or to the data set RTE_PLIB_HILEV.SYS1.PROCLIB (default behavior). Configuration Manager will not rely on the formatting of the procs for further operation, so you may edit them as needed.
If you have any SYSVAR use cases not currently covered, please raise an IBM Idea and we will consider it. SYSVAR-enabled environments are advanced, so do not hesitate to seek help - you may contact support for assistance, or leave a comment here with any ideas you might have.
#OMEGAMON #IBMZ #AIOpsonZ