Guys/Girls

I’ve been searching primarily here on how to setup a truly global variable. A variable that can be shared across services/flows.
I want to be able to modify the variable as well.

Of course I could set something up a Db Table, but I was hoping that this was more fundamental inside the product.

Any ideas or am I missing something fundamental?

thanks,

tim wiley()

Using concept called create a new Specification on File/new menu tab(near to creat new Flowservice,DocumentType…)inside the IS6.01 Developer tool.

Using this we have to define a new specification containing all necessary fields,records ,create once and use it globally across all the services in the pipe.

HTH.

Tim,

You can use a repository of the webMethods. There are good amount of services in pub.Storage where you can read and modify the varibales.

HTH

Uday

Specifications are for something else RMG…for a consistent interface to a flow service. It has nothing to do with a using a global variable.

Anyways, you can use a java service to make a static variable which will be available across all flows, but it’ll be in memory and will get wiped when the server goes down. So you could create a startup service which would load the constants at startup time.

Guys,
Appreciate all the help… Uday I can’t believe I overlooked the Storage aspect of webMethods… Not much info on this in the documentation I have. I am actually using Sap Business Connector( webMethods v4 ).

I have got the storage thing working in a matter of minutes… I learn something new about this product all the time. For some reason i was thinking the storage functions required allot of backend preparation. Boy was I wrong.

Thanks again for the help Uday, I certainly appreciate it.

Kristi,
Thanks as well. I was thinking about doing what you mentioned or a db table lookup but I think the storage routines are a natural place for this type of functionality.

thanks,

tim wiley()

Tim,

Anything can be global in webMethods land - unless you take pains to limit the scope of a variable - or drop it - or clean up etc…

If you know a variable is going to be in the pipeline - put it in the INPUT pipeline of your flow and map it to the desired target.

I have used the repository for this purpose as suggested by Uday - and I believe Ray Moser may have written an ezine about this approach.

There is a risk with using the repository - as it is used heavily by the IS server - especially in clustering - if you read the clustering docs - they recommend against using it. I think its a bit awkward for this kind of use - as you have to register a store before you can use it - so depending on your requirements - you may have to consider cleaning up the store as well. Furthermore - this amounts to storing data in a database - so performance takes a hit over just keeping a value in memory. Again - depends on your application - if your flow logic requires many hits to the store - the hit will be more expensive.

You can create your own “storage” flow using a hashmap (or whatever collection is appropriate). You will probably end up writing a “get” and “put” flow and away you go. May be a little more readable than declaring values in the input pipeline.

As is always the case with webMethods - you have to keep in mind that the design time view you have in developer does not tell you the full picture - the pipeline usually has way more stuff in it than you can view in design time.

Pipeline is not global across sessions though right? I thought each session gets its own pipeline. I know all variables within the pipeline are global to that process unless you scope them out, but they aren’t shared across services started via another session.

I hope I’m not totally off base here on this, because I’ve written tons of flows that get called in parallel and none of them have confused variables yet…

Personally I’m afraid of the repository. You have to worry about deadlocks, the corruption of the file based repository system, figuring out what’s actually in the repository (over time you could be leaving extra junk in there), etc.

You could have an “initConstants” flow where you either hardcode the constants and have all the other flows call this service to obtain the constants, or move the constants into a java properties file and load them at startup. My vote is NOT to use the repository from a couple years of painful experiences.

You are correct that each session gets its own pipeline. No worries!

When I answered your original post - I wasn’t thinking across sessions.

To accomplish this - you will have to persist the value somehow.

I’m with Will on this one - I would stay away from the repository.

But - since you mentioned you will need to modify the value - concurrancy will be an issue.

Haithem

Hi Haithem. You wrote:
“There is a risk with using the repository - as it is used heavily by the IS server - especially in clustering - if you read the clustering docs - they recommend against using it.”

Could you comment a bit more on complications with using the repository in a clustered environment?

The PDF document on Clustering that is in the IS_Server/doc directory didn’t have many negative comments. Is there something about this we can download from Advantage?

Cheers!
Sonam

Yes the flow constant or the java init would be ok… except Like I said I occasionally have to modify these values and unfortunately the flow method will only allow static constants.

Has anyone truly corrupted the repository and under what circumstance, because what I’m talking about is not high concurrency. We may have one or 2 writes over a period of days to 1 variable and all of the other time reading the value of this variable.

This is quite common in all web servers having system level variables that all sessions can reference in code.

Unless people have experienced severe predictable problems with using the storage I’m going to go ahead. This server gets nightly backups. So restoring this thing is a piece of cake since its just one directory. But if you guys are saying this thing will corrupt on low usage and its guaranteed then I guess I will have to go back to using a remote database to hold my variables.

Also… just a thought but would the Administrator utility use the same storage routines to modify its own parameters.

So when I make config changes to the server aren’t we under the same risk of corrupting the repository. I mean we could be adding users or changing passwords right and corrupt the repository.

Anyhow this was just a thought about the storage routine stability.

FWIW, we were using the file-based repository for clustering in a development environment and it got corrupted when our disk filled up from unpurged log files. Support of course recommended we not use the file-based repository for production clustering, but the database approach instead.

Re: the Administrative settings, many of the settings you’d configure as an Administrator are stored in the server.cnf file, not in the repository. I’m not sure about usernames, passwords, ACLs, etc., though.

Personally, I’d store the global values in a config file somewhere and write a couple of Quick services to get/set them.

Sonam - Regarding the Clustering Docs:

The section I was referring to is Appendix A of the document page 78. There they express concerns over “using the repository for large volumes of data storage or permanently archiving such records”.

It sounds a lot less negative now as I re-read it.

I guess my impressions have been formed from having to rebuild the repository a few times due to corruption.

It still leaves me with the sense that the repository is not intended for “programmer” use.

No doubt, the pub.storage flows will do the job - and they did put the flows in the WmPublic package

Thanks Haithem! I’m a bit scared of using the repository myself.

You might know this already, but in case you have TN, its a good idea to run wm.tn.enumerate:deleteQueryResults as a weekly scheduled service so that TN’s query cache is deleted from the repository.

What does the WM Integration Server 4.6 Repository actually do. Does it save operational data? Is it recommended to use it in a production enviroment, even if I am not going to use clustering?
It would be great if somebody could advise me on production guidelines, especially with regard to the above mentioned question.
Thanks
DJ

DJ.
The repository allows you to store variables global to all applications/services running on the server. If you have many process or threads that need common information but you don’t want to have to hard code these variables into each service, then this is your answer.

You can create your own repositories to store different types of information for different applications. You can lock and release variables if they get updated when your services run, so multiple services can have access to the variables without contingency issues.

Is it stable? Very, I have several processes performing infinite loops and checking for files across the United states. This was to replace another product. Its controlled via a web interface. Each process gets its configuration information from a database, but there is some common stuff that i lookup in the repository.

So if you don’t want to use a DB for a repository the native one in webMethods will work fine for small amounts of lookup/configuration information.

Thanks Tim,
So the repository won’t help with the tuning of the integration server? In other words help services execute quicker or more robust? It just stores information that you might use in services in a global sense? You don’t have to reply if I am correct in these statements.
Thanks again

I am trying to implement a global Collection of key-value pairs that will be used by several of my mapping (flow) services as a lookup for data transformation. I would like to initialize this object from the database at server startup. I will not need to modify any of these values within webMethods. The idea is to save the database read time, considering that this data will remain relatively static.

After reading this thread I see that some of you have advised against the use of the repository for similar things. In any case the repository will not serve my purpose since it locks an element when you “get” it.

Will Kriski has also talked about “a java service to make a static variable which will be available across all flows” above.

Could someone “please” suggest a solution to this requirement.

Thanks.

We have a start up java service that takes our global variables from a table and caches them when IS comes up. Another utility service gets them from memory for the flow. If the table values change you have to remember to reload the cache with the updated data. We have also used an XML document stored in the package ns in a similar manner.

I’d suggest a Java Service that stores a list of key-value pairs in a static variable.

The way that the java services work within webMethods is that all java services in a given folder belong to the same underlying java class with each java service in the folder being a method in the class.

A static variable (I don’t want to use the word global would be defined in the shared section of the java service and would be accessable by all java services (methods) in the folder.

You could then develop an init service to initialise the list from the db and maybe getter and setter services that allow you to access the key-value pair list from flow services. (e.g. A service called getKeyValue that has input of “key” and output of “value”) The service would be invoked in flow when you need to get a key value from the list.

From your first question, this is not global variable - it a class variable that would be accessable by calling the appropriate service (method) through flow but we don’t need to go into the debate around global varibles do we You don’t really want to expose the list to every service in webMethods, you should try to limit the interface to the list using a java service such as getKeyValue as suggested above… but these are all design decissions that you will need to consider.

If you havn’t used java services before, I would recommend a bit reading first… The webMethods doco has some coverage for this. Developed and tested properly, java services can be very effective.

Hi,
Boy this is an old topic. The native repository for variable storage does not get locked upon reading. You only have to lock it for updating… Similar to ASP application area varaibles. For Reading there is no locking involved and they are available globaly. This is what I used. And it worked fantasticly. Repeatedly and under high volume for simple global variable access.

I think someone got confused reading this post. Implementing a shared Java service was much more confusing and far more complicated to maintain with the development utility for a normal WM developer that isn’t a java expert.

Someone had mentioned a potential for corruption. Just hasn’t happened during our usage at all. never… And we have 6 or 7 processes accessing these variables repeated hundreds maybe thousands of times a day.

Create a unique store and load your value pairs. Its easy and works great.

Hope this helps.

Will,

I do it normally using properties file and caching the results. Altho not experienced personally, I had heard about repository corruption a lot.

Thanks Monica, Cameron, Tim and Thahir. I was really in a fix about how to implement this lookup cache. Your valuable advice is highly appreciated. I am currently implementing Cameron’s solution and will post my experience a little later. Thanks again!

Cameron, when you referred to “webMethods doco” for java services, were you referring to their java API. Coz thats the only thing I found. Could you please point me to the right documentation.

Thanks again guys!

Will,

Start with the webMethods Developers guide - the chapter on developing coded services will give you an overview on how java services and the underlying java classes are maintained in IS.

After that, refer to the API for getting data in and out of the pipeline (IData) and utilising the webMethods IS API.

Some samples in the next post.

-Cameron

PS: My suggested solution in the earlier post doesn’t necessarily require you to access the key-value pairs from a db. Monica’s solution, or even a properties file would work well also.

Putting vales into the pipeline

// Get the pipeline cursor
IDataCursor pipelineCursor = pipeline.getCursor();

// Add the monitor object to the piepline
// - assuming pipeline variable is called myVar
pipelineCursor.insertAfter(“myVar”, “Hello World”);

// Destroy the pipeline cursor
pipelineCursor.destroy();

There are also other ways of inserting values into the pipeline in different orders (other than insertAfter which inserts at the end of the pipeline). Have a look through the API doco.

Also, remember that with Java services, you can import any packages or classes that you could in a normal java program, so you have access to all the standard java functionality such as properties files, date formatting, string formatting etc. etc.

-Cameron

[ Wow, this is an old thread - my last post in it is more than 1 yr back ]

I just wanted to pipe up to say I’ve had good experiences using the repository for global variables in one instance where the data changed infrequently, but was read often. The main advantage is this code is safe if it is moved from a standalone to a clustered environment – all servers look at the same repository.

I also set the “getter service” as a cached service. Due to how WM handles caching (IS caches pipeline outputs, not just declared service outputs) there is a one big gotcha with caching. I’ve documented this another post, but to summarize: one needs to call clearPipeline as the first step any cached service and remove everthing but the declared inputs of the cached service.

Cameron, thanks for all the information. That helped a lot. I am still coding away. Having some problems debugging the code. Trying to manage using print statements to the server log.

Sonam, thanks for your response. I see a few people on this thread that have successfully used the wm repository. Not really sure if one is a better option over the other.

Will,

If you’ve had any experience with log4j, I find it a useful way to add debug statements to your codewhen developing java services.

If you havn’t previously used log4j it may burn some time working it out but the effort is well worth it… and you’ll use it again and again Refer to the logging.zip software upload that was posted by Mark Carlson for log4j implementation.

-Cameron

Will,
While Camerons solution will work you will ultimately be storing global variables that you will continue to have to load into the pipeline using the java code.

Maybe I don’t understand the scope entirely. Because the store repository is very very very easy and we have found it to be bullet proof for over 1.5 years now across thousands of transactions a day.

Here are the steps.
1. You create a unique store using the developer tool which really creates a unique XML file on the server. That’s right a store is an xml file that WM creates.
2. You add variables and values to the store as well as their values.
3. Any flow needing access to the store just needs to access the store name and variable name at the appropriate times in your flow to have access to the values.
4. There is no contention for read access across multiple flows.
5. If it has to updated frequently just make sure your other services are built to wait long enough for the write locks and unlocks to release from the other services that perform updates. Yes anytime you update a variable it has to be locked just like a database. WM gives you the ability to put timeouts every where… Whether its on updating the variable or waiting for access to a variable that’s being written to. Its lightning fast as well.

 While there is nothing wrong with going through java to integrate with the pipeline.  The bad thing about that things get harder to debug inside the developer because you mix java with the design elements of WM.   
While this is necessary many times when doing things that WM just doesn't do natively yet.  I think this is something so simple that WM was already built to do so why not use it.

Tim,

Just to clarify (and to avoid confusing Will), the approach to getting a value for a key into the flow service should be similar whether it is done through a java service through my approach or through storing the key-value pairs in the repo. Essentially, there would be a service for either of these approaches that implements the getValue functionality.

ie.
For the java approach there would be a service called say “getKeyValue” that has an input of “keyName” and returns “keyValue”. This java service would go to the key-value pairs list and pull out the appropriate value for the given key. The java service would then be called in flow (as a normal service) everytime you need the value for a certain key.

For the repo approach there would be a service called say again “getKeyValue” that has an input of “keyName” and returns “keyValue”. This would be a flow service that uses the inbuilt repo services to retrieve the data from the repo. The service would then be called in flow (as a normal service) everytime you need the value for a certain key.

I agree both approaches would work, it just depends on the implementation. As per a previous post, the shortcoming I see with the java approach is that in a clustered environment, there is an issue if the key-values need to be shared.

-Cam

Cam,
Yes I agree, however the services for getting key values out of the repository are already there built into the product require no coding.

There are data store components that fit right into the flows to retrieve and set values from the data store/XML file.

From my perspective there are several things missing from the WM tool that require java, but this area is not one of them. I think I had to read like 3 pages of documentation to figure out how to use the whole datastore thing. Its very easy. However I think many people get confused with the backend db repository for advanced transaction management. That is an entirely different thing and much more complex.

It takes 5 minutes to create a new data store for global variable sharing, add a variable, set the value, retrieve and so forth. The 2nd time you do it… It will only take a minute. There’s no configuration changes nothing. Best of all its what they designed it for.

Well to each his own.  I just thing the trick with the WM tool is only using jave in the developer tool when you have too.  It minimizes flow confusion and reduces complexity when you can use the native components as much as possible.  Putting to much java into a WM flow when you don't have to is bypassing the good parts of the tool.  Automatic documentation of development projects.  Easy to follow schemes etc...

am using the repository storage facility to application variables e.g. email addresses of users who should be notified when an unexpected error occurs in any application.

Each application has it’s own store with a key of ErrorDistributionList.

The initial set up of these store values is simply achieved by using a service that runs on startup to register the store and add the key values needed.
I am trying to create a dsp application that allows these stores to be maintained.

I present a list of available stores, I then select the required store, I then use getStoreKeys to display the available keys for that store.
I then display the key value pair and allow the value to be maintained.
I have been unable to get my processing service to update the key value with the value input on the dsp page.

Does anyone know a way of dynamically amending the value held within the store.

To set up a global variable -
I think you should get the PSUtil package from the webMethods Advantage web site and look at how they set up the getProperties service. The code is exposed in this package. This is the one of the ways I know for setting up global variables.

Another way is to open the Server admin screen and under setting, select extended. There you can enter a key and value. The value is stored in the server config directory in the server file. There are many reserved names in this file but you can also make up your own field names.

When you add an extended setting your new property key should start with “watt.”.

Properties that you add to the extended settings list are accessible using the System.getProperty(“propertyName”) method.

I wrote a simple java service that accepted a key name and returned the key value so that I could retrieve System properties inside Flow services.

It accepts the String “propertyKey” as input and returns a String “propertyValue” with the value of that property (or null if it can’t be found).

Hope that helps,
Mark