Little background story: we have a processing service that sometimes will hang (network connectivity pb). This cause the IS trigger to ‘wait’ and not consume any other document (until a restart)

One workaround was to increase the number of threads so even if one ‘hangs’ the others will keep processing documents.
This is of course not a final solution.

But in the meantime we want to be able to identify the problem. One easy way to monitor our triggers with the Broker was to check the number of documents in the queue.

But I cannot find a way to monitor UM to know the number of documents not consumed (we know they are pilling up somewhere, because when we restart the trigger, they all get processed).

Any advice on how we can do that ? can do it by physically monitoring UM’s files ?

any advice will help,

Thanks !

Jonathan,

we introduced that capability in UM 9.9. In Enterprise Manager, you can click on a channel (linked to an IS doc type or JMS topic), then go to Named Objects and see the number of outstanding messages for that trigger.
The same information is also accessible using the UM Admin API.

In UM 9.12 we made it even easier, with visibility in Command Central. In Command Central there is now a single table view of all durable subscribers where you can see the number of outstanding messages for all triggers in one place. You can even sort the table to bring any stuck triggers to the top.
This information can also be accessed using the Command Central CLI or REST API.

For earlier versions, the only way to figure out the backlog is indirectly by looking at the last Event ID on the durable subscriber and comparing it with the last event ID on the channel. The bigger the gap, the bigger the backlog.

Hope this helps.

Thanks for the reply!
As we are on 9.8, I’ll look into the lastEvenID checks.
It is sad it was sold to us as ‘a better broker’. Version 9.8 is not.

Thanks again

Jonathan, do you know the impact of having UM version 9.12 installed in a 9.8 environment ? (all other products version 9.8 )

Thanks

Jonathan,
first, you would need to install UM 9.12 in a different installation directory (or different machine) from the 9.8 products.
You would also need to manually copy the UM client jars (nClient.jar, nJMS.jar, nAdminAPI.jar) from common/lib in the UM 9.12 installation to common/lib in the 9.8 installation. You will also need to be careful to NOT accidentally install an UM 9.8 client fixes in your 9.8 installations, as they would overwrite the manually copied jars.
This combination should work fine, but is not officially tested or supported and it will only work for IS connecting to UM. We do support the combination of IS 9.8 with UM 9.9, which might help.

Thanks again for the reply,

regarding the workaround you mentionned ; I managed to find the channel’ last Event ID, but i cannot find the durable subscriber’s one.
As we come from the ‘broker’ we do not use JMS triggers, but webm triggers.
Is it possible to do with the native webm triggers ?

Thanks

Hi Jonathan,

Thanks for the info on how to view the queue size for Topics.

Do you know how I can do the same thing for a Queue? Given there is no durable subscriber, I cannot use the same technique.

This is for v9.9

Can Command Central do this?

Thanks,

Steve

For queues it is a lot easier. In Enterprise Manager you can simply see the queue stats which show the number of events. Look on the Status tab at the Events number at the bottom-right in the Event Store box.
This is available in all UM versions.

We have a requirement where the messages are sent to the Topics in UM but there are no consumers defined. So in such cases can some one please help us to have the topic flushed without messages getting piled up. Is there a concept of volatile documents in UM as in broker. Thank you.

Suneetha,
if you have a Topic (= UM channel) that is defined with the JMS engine (which is default if you create the topic from the JNDI tab), then you won’t get messages piling up if there are no subscribers defined. If your subscribers use durable subscriptions (DS), then messages will indeed pile up while the subscribers are offline. If you know such a subscriber is not coming back, then you can remove the durable subscription using Enterprise Manager by going to the Named Objects tab for the channel, clicking Get Names, selecting the DS you want to remove and then clicking Delete.
All of this has nothing to do with volatile messages. UM does have such a concept (called Non Persistent), but that does not prevent things piling up if you have things configured wrong. The only difference is that non-persistent messages pile up in memory and are lost when UM is restarted, while persistent messages pile up on disk and do survive a restart.

Hope this helps.

Thanks Heywood , providing your inputs…

i have one concerns on topic.

If TTL or capacity is set - the messages get deleted once the limit is reached - irrespective of whether the messages are consumed or not.

I would like to keep the messages in queue until they are consumed and delete once they are acknowledged.

If you set TTL and capacity to zero, then both are effectively unlimited and messages won’t be removed until consumed or explicitly purged.
Note that with topics, you need to use Durable Subscribers to ensure messages continue to be queued, even while the consumeri s disconnected.

Thank Jonathan for your info…

It is regarding Non Durable Subscriber, I want to know whether messages will be removed automatically from topic if no consumer is connected.

Please confirm my understanding, If we use Non Durable Subscriber then we cannot re publish the messages whenever we required (In topic if we select channel type as Mixed) and if we restart UM then we may loose messages from topic.

Also, Do we get any acknowledgement when data is consumed or Not consumed and where we can see these details.

If you have a non-durable subscriber and you use the JMS engiine (property of the UM channel), then any messages published while the subscriber is disconnected will be discarded. If you want to ensure they are stored in UM and delivered when the subscriber comes back online, then you should use a Durable Subscriber. That is what Durable Subscribers are designed for.
If you use the Default engine for the channel, then messages will be stored on the channel until purged, either explicitly or through TTL or capacity settings. This is irrespective of subscribers having received the messages.
Whether messages stored on the UM server survive a server restart, is determined by the channel type. If the channel is Persistent, or it is Mixed and messages are sent as Persistent, then messages are stored on disk and will still be there after a server restart.

From your questions, it sounds like you should be using Durable Subscribers in combination with persistent messages over a Mixed channel. That will ensure that messages are stored while a subscriber is offline and will be guaranteed to reach the subscriber eventually, even after UM server restart. This is a very common pattern and doesn’t require extra coding, resubmission, ack checking, etc.

Thank you very much your info…

we were using Non-Durable subscribes in combination with Mixes channel. would you like let us know, what happened in this scenario.

Also how to get any acknowledgement. when data is consumed or Not consumed and where we can see these details in MWS or UM logs.

If you use non-druable subscribers with a mixed channel, then the behavior is as follows:

• If you have JMS engine enabled, then messages are passed to active subscribers when they are published. If a subscriber is offline when a message is published, then the message is discarded and there is no record of it having existed.
• If you use the Default engine, then messages are passed to active subscribers and are stored on the channel. If an inactive subscriber comes online then they will get ALL the messages on the channel, including those published in the past. Messages must be explicitly purged, or set to purge using TTL or capacity settings.

As you are using a Mixed channel, the way messages are stored depends on the DeliveryMode header property. If you set this to PERSISTENT (default for pub.jms:send from IS), then they are stored on disk and will survive a restart. Otherwise they are stored in memory and will be lost if UM is restarted.

Generally you cannot see anywhere when messages have been consumed if using non-durable subscribers. If you use durable subscribers, then the more recent versions of UM will allow you to see exactly how many messages are waiting for each subscriber.

Is there a reason you are not using Durable Subscribers?

Thanks Jonathan for explain durable and non-durable step by step info…

We were using 5 subscribes and 4 consumers message are subscribe and 1 consumer is not yet subscribe. in this scenario how to identified which one is not yet consume to subscribe whether will check for logs
which one not yet subscribe…

Could you please explain above scenario…

I’m sorry but I don’t understand your question.
When you say you have 5 subscribers and 1 is not yet subscribed, what do you mean? Do you mean that the 5th subscriber is not yet running?

Hi Jonathan,

Thanks for the response.

Suppose We have 5 Subscribers on Target Side. We have published the data. Unfortunately one Subscriber is down. So 4 Subscribers have subscribed the data. How we would know which subscriber has not received the data. Do we get any acknowledgement from UM regarding which subscriber has subscribed the data.

If you use non-durable subscribers, then you cannot find out which subscribers did and which didn’t receive the message. UM does not keep that information.
As mentioned before, in your use-case you really should be using durable subscribers. They will ensure that all subscribers eventually receive every message, regardless of whether they are down when the message is published.
Is there a reason that you don’t want to use durable subscribers?

Thank you very much…

we are using Non Durable because there is no consumer present at this moment. Consumers will be added later on…

This is the reason we want to select as Non Durable… So that messages can be lost after placing topic.
Whenever we add new consumer then messages can be consumed (While adding consumer will select Durable Subscriber option).
If we use Non Durable then we will not get to know which subscriber received and which one not picked.
How we can verify same when we use Durable Subscriber need to know.

I’m sorry but I am still having difficulty understanding your question.
You say you start by creating a topic and publishing messages. You say it is OK if those messages are discarded as long as there are no active consumers.
When you first add a consumer, it will be a durable subscriber. From that point onwards, that consumer will receive messages, including those published while it is offline.
So, let’s take an example:
On Monday, you create the topic and publish 100 messages (ID 1-100). There are no consumers so the messages are discarded by UM.
On Tuesday, you add your first two consumers (C1 and C2). You then publish 100 more messages (ID 101-200). Those will be delivered to C1 and C2. If C1 goes offline after message ID 150 is published, then messages 151-200 will be queued for the C1. Let’s say C1 is still offline at the end of Tuesday.
So, what information is available to you at the end of Tuesday?

• You will be able to see that 200 messages have been published on the topic.
• You will be able to see that you have two durable subscribers (C1 and C2) for your topic.
• You will be able to see that C2 is connected and C1 is not.
• You will be able to see that the last message delivered to C1 is ID 150 and the last message delivered to C2 is ID 200.
• You will be able to see that 50 messages are queued and waiting to be delivered to C1.
• You will NOT be able to see exactly what the first message was that C1 and C2 received (ID 101).
• You will NOT be able to see exactly how many messages C1 and C2 have received to date.

Does this clarify things?

Thank you very much…

i have one more concern for GIMA format,

we were using GIMA format in our project.
GIMA format example sample data in below,

<sActMsgG3:ActorReadRequest xsi:type=‘sActG3:ActorReadRequestMessage’ >



a763bc50.09f55fd2.911c5b41




TCAS11001_PersonEntityAssociation

Etc…

problem is we did parsing in webMethods. because above structure depending reference document.

We are working on parsing/validating an XML file that is specific to our partner. It has SOAP data in the XML file. We are trying to create a document type for the XML file but not able to get the complete structure. Please help in getting the document type created for this XML.

This new question is unrelated to UM.
Can you please repost it as a new thread in an Integration Server forum?

Hi Jonathan,

I Hope you are doing Good.

I need your help one more time.

We are using Durable subscribers and Type is Persistent, If Subscribe not yet available Data not yet store event and i sea event showing 0 values. if i am snoop was start and will start publish data. it;s stored in Snoop once subscribe will start and try to re-publish it’s reached to Subscribe.

My requirement was, want store data in Event and if subscribe will start will automatically subscribe all data store in UM.

What are steps are will followed for above scenario will do…

Please help here…

Usually a durable subscriber is created the first time a subscribing client connects, and it remains in place after that. So any events published before a subscriber connects for the first time will not be available. Is this the scenario you are referring to?

If you want any new subscribers to receive messages that were published before the subscriber came online, then you can achieve this by disabling JMS Engine on the channel. But if you do this, then all events will be stored indefinitely and you will have to take steps to purge them at some point. You can do this by defining a time-to-live (TTL) on the channel or individual events to ensure that UM removes them when they reah a certain age.

You say you see the event count as zero. Where precisely do you see that? On the channel or on the durable subscriber (named object)?

Also, what version of UM are you using?

Thank you Jonathan,

it’s working fine…

Hi Jonathan,

Scenario 1:- If subscribe is down after subscribe UP.
Subscribe is down after UP. i am trying to re-publishable in UM subscribe. it’s showing event count increase. how did know data sent to subscribe

Scenario 2:- already Subscribe UP
i am trying to publishable. it’s showing event count increase

My question is:-

which data send to Subscribe or not, how did know from UM?

Hi Jonathan,

We need one more help…

Subscribe was down, when subscribe is UP all messages are send to Subscribe. what are the steps are followed

We had used TOPIC Durable
Type: Persistent

Please help here…

Hi Varadharajulareddy ,

If you are using topic with durable subscriber , and the subscriber is down when the messages were published. Once the subscriber comes up it will subscribe all the messages.

Are you looking for any specific steps to be followed , can you please describe what steps do you need. It is not clear from your post.

Hi Jonathan,

I need to roll back the message to queue if there is any error (transient) while processing the trigger service. Also I need to suspend the JMS trigger.

I have set the property suspend on transient error. I want to know how trigger will get to know in case of transient error. How to implement this. Can anyone please help on this.

Vara,
IS services invoked bya trigger are generally smart enough to distinguish between transient and permanent errors. For example if you call an adapter service that fails with a connection error, then that is treated as transient (ISRuntimeException), where as a duplicate unique key error, or SQL syntax error would be treated as permanent (ServiceException).
If you want to control it yourself, then you can call pub.flow:throwExceptionForRetry in your code (possibly in a try-catch block), to tell the calling trigger that this is a transient exception.

Thanks jonathan,
which you mentioned all steps we had followed. But we have used Http connectivity for this steps are followed. i didn’t see JMS re-trigger the Subscription Topic messages

PFA…

Are you saying that you make an outbound HTTP call in the service tha is invoked by the trigger?
And that the HTTP call fails?
Are you using pub.client:http?
If so, what sort of exception does it throw?
It is possible that it always throws a ServiceException, which will not cause the trigger to suspend.
You can put a try-catch block around the pub.client:http call and in the catch block invoke pub.flow:throwExceptionForRetry.
That will flag this as a transient error, which would cause the trgger to suspend (if you have configured it that way)

Thanks jonthan,
Yes, we have make an outbound HTTP call in the service that is invoked by the trigger and HTTP call failed like as Service unable , service down etc…

Please find the attached trigger monitor and subscribe flow in try.catch block.

Background story:-

we had used UM combination with type Persistent, if any time Subscribe was down. after some time subscribe will come online(server up). all storage topics will automatically send to Subscribe.

That only we had implemented transient error and re-trigger to JMS.

Please explain in this scenario how to sent to subscribe server

Hi Jonathan,

I believed it. i heard by using Universal message (version 9.9) , while publishable and subscribe scenario, for example subscribe was down. while data message was publishable and store data from Topic(topic should be durable and combination of type is persistent). subscribe comes online. whatever data store in topic, automatically sent to subscribe. Is it correct or i am not??

If correct, we have followed same scenario, it’s not yet sent to subscribe server.

if any wrong to my end… please explain the scenario, which scenario sent the all messages to subscribe server automatically.

we were using below,

UM version :9.9
Topic : Durable subscribe
subscribe server http

Please let us know if you any requested information.

What you are describing is basic JMS and is fully supported by UM. Here’s the sequence:

• Create a topic in Enterprise Manager using the JNDI tab. This will create the JNDI entry and the channel. The channel will be of type Mixed, which is correct.
• Create a connection factory in Enterprise Manager using the JNDI tab. Enter the URL of the UM server in the CF definition. I recommend setting Shared Durable to True.
• In your subscriber, establish a durable subscription to the topic you previously created. If you are using Integration Server as the client, then set your JMS trigger to Topic (Durable) and enter a name for the durable subscriber in the trigger.
• Publish some messages to the topic and verify that they are received by the subscriber.
• Stop the subscriber. (Note: the durable subscriber will still be present on the UM server, visible in the Named Object tab under the channel).
• Publish some more messages
• Restart the subscriber and verify that the messages that you published while the subscriber was down, are received correctly.

As I said, this is a very basic, standard JMS use-case and works exactly as I describe here.

I hope this helps.

Thank you for step by step explained Jonathan,

It’s published all are message while subscribe was down, after that all publishable message while subscribe serve come online. is it automatically sent to subscribe server for all are messages ???

Thank you for step by step explained Jonathan,

It’s published all are message while subscribe was down, after that all publishable message while subscribe serve come online. is it automatically sent to subscribe server for all are messages ???

or any need implementation transient error code…

Please help here…

I don’t understand your question fully.
UM will automatically send all messages to the subscriber that were published while it was down - there’s nothing special about that, no special code needed.

Thanks you Jonathan for providing your inputs…

Now the subscribe was down from Target system. source system keep on sending publishable message, whatever publishable message are stored in Topic as well as event count also increased. if any possible to see event count data store in physical system (local system) from UM server.

we need send mail to source system to every half hour, how many events are available while subscribe was down.

if yes, could you please let us UM server event store path…

Thanks you Jonathan for providing your inputs…

Now the subscribe was down from Target system. source system keep on sending publishable message, whatever publishable message are stored in Topic as well as event count also increased. if any possible to see event count data store in physical system (local system) from UM server.

we need send mail to source system to every half hour, how many events are available while subscribe was down.

if yes, could you please let us UM server event store path…

Please help here…

You cannot determine the number of pending events by looking at the file system. Events consume space in files even after they have been subscribers and that space may be recovered later.
If you want to monitor it automatically, I recommend you use the Command Central command-line tools or REST API to retrieve the number of waiting events. You could do this from a script or program to then send an email.
Alternatively write a Java program using the UM admin API to do the same thing.

Helllo,

Can you anyone explain below.

ideally on broker we can see how many messages are queued up on the trigger client for oth SERIAL and concurrent trigger.

Is there a way we can identify how many messages are queued up on serial trigger client.

For concurrent trigger im able to see how many messages are queued up and yet to be processed using Outstanding Events tab

Thanks
Arun Cholleti

Hi Jonathan,
we are trying to get the Topic Durable subscribers messages count using Command central REST API which one you have suggested .
but looking that WADL , there is no URI available
Could you please share the relevant methods in that REST API?

The REST API is not explicitly documented, however you can get the resource URI, request/response and HTTP Header details by executing the Command Central commands via Command Central Command Line tool. For more details please refer to

https://techcommunity.softwareag.com/ecosystem/documentation/webmethods/command_central/cce10-3/10-3_Command_Central_webhelp/index.html#page/command-central-webhelp%2Fcc_rest_api_resources.html%23

Universal Messaging’s Command Central commands are documented in “Using Command Central to Manage Universal
Messaging” section of the “Universal Messaging Administration Guide” or
https://techcommunity.softwareag.com/ecosystem/documentation/onlinehelp/Rohan/num10-3/10-3_UM_webhelp/index.html#page/um-webhelp%2Fto-header_using_command_central.html%23