webMethods

Is there a ‘smart’ way to export data out from Trading Networks (TN)? Specifically, I want to export data on outbound TN document deliveries.

Let’s say TN tries to deliver an invoice to Ariba, but Ariba returns an error message during delivery - (say ‘unit of measure mismatch’, or ‘PO not found for this invoice’). This delivery error is actually important business information but it is stored only in the Trading Networks database and can only be queried via MWS. We’d like to import this info into a Business Intelligence system, so Accounts Receivable staff can inspect reasons why an invoice is unpaid.

I’ve prototyped custom code that uses TN document query and delivery job services from the WmTN package. But this is awkward since it has to run in batches and the batches need to be synchronised.

I was wondering - is there a ‘baked in’ way to access this data and export it from Trading Networks. Surely other customers have had to do this in the past!

I’ve read about ‘Digital Event Services’ which sounds promising (see blurb below from its documentation). But I have no idea about this functionality - can it be used to export document data out of Trading Networks?

Digital Event Services (DES) enables other Software AG products to intercommunicate
by exchanging digital events. Digital events are typed and serialized data structures
that are used to convey or record information about the execution of a runtime. This
information can be application information, such as the state of a business process step,
including any associated business data, or it can be system information, for example,
how much memory and how many threads an application is using.

Interestingly, we recently had that exact requirement: loading Ariba errors into our data warehouse for reporting.
We used the wm.tn.query services. It is a batch process as you noted, but it can be run fairly often, which was good enough for our stakeholders.

TN does have run-time events

https://documentation.softwareag.com/webmethods/trading_networks/otn10-15/webhelp/otn-webhelp/index.html#page/otn-webhelp%2Fto-admin_concepts_37.html

The “Document Processing Complete” may be what you are looking for

https://documentation.softwareag.com/webmethods/trading_networks/otn10-15/webhelp/otn-webhelp/index.html#page/otn-webhelp%2Fto-managing_events_3.html

Good to hear from you Dave! Yes, I figured I wasn’t alone – companies like being paid on time

Thanks – the TN run-time events seems to be exactly what I was looking for. I poked around today and could get IS to emit delivery events using the sequence of actions. However, I haven’t yet figured out how to ‘subscribe’ to events yet. Anyone have any have any pointers on what I’ve done so far, and on subscribing to events?

(Note: I’m using version 10.1 on extended support, so the document links below are to 10.1)

• TN Administrators Guide – Page 291 states that the tn.eda.events.deliveryTasks TN configuration property must be set to true. So I set it up in MWS. However, I could find no trace of any event being published to UM in Enterprise Manager
  
  

• The Event Routing PDF is very complex, but on Page 27 it states the event routing can route events to UM. The setup requires reconfiguring the ‘Provider URL’ Command Central.
  

  https://higherlogicdownload.s3.amazonaws.com/IMWUC/webMethods/Files3/f00202441ad9ce23ffc41e44daadadb3de852e3f_2_604x489.png 1.5x, https://higherlogicdownload.s3.amazonaws.com/IMWUC/webMethods/Files/f00202441ad9ce23ffc41e44daadadb3de852e3f.png 2x" data-dominant-color="EFEFEB">

  image001.png672×544 104 KB

• So I ensured SPM was running on the B2B IS instance and the instance defined in CC, then reconfigured the B2B instance’s ‘Provider URL’.
  (Interestingly, this screen does not permit certificate authentication to UM – only username and password.)
  

  

  image004.png713×405 22.9 KB

• This led to 7 new event types suddenly appearing in UM as shown in Enterprise Manager below.
  

• Events were being published to these queues. I haven’t yet figured out how to subscribe to these events. So I used Enterprise Manager to snooping on the ‘ImmediateDeliveryComplete’ queue. This snoop data seemed tab-delimited, so I copy-pasted into Excel for legibility. It seems the events contain the TN doc ID and task ID of the delivery task, and also the deivery status and status message. The status message is important because that’s where Ariba or VAN detail error messages appear.
  

  

  image003.png334×912 16.2 KB

• I think I’ll need to subscribe to this queues, then use wm.tn.doc:viewAs to obtain the original invoice from the internal TN document ID. I must then push the invoice ID, status and status message to the BI system.

• Anyone have pointers on subscribing to these events? I can see this event type being returned during snooping –
  $Event$Type={Software AG Product Documentation}ImmediateDeliveryComplete
  

  https://higherlogicdownload.s3.amazonaws.com/IMWUC/webMethods/Files3/af144a109aac568083036c4eae98346f987451b0_2_1035x268.png 1.5x, https://higherlogicdownload.s3.amazonaws.com/IMWUC/webMethods/Files/af144a109aac568083036c4eae98346f987451b0.png 2x" data-dominant-color="E9ECEE">

  image007.png1112×290 22.6 KB

I’m not so sure this is a good idea in general. In 10.15, do TN EDA events still work the same way?

I think the approach is neat in that it’s event driven. But what if events are lost? Is there a way to regenerate events. Probably not, heh? With this approach, we just need to make the durable reliable.

Nice progress! Looks promising

No mention of this in the deprecation notes in any version up to 10.15. TN is a stable product at this point

I think the question for Software Ag is if this is a supported pattern in general? I believe these events are primarily for the built in TN dashboards and charts

Can and should customers create their own subscribers for these events?

Thanks Dave.

I think TN EDA is supported for other uses (other than charts) because it’s pretty old architecture. TN EDA is available in 10.1 (GA circa 2018) and probably prior versions, when TN charting didn’t exist - not to my knowledge anyway.

The problem with EDA events is TN only generate them for transactions created after you enable events. And they can be published one-and-only-once.

But the batch export approach you took can look back at historical documents in the TN database, can ‘replay’ data exports, etc.

Hi @Dave_Laycock1 (and anyone else who can help) -
Sorry to disturb you again. I tossed up creating a new thread and decided to continue this one…

I got around to coding the TN document/delivery export process using this code pattern. Have you used this code pattern before in your TN data exports?

INVOKE wm.tn.query:createDocumentQuery
INVOKE wm.tn.query:documentQuery (Returns enumeration identifier to access paged query results)
SET N=1
REPEAT until 'enumerate:nth' call below returns no results
INVOKE wm.tn.enumerate:nth (Read page N)
SET N=N+1

Unfortunately, with large datasets I’m getting inconsistent result sets in consecutive runs. I suspect the wm.tn.enumerate:nth call may be outpacing the results returned by the server-side query thread. But I am not seeing error messages such as “Requested page is not available” or “Invalid or missing page number” I’d expect if this were the case. TN is silent. So I’m not sure how to handle this: inserting delays makes it worse and the WmTN documentation is of no help. I’ve increased the tn.query.maxrows parameter to tn.query.maxrows=500000 to ensure TN doesn’t truncate results. It’s a situation.

I suspect my code may need this additional logic :

• loop over the current page (LOOP over wm.tn.enumerate:nth with pageNum=N), and delay reading the N+1^th page until the ‘enumerate:nth’ call signals it is available (N < pagesRead)
• only exit the loop if it is already at the last page (N = totalPages)

But it’s hard working with large datasets and TN policy of silence on any errors. So I’m asking in case you or anyone has experience or ideas on using wm.tn.enumerate:nth

PS: I’ve updated the code and still hit the same problem. I’ve posted this now in a new thread: