I am trying to configure EDI batch feature in 6.0.1.
read the pdf and here are steps that I tried

1. create a public queue. deliveryserice =EDI batch, set input. schedule the service
2. create a TPA, with sender, receiver and ArgeementID= X12 Envelope. set the inputs for TPA record.
3. create a processing rule with Deliver Document by = Scheduled Delivery = public queue.

Now I submit few EDI document, processing status is set to QUEUED.
That is it, I do see the batched EDI, scheduler is involved but the EDI documents are not batched.

What am I missing here ?
Please help

I found some shortcomings in using the EDI batch feature. If this doesn’t relate to your circumstances ignore this posting. I went through a lot of pains trying to use the EDI batch feature just to realize it doesn’t meet my organization’s requirements and created my own using public queues instead.

The biggest road block I found in using EDI batch is I could not control the TPA. The TPA in EDI batch is hard coded to the default EDITPA and you cannot override it. We only use a unique TPA when necessary, but for one trading partner we needed to extend the TPA to control the receiver id at all levels since it was different from the GS to the ISA and also different to the AS2. We also had to control a different version in a non-standard way from the GS to the ISA (directive from management to have no impact to the customer- do what ever they want). Because of this the EDI batch feature was not fully functional for our circumstances. If that is your case I will give you a run down of how I solved this.

One other detail too is that when we were requested (last minute scope change 10 days prior to go live after 1.5 months of testing) to batch this new trading partner’s EDI documents we hit a huge performance issue on the run time because the batch EDI document became so big when batching 1900 documents into one. Thru monitoring the thru put of the service I saw a huge hit on performance after I believe 300 documents had been processed and it got worse as it ran. At first before doing anything it took 2 hours 45 minutes processing time. After some tweaking it took total runtime (including delays with queue initiations 3 of them) I got the total runtime for 1999 documents down to 25 minutes. This still was not great but I only had about 3 days to have a complete solution in place.

Here’s a quick over view. I have 3 public queues in use. The first is the queue for my canonicals. I had to ensure that this batched EDI file was sent only once at the end of the day so I batch my canonicals upon receipt into TN by delivering to the first queue that is sheduled to run every day at a specified time.

The first queue drains at the scheduled delivery time and thru nested repeates which creates batches of 100 EDI documents enveloped to the group (GS level) with an attribute of EDI batch with the value of group. After updating the attribute I route the bizdoc to TN enveloped at the GS level.

In TN I have a processing rule setup at the X12 group envelope level with an attribute of EDI batch equal to group and the action is to deliver to my second queue for batching the 100 EDI documents enveloped at the group level. This second queue I run once at the setup of the queue and do not have setup for scheduled delivery. It is just a queue to hold all the document in batches of 100’s. I determined the 100 document batching level by doing several identical runs with the repeat parameters set at varying levels to determine the optimal performance.

So my first queue’s scheduled service continues processing all the batched canonicals and mapping to the X12 format, adding group envelope, updating the EDI batch attribute, and routing to TN until all the canonicals are processed. After it completes all the canonical processing it submits the service that drains the second queue that contains all the batched group envelopes.

This triggered service then drains all the group enveloped 100 batched EDI documents and appends the contents to create the single batched EDI string. I get the TPA and profile information only once for the entire batch which saves a lot of the run time on 1900 documents. After all the documents are combined into one I re-envelope and receive it into TN.

In TN I have a processing rule that recognizes the ISA envelope for that TP and triggers a se

Thanks Dawn for sharing your services.
I am still at the very primary step. trying to setup the EDI batching. Somehow I am missing something, feature is not creating the batched EDI.
Tasks query shows, a status of QUEUED, even after the schedule ran.

can you help
saujanya

Dawn,

Thank-you very much for sharing this. This is a very interesting approach!

I can think of some situations where I could use this. Would you consider turning this into an e-zine article? I think lots of folks could benefit.

Regards

Dawn,

Yes,really it would help all.we will wait for your e-zine article on this topic.Ofcourse only if you have sometime to do that.

Thanks,
RMG

saujanya,
Are you updating the queued task status? The queue is a series of tasks and after processing the document associated to that task you use wm.tn.queuing:updateQueuedTask to update the task’s status. Here is the basics of what I do when I drain the queue.

repeat until tasksComplete

batchGS100 Second repeat to throttle the GS enveloped EDI documents to 100 per GS envelope

wm.tn.queuing:getQueuedTask
If there are tasks log that the queue has tasks wm.tn:log
wm.tn.doc:getDeliveryContent
wm.tn.doc.xml:bizdocToRecord

mapping flow service to create X12 820 document

If it is the initial task
wm.tn.profile:getInternalID for both sender and receiver
wm.tn.tpa:getTPA needed for enveloping
end initial tasks

wm.b2b.edi:convertToString
pub.list:appendToStringList
wm.tn.queuing:updateQueuedTask

end repeat for batchGS100

wm.b2b.edi.util:addGroupEnvelope
wm.tn.doctype:view
wm.tn.doc:createNewEnvelope
wm.tn.docattr:view
wm.tn.doc:setAttribute
wm.tn.doc:updateAttributes
pub.string:stringToBytes
wm.tn.doc:addContentPart
wm.tn.route:routeBizdoc

fall back into repeat until all tasksComplete
end tasksComplete repeat

log tasks complete
trigger service to drain the queued GS documents (a very similar drain process)

There is a bit to it but you do need to update the task status after you process it.

An e-zine article? What’s that? Sure if you give me some direction on what is expected and how to do it.

Hope that helps! If anyone needs more details on the above let me know.
Dawn

Sorry,this is the section we are talking about
[url=“wmusers.com”]wmusers.com(KnowledgeCenter-E-Zine Article)it has archives on Documentation/Development/services/BestPractices etc…

Thanks,
RMG

Thanks Dawn. From your article I came to know that, default EDITPA is the one that is used for batching. Initally, I created individual TPA for each partner. Which was my mistake.

Later, I setup the values in Default TPA. and now it works fine.

We too ran into snags with the out-of-the box batching facilities. Never had issues with the TPAs though–we always enveloped each individual transaction set before queueing it. That way, the default TPA was never used for enveloping during batching–the default is only used when queued docs are not enveloped.

We didn’t use public queues cuz in previous experiences we had run into troubles using them so avoided them here. We used an undocumented facility to allow the batch EDI service to be used without public queues. Basically, we collected the list of docs to batch from the TN store (marked with a particular user status) and passed that list to the batch service. We too added sizing controls to prevent overrunning the server with too many docs in one batch.

To clarify, even though we enveloped each transaction set with both GS and ISA envelopes, the batching still created a proper 1-ISA, 1-GS and multiple ST/SE sets with appropriate control numbering. The envelopes on the transactions sets are used by the batch service to do proper grouping and use the proper identifiers. In this way, the default TPA is not used in any way by the batch process–basically you control which TPA to use when processing the individual transaction set.

This is interesting…I’m planning on using the batching facility too, but I was trying to send the transaction to TN without a group or isa envelope to avoid using up the control numbers and avoid unnecessary enveloping. But when I do the convertToString before sending it to TN, the service inserts the control numbers anyway unless I insert a dummy number in the ST segment. Based on this thread it appears that we should be enveloping each transaction so that the final enveloping works properly.

I have noticed that control numbers for ST/SE segments are not in order. All ST have same number. The control numbers are not incrementing.
What do I have to do

UMG,

Are you creating custom controlnumbers for ST02,SE02 or trying to use webMethods generated ones which convertToString will takecare when you setValue empty?

Please let us know in details.

I am using custom control number, actually the PO # from backend system

So you are using custom number,then those will not increment,as long as your PO# from backend system increments.

HTH,
RMG

here is the situation.
create one interchange with 3 ST with custom PO#(which increment) and submit to TN for queuing
Another interchange with 2 ST and PO# quite different for previous interchange and submit to TN for queuing

So now in my queue, 2 interchanges are waiting for EDI batch.
When the EDI batch is execute(mode=IC&GP), I see one Interchange with 5 ST in it, control number for each ST is set to 0001 only and GE01=1

ST control number must increment and GE01=5 which is not happening

Will:

Here’s a more complete description of what we did at my last EDI project.

• When creating a single transaction set to be batched, envelope the set with the appropriate values as indicated by the appropriate TPA. Do not use the EDI control number table for the control numbers–just let convertToString generate the (bogus and unused) control numbers. It may seem like “unnecessary” enveloping but it actually is necessary to control batching and use the proper TPA for each partner. Without the envelopes the batch process has to make (sometimes erroneous) assumptions.

• When batching and enveloping, transaction set control numbers within a group always start at 1 and increment by 1 for each additional transaction set. batchProcess parameter: contentControlNumber=Sequentialize

• Group and interchange control numbers use the EDI control number table managed by TN for each sender/receiver combination. batchProcess parameter: controlNumber=fromTable

HTH

UMG:

I think you may need submit each transaction set individually to the queue. You’ve done a “partial” batch yourself, which I’m not certain the batchProcess service supports. What are you setting the “contentControlNumber” parameter to in the batchProcess service call? If you intend your ST/SE control numbers to be the PO#, then it should be set to “none” and you need to set ST02 and SE02 to the PO#.

HTH

Very helpful indeed. Thanks Rob!

In my case contentControlNUmber=Sequentialize, I changed it to None and now resulting document has original PO#.

Why I am batching is backend system pushes orders in batchs, so I create a one ISA for all the orders and submit to TN, instead of submit individual orders. As per you, I have to submit individual orders and increment by 1.

Rob:

I am trying your approach, thanks. The documentation says you can use batching without interchange envelopes - to me, the sender/receiver pair in the individual transaction without envelope (via the bizdoc) is enough to find the TPA with all the info contained within.

Using your approach do you split the outgoing envelope (containing 1 transaction) into 3 documents (not really necessary to me) and if so do you base the processing rule (to batch) on the envelope, group or transaction or does it matter?

Right now using an envelope as you suggest, TN splits the envelope into 3 docs and triggers the batch process. But then TN is complaining about the sender/receiver id or qualifier during batching which should be available since it’s in the matching TPA. Obviously I don’t want to hardcode this in the batch input paramters since it can be used for multiple sender/receiver pairs.

Will:

Yes, wm.b2b.editn.batch:batchProcess can be used to process queued documents that do not have envelopes. There are pros to this:

• Reduces the amount of data stored in each queued transaction set.
• Avoid the work to envelope each transaction set, which includes looking up the appropriate TPA to set the envelopes appropriately.

The cons:

• Each queue must contain documents for just one partner. If the number of partners is large the administrative work can be tedious.
• Cannot invoke wm.b2b.editn.batch:batchProcess directly from the scheduled task without hard-coding parms. Must use a wrapper service that looks up the appropriate TPA to set the majority of the batchProcess inputs–e.g. sender & receiver IDs and qualifiers, standard, version, environment, control number.

We chose to put the envelopes around each transaction set before queueing so that we didn’t have a bunch of public queues to manage.

When queueing the resulting enveloped transaction set (X12 Envelope with attribute “EDI Batch” not set), we’d queue the X12 envelope. No split is necessary for queueing BUT because the TPA is used to determine splitting, and we need to split the final outgoing envelope at the group level (see below) it gets split to the group level.

When invoking batchProcess, the sender & receiver ID and qualifiers are (unfortunately) required inputs so you must set them to something. Since all your queued docs have envelopes, these inputs will never be used, so it doesn’t matter what you set them to. This was a painful, several hour debugging effort for me to find that out!

For the outgoing final envelope (attribute “EDI Batch” = “Interchange”) we have a rule that does the delivery. We split to the group level as FA reconciliation says that the split must be done to that level (there’s a side discussion we can have about whether the batch process creates the necessary entries in the tracking tables, which would eliminate the need to split to the group level but I can’t remember how it works for sure). We do not split to the transaction set level. The group envelope is explicitly ignored in processing rules.

Hope this clarifies, not clutters!

Thanks Rob. I guess my point is that TN “should” be able to get the TPA strictly from the sender/receiver pair in the bizdoc and you “shouldn’t” have to hardcode unused values in the batch input. The other thing I noticed was that the sender ids and qualifiers as well as delimiters have to be real values not dummy values (maybe in my case because they’re actually being used - see below).

Okay - I got the batch envelope to be created by TN. However I had to provide the sender/receiver id and qualifiers as well as the delimiters in the batch input, and TN actually used these values not the ones in the envelope. I don’t want to put company specific info in the batch input for obvious reasons.

Agreed that supplying a sender/receiver shouldn’t be necessary.

According to the docs (and behavior I’ve seen previously) the receiver/sender ID and qualifiers are only used when the queued document does not have the group and interchange envelopes. If all docs being batched have envelopes, then the receiver/sender inputs are not used. If that’s not happening for you then something is wrong.

The delimiters parameter is indicated to be optional. If set, they are indeed used regardless of what’s in the TPA or the doc being batched. Trouble is, since the parm is a record either all of them must be specified or none of them can be specified.

I’m working on something else right now, but when I ran it the batch didn’t fire. The activity logs showed an error that the sender/receiver id or qualifier were missing. So I put them in, then it complained about the delimiters. So I put them in too and then it worked but it’s using the values I supplied not the ones from the TPA. What version of IS/TN did you do this on?

6.1

What are your batchProcess input parms? I’m chasing something related to this so I’m eyeballs deep into making sure I understand the entire process so can keep an eye out for what might be causing the behavior you’re seeing.

I’ve tried a variety of combinations but here’s the latest. These should all be dummy values and not used in the batch process. But when I leave them out there are errors in the activity log such as ‘sender id or qualifier error’. I changed the receiver to a different company than what’s in the edi file so that I could see if it was using the hardcoded values below or from the TPA.

senderid 0000800SARD
senderqualifier 15
receiverid 00003F0SARD
receiverqualifier 15
mode IC&GP
oneBatchQueue SINGLEOUTPUT
standard X12
version 2003
environment Production
controlNumber from Table
contentcontrolNumber Sequentialize
acknowledgement false
delimiters
record (carriage return using large editor)
field *
subfield :
release (blank)

Will, I’ve dug into this about as deep as I can but have been unable to explain the behavior you’re seeing. But lemme summarize one last time just in case we’ve crossed wires in our posts:

• The docs say the sender and receiver IDs and qualifiers are used when generating the group and interchange envelope only when the transaction set being batched does not already have a group and interchange envelope. This is not the behavior you’re seeing.

• When oneBatchQueue is SINGLEOUTPUT, the specified sender and receiver IDs and qualifiers are used to set the sender and receiver of the resulting bizdoc.

• According to the docs, if the delimiters parameter is set (any one of the fields) they are always used. If not specified, the delimiters from the appropriate EDITPA is used. If the EDITPA doesn’t specify delimiters, “batchProcess uses its own defaults.” The behavior you’re seeing is that they are always required and used. Do all the partner-specific TPAs exist? Are the external IDs on the profiles set correctly (no typos)? Are delimiters specified in the default TPA?

If this isn’t acting the way the docs indicate it would probably be worth getting tech support involved.

Thanks Rob - I really appreciate your efforts.

I agree that tech support may be required. Since TN is showing the docs with the correct sender and receiver I can say that the profiles and doc types are correct (ie. the extended ids are matching and the 3 doc types are shown). Also, I turned on and off FA generation so verify that my TPA is being invoked. Once I hardcoded the sender/receiver I get the following error - “Segment, field or sub-field delimiter has not been setup [EDIFTN.000010.000778]”. Based on the docs saying the “batch process uses its defaults” one question is whether the default TPA values are used or some other values within the core code.

Hello,

What is the release of your EDI package ?
I’m still “experimenting” the batch process, but I’ve got better results since I installed some fix (WmEDI_6-1_Fix18, WmEDIforTN_6-1_Fix25, WmEDIforTN_6-1_Fix9 …)

Many of you are using the batchProcess with a large amount of data. Could someone explain to me how you manage the �tracability� of transactions ?

In fact my problem is:
I have put documents in a queue, but how to know when my documents were batched and sent to my partner?

For me, the related documents should have been set. I discussed with the support on this point but webMethods told me this is not a bug or a mistake. They do not provide this functionality.

Any idea should be appreciate,

Regards,

Gilles

Gilles,
I assign a conversation id to the documents that way I can track all the related documents including the returned MDN through that one filter in TN. I used sender ID, Reciever ID, and document id as stated in the user guides.
HTH
Dawn

Thanks Gilles - I will see if I can get the administrator to add these fixes. For anyone else having this problem what I may have to do is either batch the docs myself (after sending individual transactions to TN) OR have them go into TN already batched as they come out of the source system.

Thanks Dawn,

How do you set the conversation Id of your batched documents?
All my queued documents have a conversation id but the batched document not.

Regards,

Gilles

“For me, the related documents should have been set. I discussed with the support on this point but webMethods told me this is not a bug or a mistake.”

I agree that the resulting batch document(s) should be related to the documents that made up the batch. IMO, this is indeed a “mistake” on wM’s part. This should be done.

To sum up the shortcomings of the batch process (some discussed here, some not):

• Relate the documents that were read from the queue to the resulting batch document. IMO, this is a serious ommission and one that must be rectified.

• For unenveloped documetns, the sender/receiver should be determined from the bizdoc rather than from parameters to the batch service.

• The batch service should be able to get the list of documents from other sources, not just public queues. For example, one should be able to pass a document list to the service. Another option would be a facility to read documents matching TN attrbiute criteria (sender, receiver, TN doc type, user status, etc.) and then update each document after batching.

• The batch service needs to be able to be throttled. If one has 1000 documents queued, it will try to process them all, which can exhaust memory.

• Control submission to TN. Sometimes it might be preferrable to not submit the resulting batch document to TN.

• The batch service should provide return parameters about the status of each document created and submitted to TN.

Anyone else have ideas on how the batch service could be improved?

Gilles - do you know which fix specifically addresses the batching problem? Are there other fixes that are related to this problem (since you put … in the list of fixes).

I’m going to request the fixes so I can look at the 'read me’s so any more specifics would be greatly appreciated!

We batch EDI documents using the EDI batch feature in 6.0.1. There was a public queue set up and the batch process is scheduled to run at 12:00PM and 13:00PM. The documents come in high volumes from 11:30PM to 12:15PM
The problem here is with the 12:00PM batch. It runs some times and it does not run some times. I have to wait till 13:00PM for the envelope.
I thought the problem is because I was trying to batch and queue at the same time. But the batch process runs on some of the days when the documents are coming in. There is no other process running at that time that looked suspicious and looked like preventing the formation of the batch at 12:00PM. I checked the schedule interval and it looked ok. Did any one run into such problem earlier?

Hi, we are using EDI X12 batching/queuing process followed by X12 envelope. Everything was working fine and we had Hardware issue with the server, due to some database issues we didn’t retrieve all the information. Eventually process started normally and when we are trying to send the X12 Envelope document types we are running into Duplicate Control numbers related to one trading partner meaning the control number is repeating from the period of data loss. I guess since there was data lost in the database. I am just wondering about is there any way to reset the control numbers? This is the following configuration we are using: mode: IC&GP oneBatchQueue: MULTIPLEOUTPUTS standard :X12 version: 4010 controlNumber : fromTable groupControlNumber :fromTable contentControlNumber :Sequentialize acknowledgement :true Could any one suggest me solution to solve the issue… Thanks in advance, capri.

The EDI administration pages let you manage the control numbers and set them to a particular current value. Refer to the docs for info.

Thanks for your help. --Capri

HI All,

we are having the following existing architecture for Processing EDI.The scheduler in webMethods picks up the flat file from the source system and converts into EDI and it will be queued as the other scheduler in the webMethods runs the user status change to DOne and it will generate x12 envloepe which delivers to the customer. But some how it is not working right now. Nothing has been changed recently.It is working till the EDI document getting generated and not ableto generate x12 envelope to be created and causing the data not sent to the customer.

Could anyone please help,

Thanks in advance,
Capri+lak

Hi everyone,
I am trying to create a public queue using a scheduled delivery service and it gives me this error when I try to set inputs “Service Signature Error : Error encountered while reading service input signatures. Please ensure that the service is properly registered”. can someone please let me know what does this mean. I did not get any error when i registered the service.

Thanks in Advance

Is your Scheduled Delivery service(selected in the processing rule actions) nothing but Queue Name that you configured in the Registry–>Public Queues page with Delivery Service as (EDI Batch) ?

HTH,
RMg

Any update on this for 7.1.2 and beyond? Does the EDI Batch process work better now out of the box?