Here’s a fundamental question:

Can anyone explain the difference between a Document and a Document Reference in wM?

Under what circumstances would I choose to use one or the other?

Thanks - MOD

It’s pretty simple and well explained in the developer guide, but here’s a quick overview:

A Document is an object that you create one time in the developer. It can be in your package, or you can create it in the pipeline for use at runtime.

A document reference will reference the document that you create in your package.

So, if you utilize the canonical approach to data management within your integrations, then you will establish, for example, one document that represents a purchase order in your system, and from that point forward, you will “Reference” the purchase order canonical.

It’s a step towards standards-based implementation.

One thing to note: you will not be able to change the referenced document in your flows. For example, if you misspell a data element, you will need to go back to the original document, correct the mistake, and then go to each document in every flow to make sure that fix worked, unless you are using symbolic mapping.

Also, since it is xml-based, case is very important. PurchasePrice is not purchasePrice and symbolic mapping will not link up the data.

Please read through the manual as it will give you some other ideas.

Ray

Ray,

I’ve had a question I’ve been meaning to post and this seems like a good thread to add it to.
When working with Documents generated by a schema your can only reference the entire document. Therefore when reusing the document reference you are force to always pass the entire document to each service.
In the past if I manually created the document all the records in the document would be in itself a document reference. Then if I only needed to work on a particular record I could reference that record vs being forced to reference the entire document which chews up more memory.

Any thoughts on how to accomplish this when working with documents generated by schema editor. if it can’t be done then IMOP this is fundamental flaw in the schema editor.

Thoughts?

Chris

Michael,

As i mentioned the same in your other thread about Document Vs DocumentReference,Ray’s comments are more elaborate than i said. So i guess you have a good understanding from these posts.

HTH,
RMG.

Hi Ray:

Thanks very much for taking the time to answer. I’ve injected comments into your reply.

“A Document is an object that you create one time in the developer. It can be in your package, or you can create it in the pipeline for use at runtime.”

Thank you, this part is clear.

“A document reference will reference the document that you create in your package.”

I’d defy anyone to explain “Document Reference” without using the word ‘reference’. Again, there’s nothing new here, IMO. Perhaps I’m overcomplicating it.

If I’m setting up a new service, and it comes time to define input and output, I can choose either Document or Document Reference.

“So, if you utilize the canonical approach to data management within your integrations, then you will establish, for example, one document that represents a purchase order in your system, and from that point forward, you will “Reference” the purchase order canonical.”

So you’re saying ‘Define the Document in your package, then refer to it in your service’s input/output.’ There’s no situation in which I’d choose Document for my input or output, always a Document Reference. Correct?

“One thing to note: you will not be able to change the referenced document in your flows. For example, if you misspell a data element, you will need to go back to the original document, correct the mistake, and then go to each document in every flow to make sure that fix worked, unless you are using symbolic mapping.”

Good to know. That’s been my experience: changing the input or output means starting all over again with a service.

“Also, since it is xml-based, case is very important. PurchasePrice is not purchasePrice and symbolic mapping will not link up the data.”

This I’m used to - Java and XML have beaten it into my head.

Thanks again, Ray. Sincerely, MOD

Perhaps it’s my C++/Java developer’s background, but I ask because I want to make sure that there’s not some hidden issue about Documents being immutable and Document References being mutable. Thanks - MOD

Michael,

In your Scenario FF acts like a Canonical document,so define this structure in a package and use this as a DocumentReference in the Service Output tab and in the service input tab use the T850DT documentRefefence from the WmEDIForTN/EDIFFSchema folder.

Depends on your team decision there you can move these documents to a new package like (EDIRepository)this will store all the various EDI DocumentTypes say (T850DT,T810DT etc…)

Anyways sorry for interrupting this post in the middle.

HTH,
RMG.

Chris,

I’ve had the fortunate/unfortunate opportunity to work on one of the webMethods XML adapters (yep, if use the PapiNet 1.0 release, you can freely curse my name).

The schema editor creates the document based on the xsd file supplied. When I started out, my Module was about 120mb in size, far too big for distribution. By the time I was finished, I had the size shrunk down to about 9.8mb, which is pretty good for the effort.

This was accomplished by creating the base set of schemas and then referencing the schemas like crazy for all the documents needed.

webMethods developer will create the sub-document that the main document will reference.

For example, I had an address document that existed once but was refererenced in the xsd file set extensively.

The compiler created the address document and this object provided the reference point for the collection of documents that required an address node, based on the references made during the xsd creation.

I hope this helps.

Ray

Michael,

It may be very helpful for you to examine the java API docs for webMethods.

Using java in some instances is more effective then flow service especially in looping.

Ray

Michael,
I can use relations in C++ for anything else that is not too clear so far. So for document and document reference, it is like a instance object and a pointer to that instance object in C++.

You may in fact want to use a document in your service if you are not sharing the document between multiple services.

I have had my share of problems with using Document references and loops. Since the reference is a pointer, if there are two or more pointers and you do an operation on one, it will affect all the others.

You can try this by making two document references that each point to a document with one internal field. Visual:
{Legend}
= doc
[] = doc ref
—MAP1
[]doc2
–field, value = 3
—MAP2
[]doc2 -> []docA, -> [*]docB
—pub.math:addInts
doc2 doc2
–field, “1” -> --field

At this point. If you alter values in the original it will cascade into the secondary documents child element at the same time. This will hold true of course for document references that point to document references. In such a case, you will need to use only documents mapping to other documents or never alter the original document and treat the two references as if they were the original.

The same is true if you have a reference that was pointing (in from or out to) an element in a Document array. A change to one will be seen in the other. This is bad when doing looping that may try to fiddle with more than one element at a time.

I have enclosed a html view of a test flow I made.

Hi Ray,

“It may be very helpful for you to examine the java API docs for webMethods. Using java in some instances is more effective then flow service especially in looping.”

Thanks for the advice - greatly appreciated.

I was hoping I’d be able to accomplish this with off-the-shelf services. I’m the first wM developer in my department, and the only Java developer. For the sake of those coming after me, I’d like very much if I could tell them how to do this without having to learn Java.

I think the problem is my lack of experience with wM. It seems to me that the built-in packages ought to be expressive enough without having to resort to custom code. Looping through hierarchical documents is the bread and butter of this product. It shouldn’t be this difficult - OR you need to get smarter customers.

%

Hi Yemi Bedu,

Thank you so much for going to all that trouble. I’ve just downloaded your ZIP file. I’ll give it a look, try out your ideas, and let you know how they work out. Thanks again. Sincerely, MOD

Michael,

One place you need to devote some time to is the GEAR 6.0 documentation on the advantage site.

GEAR provides best practices framework for your integrations.

Keep in mind that best practices in general may not always be best practices for your organization.

In regards to the loop functionality, the gear documentation covers it pretty well. From my perspective, looping is one of the weak spots of the product only from the standpoint of large document handling.

There is a way to loop properly and there are may threads here devoted to the topic.

If you get a race condition on the jvm, have cpu pegged at 100% or the system is unresponsive, the culprit in many cases is loop code.

HTH

Ray

Hi Ray,

Thanks for steering me to the GEAR document. I wasn’t aware it was available. (Found the EDI GEAR, too. Most helpful, I’m sure.)

I’ll start reading right away. Thanks - MOD

Hi Ray, Yemi, Chris, RMG, and all wM forum members,

Thanks to your patience, and some dogged thinking, I’ve finally seen where I went wrong.

I couldn’t understand why the output Document Lists that I was mapping were getting lost in the pipeline. The key was to make a Document List pipeline output with the same name as the service output Document List and then drag & drop it up to the parent document to make it a child. Map the service output Document List to this new child Document List, drop the service output Document List from the pipeline, and everything is perfect.

Looping in my case turns out to be relatively easy, in spite of all the difficulties I’ve had. The input document array generates one output array entry for each iteration through the loop, so I can use the LOOP /input and /output array notation to good advantage. No need for appending documents to list or anything else.

I’ve done this trick several times since I discovered it. My EDI-to-FF mapping is progressing nicely now.

Thanks to all of you for your patience and insight. I hope this thread will help someone else who’s facing wM LOOPing for the first time in the future. - %

Michael,

A few comments about looping:

The method that you are using is the basic loop with an in-array and an out-array. The loop should map one an object from the in-array to an object of the out-array. This is the typical looping scenario. One for one.

What if you are looping over a purchase order and you need to filter all orders whose values are greater than $10,000.00?? What you will do is create a loop with only an in-array. In a map step prior to the loop (i always create a map-step just above the target loop), instantiate the array object that you will use for the output to your loop. It could be a document, document reference, sting list reference or string list.

When you begin your looping, you will probably use a branch to branch on the value of the purchase order. For all values above $9,999.99, you will append ONLY that record to the newly created output array (that you created in the map step prior to the loop.) On the branch step, I usually create a sequence that will contain all of the relevant code.

The end result should be an array with all records containing values above the constraints listed above.

Hi Ray,

Thanks so much for sharing that idiom. I’ll keep it in mind. I don’t believe I have a situation like that in the project I’m working on now, but it could be handy in the future.

More importantly, I can visualize exactly what you’re telling me - for the first time. I feel like an eejit, but it’s taken me a while to get my mind around the pipeline and how it works. I think it’s a personal flaw/characteristic in the way I learn. It can be frustrating, but when the pieces finally fall into place it’s a great feeling. I like to think that they have. Let’s hope ongoing work doesn’t make a liar out of me.

I appreciate having experts like you available on this forum to educate noobs like me. Thanks for your time and patience. Sincerely, MOD

Jumping in a bit late to the thread:

Document references and documents are not like a C++ or Java reference/instance. An Integration Server document reference is a declaration only. It’s not an instance. It’s not a reference. It’s a definition.

When using a “document” in a FLOW input or output, or directly in the pipeline, you’re defining the structure of the document right there and then.

When using a “document reference” you’re indicating that you want to use the structure of the document that is defined elsewhere.

At runtime, instances of what are essentially variable pools are created. It doesn’t matter how the “instance” was defined–a document or a document reference–as they look exactly the same in the pipeline (the pipeline itself is actually a variable pool).

That said, mapping structures does have some characteristics of instances and instance references (in the C++/Java sense). Mapping structures uses by reference, mapping strings uses by value (that is, a new string with the initial value coming from the source is created). This is one of the reasons looping can cause confusion. Within loops, you quite often need to take explicit steps to create independent instances of structures.

To use a Java analogy, if you loop over a source array and simply assign objects to a target array, both the source and target arrays will refer to the same objects. Modifying an object in one array “appears” to mistakenly modify the object in the other array. But since both arrays point to the same object, this is actually correct behavior. If that’s the behavior one wants, perfect. But it most often is not the desired outcome. It’s very easy to make this sort of mistake in FLOW.

Another common error in FLOW is to use a temp record variable over and over, the result being that all records in a list have the same values. The key step is to drop the temp variable at the end of the each iteration. This will force the runtime to create a brand new instance, and independent instance, in the next iteration.

HTH.

Rob

Hi Rob,

This is an excellent explanation of the difference between documents and references. Now that I’ve had some time to think about it and do some work with webMethods I can see that you’re spot on here. Thanks so much for posting it.

I’ve been following the advice I’ve been given here to good advantage over the last few days. Thanks to the forums, I’m starting to feel more comfortable with webMethods.

Sincerely, MOD

Hi Rob,

Nice to hear from you again. This is your favorite topic and one that you taught me some time ago, through your very well thought out and well written answers. Thanks!

Ray

Michael,

Hope you got the nice explanation what you are looking for from Rob,i know this thread will definetely help for basic starters.

Goodluck,

So until proof document references behave like a reference into an object instance.

Regards,

Luis Cordeiro
_TEST_DOC_REF.zip (21.4 KB)

I downloaded the package and tested on 6.5 with SP2. I’ve not been able to duplicate what you observed. The doc returned from both the cached and non-cached services is always the same and is always as expected. The non-reference services behave the same way. What version of IS are you using?

I’ll clarify my description on documents and document references–and document type declarations.

An IS document type is a simply a definition. It describes the fields of a document. The document type is used primarily at design/debug time by Developer. It is (optionally) used at run-time by services that perform validation and conversion to/from IS documents. For example, the document type name is passed as parameter to xmlNodeToDocument or pub.schema:validate.

An IS document is an instance of an object (a class which implements the IData interface). The pipeline var is a pointer to that object (it’s a classic Java var). It contains fields which are either primitive data types or are also documents. Change one of the fields of the object, through any reference var, and the change is reflected appropriately for all those references. But this has no connection to IS document references. An IS document has no connection to an IS document type–even if that document was created through the design-time use of a document reference. This is why we have to explicitly pass the document type name to various services.

An IS document reference is a declaration of a named variable that refers to an IS document type. This is purely a design-time/Developer association. Developer knows that a var is related to a document type, so it shows that relationship. But that doesn’t change the nature of the variable. The variable is still a reference to a Java object. And that variable and Java object have no idea about the IS document reference. If it did, we wouldn’t have to pass the IS document type name to documentToXMLString–it would already know.

As further evidence of no connection between a document var and document reference, create an arbitrary XML string. Pass that to xmlStringToXMLNode and the resulting node to xmlNodeToDocument (don’t specify a documentTypeName). Map the output document to any document reference variable. No errors. No issues. And you’ll see the document reflects the fields and structure of the XML string, not the IS document type referred to by the reference.

The vars in the pipeline behave, as all Java vars do, as references. This would explain the behavior you’re apparently seeing (which seems to be a bug of some sort). But this behavior has nothing to do with IS document references.

Thanks for your reply Reamon.

My IS is 6.1.

I have opened an SR with SAG in the meantime, but I will try this on other 6.1 instalations and on a 7.1 also.

I’ll post the results.

Hi All,

We are working on parsing/validating an XML file that is specific to our partner. It has SOAP data in the XML file. We are trying to create a document type for the XML file but not able to get the complete structure. Please help in getting the document type created for this XML.

All xsd files are depending on references like as in screen shot,

Please help here

https://higherlogicdownload.s3.amazonaws.com/IMWUC/webMethods/Files3/5bd01944022c4bbe8d9255f3cf1c57065e4c33b2_2_1035x528.png 1.5x, https://higherlogicdownload.s3.amazonaws.com/IMWUC/webMethods/Files3/5bd01944022c4bbe8d9255f3cf1c57065e4c33b2_2_1380x704.png 2x" data-dominant-color="FAF5FD">

xsd.PNG1460×745 68 KB

Hi,

where are the XSDs stored?

Do you have them locally available or are the located on a remote server?

There are two options for importing the XSDs to generate the document structure:

• generate from top level XSD and have it creating the intermediate schemata on the fly.
• generate the intermediate schemata from their XSD first and then create the document from the top level XSD.

Regards,
Holger

Thank you for reply Holger,

we have stored in local system and i have generated top level XSD’s but it’s showing depending references and different depending name speaces . like as,

Hi,

I cannot see any error or problem here.

Can you explain what issue do you encounter?

Regards,
Holger

Hi Holger,

We were working on parsing/validating an XML file that is specific to our partner provided different XSD (around 200 reference xsd’s. all XSD format GIMA format ). But we haven’t GIMA format imported in Designer and It has SOAP data in the XML file. We are trying to create a document type for the XML(XSD also) file but not able to get the complete structure. we didn’t import GIMA format XSD.

Hi Vara,

you will have to import all XSDs to resolve the complete structure either implicit or explicit.

When you are using LocalServiceDevelopment you can import from your local development.
Pleaaste note the schemaLocation in the import tag should contain “./name-of-imported-schema.xsd” to indicate that it can be found in the same directory as the top level xsd.

If LocalServiceDevelopment is not available you should either import the XSDs from an URL pointing to the partner side storage location or to a Tomcat (as example) running on the same box as the IntegrationServer where the XSDs are stored under webapps/ROOT/.

Fields whose name start with an “@”-sign are attributes for the field above and not child elenents.

Regards,
Holger

Hi Holger,

Thank you for your inputs,

Please find the attached sample files for GIMA XSD’s. just i mentioned root and child GIMA xsd’s,

I have change partner names in attached files.

Please help here on this. we didn’t parsing in Gima format in webMethods.

GIMA request.xml (3.36 KB)
subroot.xsd (3.6 KB)
root.xsd (1.34 KB)

Hi Vara,

are there any messages in server.log or eror log when you try to import the schema files pointing to missing imported XSDs?

If so, can you provide us these?

Regards,
Holger

Hi Holger,

PFA…

ActorRootDoc.PNG770×378 11.7 KB

Hi Vara,

can please post a list of the content of the doc folder if possible?

This will help us to identify what was already generated and what is missing.
The Elements under ActorRoot appear to be geerated somehow but without content (at least partially).
Tths might due to the fact that XSD for these document types was not available during generation.

Regards,
Holger

Could you please send me your mail Id . I will send separately

My mail id varadharajulareddy@gmail.com

Hi Vara,

Can you post the list on my wall plase?

Regards,
Holger