I've wondered this for a long time.  Should compilers, and even other environments (CICS et al) really have the responsibility for the support of transformations between programming language structures and XML/JSON/etc.?  It seems to be that between COBOL, PL/I, CICS, DB2, etc. the parsing/generating between XML/JSON and programing language structures seems to have happened at least a few too many times.

DB2 I'm going to leave out because it probably really (perhaps) needs explicit XML (and JSON?) support.  But do we really need all of these other ways?  In my opinion, and no offense to COBOL development, but its only with the recent support of the SUPPRESS clause of XML GENERATE that Enterprise COBOL is even generally usable for transformation to XML.  And it still has what I consider a fairly large limitation, which is that you must know the maximum number of occurrences that you can ever have for an element, and make the table "OCCURS clause" match that.  Yes, I realize that there is OCCURS UNBOUNDED, but that requires:

• Dynamic memory allocation (made simpler in V6 with ALLOCATE statement, but...)
• Still knowing before you allocate memory for your structure how many occurrences will be in it!

This being said, I think that perhaps COBOL itself (compiler and run-time) are not the place for this support at all.  Let them concentrate on improving the compiler and supporting the actual COBOL standard!  I envision an API, perhaps built by the group that built z/OS Connect Enterprise Edition, that can be used by any of the more "static" programming languages (COBOL, probably PL/I, and perhaps C) to perform transformation between the language data structures and XML/JSON/(others?).  In fact I imagine that configuration of a JSON transformation and configuration of an XML transformation would be pretty much the same (except where one format has a feature that simply cannot be represented in the other format), so that when calling the API to transform to one of the formats you would simply choose XML or JSON as the destination format.

I have many more, specific, thoughts on this, but I'm wondering if I am just "off the beam", or if this is a valid thing to request from IBM.  It seems to me that it could eliminate the need for XML PARSE and XML GENERATE from COBOL and the XMLCHAR and the PLISAXx functions from PL/I.  For CICS, the TRANSFORM command (which just recently added DATATOJSON and JSONTODATA options) could either be eliminated, or at the very least could be modified internally to utilize my envisioned API.  Also the transformations required by SOAP and REST support in CICS.  Others I don't know as much about (IMS TM, for example), but I can't see why they wouldn't use this as well.

I'm not really suggesting XML PARSE and XML GENERATE be removed, as it is used to heavily, but that COBOL no longer need to make enhancements to them.  If this responsibility is in "once place" then hopefully we wouldn't have all of the "re-inventing of the wheel" that seems to have occurred up to this point.  Additionally, hopefully the group supporting this API would be more focused on supporting all of the features that are truly needed for these transformations.  It seems to me that COBOL support, and I am truly not trying to offend, only adds support for features as requests come in for them; rather than having taken a holistic approach and supported "everything" (or everything reasonable) at the get-go.  Specifically I refer to how originally there was no support originally (in XML GENERATE) for attributes, assigning names other than the "COBOL names", applying individual types on a "per field" basis, suppressing "empty" fields, etc.  All of these are now supported, but it seems to me to have been a long time coming and should have been implemented on "day one" of XML support (10+ years ago?).  Or rather, they should not have been implemented at all, but rather as part of this API I am thinking about.  (And I won't even get in to the horror of XML PARSE, which is so low-level that I've never actually used it in a production program.)

Note that I am not referring to the low-level support that is provided by z/OS XML System Services and the z/OS JSON Parser.  Certainly this new API, which I see as being at a higher level, would use those services under the covers.

Its interesting to note that the COBOL standards committee, who released the COBOL 2002 and COBOL 2014 standards, had long and heated discussions regarding if COBOL should support XML at a language level.  There was a proposed language support, and interestingly it was not the XML PARSE and XML GENERATE statements supported by Enterprise COBOL, but rather something based on an "XML file" where you would "describe" your transformations in an "XD entry" in the FILE SECTION.  This feature is not present in either release of the standard, so I must assume the proponents lost.  And perhaps that's all for the best.

I guess I should briefly (??) describe how I envision this API would work.  Basically, you'd have a "configuration artifact" that might be very simple to how a "BMS map" is developed.  You would describe the "programming language structure" and how it maps to the desired "external formats" (XML, JSON, other?).  You'd use this to generate the actual programming language structure(s) (COBOL copybook, etc) as well as an "internal control block" that is understood by the API.  The program itself would simply invoke the API with the "name" of the structure, the direction (to/from), the external format (XML/JSON), and the address of the source and destination.  And probably a "results" field to hold error conditions, etc.

It should also allow not just a single call to the API, but also a "streaming" type of transformation, where you could feed in "chunks" of source data and also retrieve in to perhaps more than one programming language structure.  This would allow for transformation of very large documents where you don't know ahead of time how large they are (think of VisaNet's "VML" files, if you are familiar with them.  They could have millions of instances of some elements.)

Am I crazy?  Should I request this?  Should be come free with z/OS, or should we have to pay extra?  What are your thoughts?

Frank

XML (can be, should be) done by System Service from COBOL, JSON is. Here's the XML http://publibz.boulder.ibm.com/epubs/pdf/gxl3z110.pdf

Now, if XML and JSON aren't processed by the compiler, and you do want to use the "COBOL structure" for the names, it'd be a nightmarish mass of literals/constants to keep an eye on, let alone setting up the API to do different things.

I'm happy they are there. Curious about why something wasn't addressed in the previous COBOL Standard ("hey, yay, we have a few new ways to code 'GO TO!'" instead).

I wrote a lot but apparently still was not clear.  My proposed API would still use COBOL group items as the basic source or destination (depending on transformation direction), and would still use z/OS services when available.  Its simply (!) the mapping that would be done outside of COBOL.  There would be a "schema" of some sort that would define both the "external" format (XML names, types, etc) and the programming language structure (COBOL identifiers, types, etc.).  Certainly this could be done (for XML GENERATE and JSON GENERATE mostly is done) from "within" COBOL, but this still leaves you with CICS doing something totally different, which just seems crazy to me.

As for why it wasn't addressed in the COBOL standard, too many people thought it should not be there.  And just think, if they did address XML, then what about JSON?  Wait for a new standard?

Here is a link to the Draft Technical Report describing how COBOL could have supported XML directly: http://www.cobolstandard.info/j4/files/07-0005.doc.

Thanks for the link. I'm glad IBM didn't go that route with XML/JSON (I find it difficult to understand why anyone would think it a good thing to treat XML as a 'file' - "here's a lump of data, can you process that please?", "ah, sorry, no, has to be in a file, old boy, this is COBOL don't 'cha know?", "but COBOL can handle parameters?", "well, err, yes, I suppose so. I bet there's a jolly good reason the Standards Committee didn't do it that way, though. Just can't think of it myself, at the mo'".

You can't use the COBOL items (names, types) as data for an API - unless you want to change COBOL, or you want to code those "manually". One of the big advantages of having the compiler doing the XML is that you don't have to do it manually.

It is great to be able to respond to JSON "quickly". Wait for the COBOL Committee, 15 years, and then just get another suggestion to use a 'file'? That's for the boids (attempt at a Brooklyn accent, don't know how well it comes off with typing).

IBM have just redesigned the compiler. As with the last new machine, when the next one is announced (January 17, 2017) a new COBOL compiler fully supporting any new instructions will be announced that very same day. Look at the new functionality provided since the announcement of V5, not even including what is in the original V5.

OK, wider disjuncture from Standard. I'm not sure it can be helped, with the time the Committee (and all the Standards Committees) take over everything.

So, I'm happy that we can generate and parse XML. I'm happier the more functionality we have to do that. Of course, it has to work, but you've done your bit on that, and will do next time, I'm sure :-)

Just finally, for anyone at IBM who saw that date and went white, the only reason I put a specific date was to make someone go white. It is Monday morning, it's raining. Gotta have something to cheer myself up...

Hi Bill.  I'm not sure what you are getting at WRT the standard requiring a "file definition".  An "XML file" no more has to be a physical file than does a sort file.  The TR specifically says "An XML document can be on a traditional file medium or it can be in memory."  Specifically, you could say:

SELECT xfile     ASSIGN TO DATA xdoc    ACCESS MODE IS XML    ORGANIZATION IS XML.

In this case 'xdoc' would be a COBOL data item containing the XML document (or the area where the resulting document is to be placed).

So let's continue with this...

FD  xfile.01  xfile-rec                          IDENTIFIED BY "root".    05  hdr-rec                        IDENTIFIED BY "hdr".        10  user-name                  PIC X(25)                                        IDENTIFIED BY "user" IS ATTRIBUTE.    05  body-table                     IDENTIFIED BY "body".        10  body-rec                   IDENTIFIED BY "records".            15  record-txt             PIC X(80)                                        IDENTIFIED BY "record" IS ELEMENT.OPEN OUTPUT xfileOPEN DOCUMENT xfileMOVE "FJSwarbrick" TO user-nameWRITE xfile-rec       ONLY ELEMENT hdr-recMOVE "record 1" TO record-txtWRITE xfile-rec       ELEMENT body-recMOVE "record 2" TO record-txtWRITE xfile-rec      ELEMENT body-recCLOSE DOCUMENT xfileCLOSE xfile

This would (assuming I am reading things correctly!) result in the following:

<root>  <hdr user="FJSwarbrick"/>  <body>    <records>      <record>record 1</record>      <record>record 2</record>    </records>  </body></root>

The main point being that I don't need to know ahead of time how many "record" sub-elements are present, I can process/generate them iteratively.

Now again, I am not suggesting that Enterprise COBOL implement this.  However I am suggesting that there be an API that implements similar behavior without the need to change COBOL.

And again, I am saying that the there would be a "API configuration" that would map "XML names" to "COBOL names".  So in COBOL I would have create one of these "API configuration" control blocks and then do something like this:

MOVE "FJSwarbrick" TO user-nameCALL 'TRANSFORM' USING isrt-token, xdoc-token, hdr-rec-token, hdr-recMOVE "record 1" TO record-txtCALL 'TRANSFORM' USING isrt-token, xdoc-token, record-txt-token, record-txtMOVE "record 2" TO record-txtCALL 'TRANSFORM' USING isrt-token, xdoc-token, record-txt-token, record-txt

Here isrt-token" would be an API supplied field representing the desire to insert data in to the document.  xdoc-token would be an API supplied field representing the entire document.  hdr-rec-token and record-txt-token would tell the API where within the COBOL copybook (the position) is the "field" we are dealing with.  And of course hdr-rec and record-txt would be the actual data to be inserted.

Obviously this isn't a fully designed system.  Just a starting point.

Rats. Just lost a lot of text by brushing the touchpad. No Undo here that I can see, and that lovely Firefox has, without asking me, removed by "save the stuff I typed in boxes, cos it's useful when there are accidents" plugin.

Anyway, I'm not writing it again. Maybe later.

For your suggested API vs IBM's use of the API for the System Service, it is not only the "name" you need to provide, but also the start-position, length and type. Getting that straight each and every time is something the compiler is good at, and we less so. If you want to do it "by hand", you can still use the System Services. How that compares to the "flow" for the proposed COBOL XML processing, I don't know.

(OT comment -- [Ctl-z] in Firefox should undo previous edits. E.g., if you delete a block of text, pressing [Ctl-z] should bring it back. Some number of previous edits can be undone by multiple presses of [Ctl-z]. I think most browsers do the same.)

As for COBOL + XML, I first used COBOL for some XML maybe 10 years ago (12 years? not sure, time flies). But most of my work is on IBM i, and COBOL was the first compiler available there from IBM to handle XML. I assume some coordination among various IBM compiler groups, and having some useful integration between efficient business logic and DB2 was handy for XML.

I know most traffic in this forum is about the mainframe COBOL compiler, but maybe the other IBM platforms also have some COBOL + XML interest. Hard to tell how that might affect perspectives for this thread.

Thanks for the tip, Tom. I've just tried the Ctrl-Z and it works to recover, and ("tested" with single characters) goes back for at least 10 undos. Normally I right-click and select Undo. That doesn't work in this edit box. Now, if only I had a time machine (or a backup where I could easily extract my previous Firefox state).