Originally posted by: Jan_Frederik_Sorge

Hello!

I have two maps, where the first is the controller and calls the second with the code below.

=RUN(   
"transform_xmofd31t_to_xml%STDMapFileExtension%" ,
"-ID1" +
" '-MDQ %STDMdqFile%" +
" -DBNAME " + DBName Comp:Seq[1]:ElemDecl Databases Comp T_std_control Type:std_control Element:std_control +
" -QUERY Query_" +DBSchema Comp:Seq[1]:ElemDecl Databases Comp T_std_control Type:std_control Element:std_control +
".XMOFD31T_Trade_SST" +
" -VAR mandant="+Mandant Comp T_std_control Type:std_control Element:std_control +
" -VAR verarb_prio="+SelectionCriteria Comp[1]:Seq[1]:ElemDecl Databases Comp T_std_control Type:std_control Element:std_control +
" -VAR kest_rel_kez="+SelectionCriteria Comp[2]:Seq[1]:ElemDecl Databases Comp T_std_control Type:std_control Element:std_control +IF(Trace Comp T_std_control Type:std_control Element:std_control=
"true" ,
" -TRACE  %STDLogFilePath%%STDPathSeparator%"+Filename Comp:Seq[1]:ElemDecl TraceFiles Comp T_std_control Type:std_control Element:std_control ,
" -TRACEERR  %STDLogFilePath%%STDPathSeparator%"+Filename Comp:Seq[1]:ElemDecl TraceFiles Comp T_std_control Type:std_control Element:std_control) +
"'" +
" -OF1 "+IF(ABSENT(Path Comp:Seq[1]:ElemDecl OutputFiles Comp T_std_control Type:std_control Element:std_control) ,
"..%STDPathSeparator%data%STDPathSeparator%" ,Path Comp:Seq[1]:ElemDecl OutputFiles Comp T_std_control Type:std_control Element:std_control+
"%STDPathSeparator%" ) +Filename Comp:Seq[1]:ElemDecl OutputFiles Comp T_std_control Type:std_control Element:std_control +IF( Trace Comp T_std_control Type:std_control Element:std_control=
"true" ,
" -TIS=%STDLogFilePath%%STDPathSeparator%"+Filename Comp:Seq[2]:ElemDecl TraceFiles Comp T_std_control Type:std_control Element:std_control ,
"") +
" -AEM -WDU" )

If the execution of the second map fails the DB Error Trace should be parsed by calling an additional map. This fails with RC 12 (see attached AuditLog). Is there a lock on the trace file? Does anyone know a solution how to parse this correctly?

I'm using WTX 8.1.0.2(38) on Windows XP.

Thanks in advance!
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender
#DataExchange

Originally posted by: phil2030

How are you running the map to parse the db trace? From within the controller map on a subsequent rule or is it triggered within the launcher. Either way, it is probably either locked or not flushed to disk by the time you come to read it. The fix depends on what you are doing with the error detail (if you assuming it is a db problem that caused the map to fail). If your using it to send an alert to an operator there are alternates, but it depends on the framework you have around your maps. Are you using the launcher or something else?
#DataExchange
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender

Originally posted by: Jan_Frederik_Sorge

I tried to call the parser-map from a second rule on the first output card of the controller and from an additional second output card of the controller.
While testing the map it runs stand alone in the DesignStudio but in production environment it will be called by a WMB flow which does the error handling (alarmind, etc). This flow needs as much information as possible, so I thought of parsing the DB error trace file.
Is there any chance to let WTX flush the output directly when the second map has completed?
#IBM-Websphere-Transformation-Extender
#DataExchange
#IBMSterlingTransformationExtender

Originally posted by: phil2030

Are you using the TX node (can't remember if it is called that exactly) or a Java node (and the platform API)? I have not used the standard TX node in WMB, but instead used Java and then the platform API (the former was severely limited in what you could do when I tried to use it). Have you not got some standard workflow for handling maps that have failed? Tracing (of any adapter or input/output) is quite an overhead in a production environment and can cause issues (disk space aside) when trying to tie the files created back to the maps that have failed.
The question is what has prompted you to go down this route ... of analyzing the trace file?
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender
#DataExchange

Originally posted by: Jan_Frederik_Sorge

We are using the default Node delivered by IBM, not a JCN. This Node has been put into a subflow which has default error handling for a map which has failed.
We are parsing the TRACEERR file as there are more detail information about the error which has occured. Or is there another (better) way to get these information?
#DataExchange
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender

Originally posted by: phil2030

Are these "data" errors or "configuration" errors (i.e. the DBA has changed the password/role/grants/whatever for the TX user). ...or both? Is this a new development or have you a lot of existing maps?

For any "data" errors you could make use of stored procs. (if not already) and throw an exception if the data is deemed invalid. This should mean the map fails (assuming any DB* funcs have a VALID/FAIL). I don't think the exception message is passed back to the map, but you could log the message in a separate table and still get it assuming you have a unique message id. or map instance number.
#DataExchange
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender

Originally posted by: Jan_Frederik_Sorge

We want to log both kind of errors, but do not use stored procs to validate the data put to db.
It is a new development where I want to design a better default handling of adapter failures (e.g. db / mq) and better error messages sent back to message broker.

I think it would be the best solution to get the data out of the TRACEERR file and get all necessary data (e.g. SQLSTATE) back to message broker by passing it through memory in a defined structure.
#IBMSterlingTransformationExtender
#DataExchange
#IBM-Websphere-Transformation-Extender

Originally posted by: phil2030

So you have a mechanism in place for tying all the trace/log files created back to a single map instance? I don't know much about the node your using.
You also know what you wish to extract from these files to present to the operator?
#IBM-Websphere-Transformation-Extender
#DataExchange
#IBMSterlingTransformationExtender

Originally posted by: Jan_Frederik_Sorge

I generate the filename for the TRACEERR file in message broker to get them unique. These files should be parsed with a second map (which will be called by RUN command) with the following search parameters.

=IF(FIND(
"<HT>|<HT>", Ein1) > 0 ,IF(         FIND(
"Reason", Ein1) > 0 | FIND(
"Adapter error", Ein1) > 0 | FIND(
"MQSeries error", Ein1) > 0 | FIND(
"anager", Ein1) > 0 
/* include further keywords here */ | FIND(
"Queue node found", Ein1) > 0 | FIND(
"SQLSTATE", Ein1) > 0 | FIND(
"Adapter command", Ein1) > 0 | FIND(
"Error", Ein1) > 0 ,SUBSTITUTE( LEAVEPRINT( MID(Ein1, FIND(
"|", Ein1)+2, 333) ), 
"|", 
"") ,
"") ,IF(        FIND(
"Reason", Ein1) > 0 | FIND(
"Adapter error", Ein1) > 0 | FIND(
"MQSeries error", Ein1) > 0 | FIND(
"anager", Ein1) > 0 
/* include further keywords here */ | FIND(
"Queue node found", Ein1) > 0 | FIND(
"SQLSTATE", Ein1) > 0 | FIND(
"Adapter command", Ein1) > 0 | FIND(
"Error", Ein1) > 0 ,SUBSTITUTE( LEAVEPRINT( MID(Ein1, FIND(
"<HT>",   Ein1) + 1,  333) ), 
"|", 
"") ,
"") )

#IBM-Websphere-Transformation-Extender
#DataExchange
#IBMSterlingTransformationExtender

Originally posted by: phil2030

So if you put this into your "default error handling" does it work?
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender
#DataExchange

Originally posted by: Jan_Frederik_Sorge

This parses correctly if it runs standalone with a TRACEERR created before. But it fails to get any data of the TRACEERR if it will be called by RUN command.

This is the problem why I opened this thread...
#IBM-Websphere-Transformation-Extender
#IBMSterlingTransformationExtender
#DataExchange

Originally posted by: phil2030

Ok, but what you have in WMB at the moment, is it the same "controller" map running your "business" map and then your "error" map (if the "business" map fails)? Or is the "error" map (that reads the trace files) in your default error handling workflow?
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender
#DataExchange

Originally posted by: Jan_Frederik_Sorge

I'm running one controller map which calls the business and the error map afterwards. But it could be a idea to run the error map after the controller has been run to completion. This should cause the TRACEERR file to be accessible.

But is there any possibility to let WTX flush the TRACEERR to the filesystem?
#IBM-Websphere-Transformation-Extender
#IBMSterlingTransformationExtender
#DataExchange

Originally posted by: phil2030

Nope, other than the map finishing.
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender
#DataExchange

Originally posted by: Jan_Frederik_Sorge

So does a map which has been called by RUN command not complete it's filesystem transactions?!
#IBM-Websphere-Transformation-Extender
#DataExchange
#IBMSterlingTransformationExtender

Originally posted by: phil2030

If you have (or can have) more than one output card, you could try setting the transaction scope to card and see if that has any effect, though that may very well break transactional integrity at the workflow level. But failing that I would say no, your operating in the same thread and there will doubtless be buffering going on to minimise disk interaction, therefore until the thread (that I would guess is created from the node in your workflow) is complete (and all I/O flushed as a result) you cannot rely on the file being there or if it is there, that it is readable and/or complete.
#IBMSterlingTransformationExtender
#DataExchange
#IBM-Websphere-Transformation-Extender