Hi Felipe,
It's hard to be sure what your specific problem is from the description, but the following info may help you provide information to the community which could help isolate what's going wrong...
The FileInput node itself can be directed to parse the inbound file using quite an array of different settings. Ones to take particular note of ... Click the FileInput in the message flow in Toolkit and the Properties appear at the base of the screen. To start with you will be shown Basic properties such as the input directory and file name etc. Given that you're able to at least direct the flow to an input file it sounds like those settings are likely to be ok. The next set of properties to note are on the Records and Elements tab. On this tab there is a "Record detection" dropdown, which will be "Whole File" by default, but can be changed to "Delimited" or "Parsed Record Sequence", which for your use case are the other potentially likely options you may have picked. This setting determines how the FileInput node will take data from the file and propagate it to the next node in the flow. Is it your intention to use the node settings to determine where one record ends and the next starts ?If so, then you would have the "Delimited" option selected. The other settings on that panel determine the delimiter which FileInput will use to determine where one record ends and the next starts. Once the FileInput has a defined piece of data parsed from the file using these settings, it will then use the settings on the Input Message Parsing tab properties to decide how to parse that single record ... for example you could set the Message domain to be "XMLNSC" and it would expect each record in the file to be XML format. Perhaps this is where you've set your DFDL message model to describe the layout of a single record? If so, it would be worth checking that the DFDL correctly models the content of your single record. Another approach you may have taken would be setting "Parsed Record Sequence" on the Records and Elements tab Record detection dropdown. This approach uses the nominated DFDL model to figure out where one record ends and the next starts. Once we have clarity on the FileInput settings, we could explore how the parse of your data is actually failing. It sounds like this may occur after the FileInput node when you attempt to reference a field in the message - message flows do this deliberately as a performance optimization, hence why you might be surprised where exactly in the flow the exception is being thrown and then caught. There are other ways you can turn on validation in the FileInput node if you want to avoid this, but for now, it would be helpful to paste the full exception in to this thread.
The full exception might help us explain the "Fail to put" error ... this sounds suspiciously like you may be trying to put an output message to a queue further down the flow and perhaps that's where you have the "real" issue?
I also note that you are using IIBv10 ... In both ACEv11 and ACEv12 the Tutorials Gallery has quite a few really handy tutorials for describing these kind of features with worked examples you can import, so that might also be worth looking in to in order to help your learning as you go.
Cheers,
Ben
------------------------------
Ben Thompson
IBM UK
------------------------------
Original Message:
Sent: Fri May 28, 2021 04:32 PM
From: Felipe Bandeira
Subject: IIB v10 - Problem parsing records from CSV
Hello,
I'm new to this IIB world, and I'm having a weird problem with the File Input node that I can't find a logical reason, I hope someone here can help me out.
The project is pretty simple, a csv file is dropped inside a certain folder, which is read by the File Input node.
There's no connection on the Failure exit point, but on the Catch is connected to a Compute node, where the error is logged and then logged in a separate folder.
The problem I'm facing is that I'm getting a "Failed to put message" error whenever I try to read a csv file. The log also shows the line, but there's something wrong with it, I noticed that all the decimal numbers appear 'quoted' like in this piece of the log:
;"131,12";18;"23,6";
In the csv file, this piece is written like this:
;131,12;18;23,6;
Every time I put a csv the log shows a different record number, I checked the csv structure and everything seems to be ok. I tried to import after converting the file to different encodings (UTF8, UTF8-BOM and ANSI), but got the same error on all of them.
There's no validation configured, the fields are configured through a DFDL schema. I really have no idea of what else could be checked.
Any help or hint is very appreciated.
------------------------------
Felipe Bandeira
------------------------------