Originally posted by: SystemAdmin

Hi there
Has anyone experienced problems with the FTP_adapter where the file does not get delivered to the receiving system eventhough the tracefile reports the transfer as being succesful ?
It does not happen very often. Right now maybe once every 500 transfers.

We run version 6.7.1 on windows 2000 SP 4 and this only occurs when we
transfer files to a native HP-UX11i ftp server.

I know the description is not very detailed but I just wanted to hear
if anyone has experienced a similar problem.

This is an example of a tracefile where the file did not get delivered:

Mercator FTP Adapter, Version 6.7(268)
Copyright © 1997-2000, TSI International Software, DBA Mercator Software
Built for WIN32 - INTEL on Jan 17 2003 at 18:25:49 by bocabuild.
Retry Count is 0. Retry Interval is 0. OnFailure is Rollback.
Fetch Unit is 0. Card Mode is Integral.
Will process 1294 bytes of data.
Run Started at 02:34:14.038 on 04/29/04.
Options used: trace,name,url
URL ftp://user:********@XX.XX.XX.XX//PROD/speed/prod/inbound/orders/0186867.dat will be used.
Using C:\WINNT\TEMP\C5D.tmp as tempfile.
Using ftp: protocol.
Setting AIX/UNIX-specific parameters.
Wrote 1294 bytes to C:\WINNT\TEMP\C5D.tmp.
Rollback specified; returning to map engine.
Cleanup call from prior Rollback; re-initializing instance storage.
Mercator map return is 0.
Data in C:\WINNT\TEMP\C5D.tmp deleted.
Run Terminated at 02:34:14.163 on 04/29/04

Any input would be appreciated,
Thanks
Alex
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender
#DataExchange

Originally posted by: SystemAdmin

We have, what I have done is set up a test after the FTP has completed to verify that the ftp has completed sucessfully:
=PUT("ftp"," -url ftp:\\user:password@ftpsite.com\filename.txt",PACKAGE(card1))/*placing the original file, which you can use a run map (which is a better way to go)*/
=IF(SIZE(GET("ftp"," -url ftp:\\user:password@ftpsite.com\filename.txt;type=D"))=0,FAIL("FTP for filename.txt was unsuccessful"))/*Here is the test to see if the ftp had completed*/

You can actually use the D2 and if the ftp site is set up correctly, you can measure the size of the file, which is a more comprehensive test. If you really wanted it to become a hands off type of process, instead of failing the map, you could set up to do the FTP again.

The only thing that I can suggest it make sure that the mercator.ini, the idleFTP is set to a really small number, so that the ftp session does not time out.
#IBM-Websphere-Transformation-Extender
#IBMSterlingTransformationExtender
#DataExchange

Originally posted by: SystemAdmin

You could also send the FTP command to a RUN map. If the 'on-failure' action of the map that you are RUNning is set to commit, the actual FTP failures DO get reported back to the parent map.

This is the best way to capture FTP problems.
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender
#DataExchange

Originally posted by: SystemAdmin

Hello Eric
Thanks for your reply.
It sounds like I have to implement a similar FTP test to verify a
successful completion.

I must admit that I know very little about the settings in the Mercator.ini
file. This is the setting in our file :
;IdleFTP=0
Which I guess means it is not set at all.
Do you think this might be a problem ?

Thanks again and have a nice day.
Alex
#DataExchange
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender

Originally posted by: SystemAdmin

You could also send the FTP command to a RUN map.

Paul, what do you mean "send the FTP command to a RUN map"? Can you provide an example of the RUN command? Where and how is the FTP effected? In an adapter command or a map rule?
#IBM-Websphere-Transformation-Extender
#DataExchange
#IBMSterlingTransformationExtender

Originally posted by: SystemAdmin

Hi there
The FTP_PUT map is triggered by a message on a MQ queue.
The FTP_PUT map uses a put command to transfer the file.
I do not get a function failure in the MMC.
The put command is inside a valid-fail statement.
The tracefile does not report an error.

Tarun if I understand you correctly you suggest that I create a map
that gets the message from the queue and then run another map that
puts the file ?
#IBMSterlingTransformationExtender
#DataExchange
#IBM-Websphere-Transformation-Extender

Originally posted by: SystemAdmin

;IdleFTP=0

This means that if you look in the Management console, you will see an idle ftp connection, that will never be terminated. The ftp server will eventually terminate the ftp session, but the event server is oblivious to that fact, so when it needs to do another transfer, it will try to reuse this connection, which will fail, and it is haphazardly reported, so changing the setting from ;IdleFTP=0 to IdleFTP=1; means that after 1 second of inactivity, it will terminate the connection, and create a new connection.

(You have to restart the event server service for this type of changes to be realized by the event server)
#DataExchange
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender

Originally posted by: SystemAdmin

Instead of using:

=PUT("ftp"," -url ftp:\\user:password@ftpsite.com\filename.txt",PACKAGE(card1))

...use:

=RUN("Blob.mmc",ECHOIN(1,card1)+" -OAFTP1 '-url ftp:\\user:password@ftpsite.com\filename.txt' " )

Blob.mmc is a simple blob-to-blob typetree based map that has the output card 'on-failure' setting set to commit.

If you wrap the RUN in a valid, you can FAIL the parent map on ANY ftp failure:

=VALID(RUN("Blob.mmc",ECHOIN(1,card1)" -OAFTP1 '-url ftp:\\user:password@ftpsite.com\filename.txt' " ),FAIL(""))

...or you could use an IF test on the result of this RUN function (test for non-zero return code) to have a wait and retry system.

Hope that this helps.
#IBM-Websphere-Transformation-Extender
#IBMSterlingTransformationExtender
#DataExchange

Originally posted by: SystemAdmin

This is the problem reported at the beginning, the adapter returned a code 0 back to the map:
...
Cleanup call from prior Rollback; re-initializing instance storage.
Mercator map return is 0.
Data in C:\WINNT\TEMP\C5D.tmp deleted.
...

Which means that even if a run map i:b196ce0f6awas[/i:b196ce0f6a] used, the map would have reported a successful transmission. So any tracking that would be reported in an audit log (-aecpm command for example), would not have been captured in this instance.
If you put the validation piece in after the initial PUT, whether it is invoked by a PUT or a RUN, it has the same effect.
#IBM-Websphere-Transformation-Extender
#DataExchange
#IBMSterlingTransformationExtender

Originally posted by: SystemAdmin

Hi
This is just to let you know that since we made the changes suggested by Paul Brett we have not had any problems.
The FTP adapter still does not deliver all files but errors are being reported
by the map like they should be.

Thanks again.

Alex
#DataExchange
#IBM-Websphere-Transformation-Extender
#IBMSterlingTransformationExtender

Originally posted by: SystemAdmin

We found the ftp adapter in 6.7.1 to be so problematic that we discontinuted using it altogether and use unix shell scripts to do all of our ftps. We have an ftp directory and an ftp router map which triggers whenever a file lands.
#DataExchange
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender

Originally posted by: SystemAdmin

Hello dbutner! Have been looking for a way to get around the inconsistent FTP adapter here.

Would it be possible for you to post or send a sample shell script and the router map. Can take it offline it that works. Please post a private message and can work with you from there if it is possible for you to send a sample script and map.

Thanks!
#IBM-Websphere-Transformation-Extender
#DataExchange
#IBMSterlingTransformationExtender